I'm internationalizing a Pinax website and I've encountered this piece of code in a template:

<input type="submit" value="{% trans "invite" %}"/>
{% blocktrans %}{{ other_user }} to be a friend.{% endblocktrans %}

The message ids for this code would be two separate blocks: "invite" and " %{other_user}s to be a friend". Both offer very little in terms of context and make the translators job difficult. Correct, in my point of view, would be the more convoluted form of:

{% blocktrans %}
<input type="submit" value="invite"/>
{{ other_user }} to be a friend.
{%  endblocktrans %}

This implies that the translators know enough HTML to notice that the value attribute needs to be translated, but the end result is a lot more flexible and provides real context to them.

TL;DR: don't split paragraphs into separate translation units. It's a NO-NO.

UPDATE: I have found what is probably the worst example of how to create a translatable template. Remember, don't assume the English language resembles anything like another language.

{% trans "edited by user" %} {{ obj.editor.username }} {% trans "at"%} {{ obj.modified|date:"H:i" }}

This should be done this way:

{% blocktrans with obj.editor.username as editor_username and obj.modified|date:"H:i" as obj_modified
edited by user {{ editor_username }} at {{ obj_modified }}
{% endblocktrans %}

I wonder what possible explanation there is for the behaviour of the date template filter.

Uses the same format as PHP's date() function (http://php.net/date) with some custom extensions.

I understand where Django comes from, but I think this sort of things should be more aligned with the rest of the Python world.

I fail to see how

{% block %} 
...
{% endblock %}

is in any way better or "less scary" then, let's say

<dj:block>
...
</dj:block>

Yet another rant, this time triggered by the error I got when writing this piece of code:

{% blocktrans with offer.offerer.username as offerer_username 
               and offer.offered_time|date as offerer_date %}
...
{% endblocktrans %}

One more thing to grudge about is that vim, even with djangohtml syntax type installed, is not very knowledgeble about how to format the template file (it treats the tags as regular piece of text). Probably this could be fixed, so I shouldn't complain about this too much.

I think Django templates are compiled to python code, so it's natural that they're treated in an imperative, dumb way, but that's not the only way of doing things. For example, Chameleon is another templating library that compiles its templates as python code, has no problem working with an XML based templating language frontends.

I have many grudges against the django templating language and its templates (in short, I hate them), so I'm gathering evidence to support what my "spider sense" tells me. Today the template tag system goes under fire.

Given the following template fragment:

{% load i18n %}
{% load avatar_tags %}
{% load voting_tags %}
{% load pagination_tags %}
{% load extra_voting_tags %}
{% load in_filter %}
{% load extra_tagging_tags %}
{% load sorting_tags %}

Which one is responsible for the following "anchor" tag?

{% anchor "hotness" "reddit-like hotness" %}

That's the equivalent of diving into a python module, with lots of "from X import *" at the top. Where do you find the definition of a symbol? At least, if it were Python, I could do a tag search in Vim, or a "go to definition" in Eclipse. If this practice is frowned upon in the rest of the Python world, why are so many programmers praising the Django templating system? Am I the only mad man here? My problems with this tag is that it doesn't translate the content, so I'll need to grep for its source and change it.

The template tags in Django are about extending the templating language, as to provide the programmer with new and specialized ways to interact with the template and its environment. The reason for this "tag inflation" is that the django templating language, for all its richness (by tags and filters numbers, I mean), is really limited. Python expressions are not allowed, and for every imaginable use case, there needs to be a tag, specialized or not.

How would Zope 3 solve, for example, a problem similar to the one the "anchor" tag handles? Well, rendering a special link for a content item could be as easy as

<a href="" tal:replace="someobject/@@hotness_link"><img src="hotness.gif" /></a>

Is this better? I think so. I'm editing HTML, and the <a /> tag is way better in expressing what the end result will be, compared to a simple {% anchor %} tag. Even more, the <img /> tag inside is purely cosmetic, just to cue the viewer of what the final result will be. The entire <a /> tag, with its content, will be replaced by whatever result is rendered calling the the someobject/@@hotness_link view. Finding the source of the hotness_link view is easily introspectable TTW using a debug tool such as lovely.skinbrowser.

The ZPT templates from Zope 3 can also give you a mechanism where you can add new expression types, but there's just one or two packages in the wild that define new expression types. Now compare this to the regular Django projects, were defining new tags is something that almost all projects do.

In conclusion, even though Django templates are much more imperative then ZPT, which are very declarative, they don't achieve the power and simplicity that they strive for.

Yet another angry rant, caused, of course, by using Django in anger. Nothing wrong with using something in anger, that's the real way I learn something. Zope 3 even has an online book on how to use it in anger.

That said, Django's makemessages administrative command sucks by being way too inflexible to anything but the ideal Django development environment. My environment looks like this: I have a project based on Pinax, which I'm developing and deploying using zc.buildout. My source code sits in src, where I have several packages. I also have a "localsettings.py" module located in the root of the buildout, because I don't want to have it inside the src folder. Pinax is located in parts/Pinax, and it's actually a git checkout, based on my own fork of Pinax. Pinax doesn't have translations at the moment (I think I saw a ticket in its tracker about reintroducing a translation package), so I'm on my own here with regards to translation.

With this setup, it is close to impossible for me to generate anything useful without a lot of hacking and swearing. Makemessages insists on being run from inside a Django project, and when I did that, it complains about missing localsettings module. Pointing the root of the buildout as pythonpath didn't do anything. A good thing that I have already extracted messages from the templates, before switching to the buildout project structure.

Some solutions that I have found:

• I can extract messages from the Pinax python modules using this homegrown script:
  

PYFILES=/tmp/pyfiles
PINAX=parts/Pinax/pinax/ 
BASE=/home/tibi/work/ProjectBuildout/src/project/locale/ro/LC_MESSAGES/
POTFILE_PYTHON=$BASE/python.pot
POTFILE_TEMPLATES=$BASE/templates.pot
POTFILE=$BASE/django.pot
POFILE=$BASE/django.po

#extract messages from python code
find $PINAX | grep ".*py$" > /tmp/pyfiles
touch $POTFILE_PYTHON
xgettext -j -L python -d django -f $PYFILES -o $POTFILE_PYTHON

#merge the templates + python messages into one pot file
msgcat -o $POTFILE $POTFILE_TEMPLATES $POTFILE_PYTHON

#merge the potfile with the po file
msgmerge -U -N $POFILE $POTFILE

• I have copied all the templates from pinax and its associated applications inside a template folder in my project. Now I can generate the po file, from my src/project folder, with

../../bin/py ./../../manage.py makemessages -e .py -e .html -l ro

Of course, I can't run this over the other apps and packages in my src/ folder to extract messages from the python modules, so I am forced to adjust the first script to take those folders into consideration.

I managed to figure out how to beat one awkward piece of workflow when starting new project: I always start hacking on a project, then I notice that I haven't been working on a svn checkout (it's the chicken or the egg problem). So I need to do a svn import, delete my copy of the project, checkout the svn version and start hacking at my project again. This workaround is agravated, though, when working with buildouts. I can't svn import the entire folder, because lots of generated folders and files will end up in subversion, which I don't want. Cleaning my original, running import and regenerate the buildout is a workflow killer.

The solution is simple, and goes like this: instead of importing the entire buildout folder, I can run

svn import . http://my/subversion/path/ --depth empty -m "initial import"

or I can just create a folder in my repository with svn mkdir

Next, checkout the empty folder from the repository into my buildout

svn co http://my/subversion/path/ .

Now I can cherry-pick whatever I want committed, with svn add. Problem solved!

A bug creeped in one of my Django 1.1 projects that is in beta-testing right now: the Pinax wiki app looks for a REMOTE_ADDR value in request.META, which was not set in my environment. My environment is a pretty standard (as far as this setup goes) nginx + fcgi (flup on the django side) + django. Further work on this issue revealed, step by step, that:

• REMOTE_ADDR needs to be somehow set by a Django middleware, based on an http header,
  
• so I've added django.middleware.http.SetRemoteAddrFromForwardedFor to the list of loaded middleware
• but that middleware is deprecated in Django 1.1 and does nothing, so I rewrote this middleware based on Chapter 15 of Django Book
• this new middleware did its job, but for some reason flup stripped that header from the request and the proper values never got to Django
• this made me look for a replacement for flup, which I found in django-cpserver.

This package adds a new admin command, 'runcpserver', which replaces the default development server by something more appropriate to production. The word on the 'net-streets' is that it runs well and does its job, so I'll be using it for the time being. The recommended solution is to run apache+mod_wsgi and proxy that to nginx, but right now I don't want the extra administrative overhead that Apache represents. If I can't get enough "juice" out of one instance of cpserver, I'll just add an extra instance and balance them with nginx or haproxy.

Probably this is documented somewhere in the Django docs, but, I mean, who has time to read the docs? :-) In the urls.py file for my project I didn't do a

from django.conf.urls.defaults import *

because that's just bad style. Instead I've just imported what I needed: patterns, include and url. Later on I got this error in my email:

Traceback (most recent call last):

  File "/home/zope/djangoprojects/lib/python2.5/site-packages/Django-1.1.1-py2.5.egg/django/core/handlers/base.py", line 118, in get_response
    callback, param_dict = resolver.resolve404()

  File "/home/zope/djangoprojects/lib/python2.5/site-packages/Django-1.1.1-py2.5.egg/django/core/urlresolvers.py", line 263, in resolve404
    return self._resolve_special('404')

  File "/home/zope/djangoprojects/lib/python2.5/site-packages/Django-1.1.1-py2.5.egg/django/core/urlresolvers.py", line 255, in _resolve_special
    callback = getattr(self.urlconf_module, 'handler%s' % view_type)

AttributeError: 'module' object has no attribute 'handler404'

Just looking at the error I could determine the cause: my urlconf module (urls.py) didn't have a handler404 view defined, so I just gave up and replaced my specific imports with the asterisk import.

On a side note, what's up with Django logging? By default, the only way to log errors in production environments is by email. Really!? If I would want emails, I'd setup the syslog to email me those entries. I already have the logwatch emailing me stuff that happens on my server. One step above is django-db-log, which I haven't tried yet, but which seems to be logging errors to the db. Word is that real logging, using the logging python module, will be introduced in Django 1.2.

I'm not sure how I feel about this... In some ways, I think it's very positive that a relative newcomer can just jump in and release an opensource project that can be taken and reused by others. Did Django allowed this? That's very positively PHP-like...

I'm talking about a glimpse at some code that I caught while helping my brother integrate the Diamandas forum to a site he's doing. It stunned me that it has the marks of a developer that is not very experienced with the language or the framework, but it's moving at a speed that is too great to stop and fix or check what he's doing.

This:

pr = False        
if forum.use_prefixes:                
    p = Prefix.objects.filter(forums=forum)                
    if len(p) > 0:                        
        pr = []                        
        for i in p:                                
            pr.append(i)

is a fragment from an apparently nice Django forum add-on. First, it could easily be rewritten as:

pr = []
if forum.use_prefixes:
    pr = Prefix.objects.filter(forums=forum)

Suppose you don't trust the Django queryset results to really behave like a list (that could happen for some legitimate reasons, but I'm sure that's not the case here). Then you could write the last line as:

pr = list(Prefix.objects.filter(forums=forum))

Of course, no more lazy loading of objects and a bit more memory consumption. More, about the looks of this code:

• "len(p) > 0" can be rewritten as "if p"
• why define the "pr" as a False value instead of an empty list, which will become at one point?
• why use the "i" name for a prefix object type, when this is typically used as an integer counter?

Further on that page there's

tp = TopicPrefix(topic=new_place)
tp.save()
tp.prefix=pr
tp.save()

I haven't tried this, but I'm pretty sure that only one call to save() is needed. Probably more odd things could be found, but I haven't tried to look further - I already found the source of my brother's problems.

These superficial aspects are the ones that make me also question the architectural choices that were made for this add-on, the overall quality of the code that might impact performance, etc. How can I trust an application that has bits of code like this? I'm pretty sure that this particular developer (I haven't bothered tracking exactly who it was) has a bright future in the Django community and will probably polish his Python skills to generate good code. Overall, the Python and Django communities will also benefit from an influx of new developers. But I'm left feeling insecure about my decision to write code for the Django platform. I'm mostly (or I want to be) an integrator and I depend on good third-party addons, which Django seems to have a lot. But if two out of three projects that I have tried (Satchmo and Diamandas) left me confused, how else should I feel? Ironically, the one project that I have really liked (LFS) came from a Zope developer... So maybe we need a bigger Zope > Django migration phenomena.

UPDATE: well, it looks like Django isn't free of promoting (IMHO) stupid solutions, too. Check out this piece of code from the Django admin documentation:

return self.birthday.strftime('%Y')[:3] == '195'

This smells a lot like stupid code that beginner PHP programmers would write. When you're dealing with a number (the year), why convert it to a string? The code itself yields correct results, but the method used feels stupid. I'd rather see something like:

return 1949 < self.birthday.year < 1960

Another one, not a bug, but a design decision from Django, which forces third party projects like Pinax to mangle with the syspath just to get django happy:

  File "/home/tibi/work/lib/python2.5/site-packages/django/contrib/auth/models.py", line 283, in get_profile
    app_label, model_name = settings.AUTH_PROFILE_MODULE.split('.')

What's my AUTH_PROFILE_MODULE set to?

AUTH_PROFILE_MODULE = 'pinax.apps.basic_profiles.Profile'

Actually, thinking more about this: "pinax.apps.basic_profiles.Profile" is not even a real path, Profile is a class in "models.py", so it might be valid for Django to request its special rules there. Valid, but non-intuitive and non-standard for the rest of the Python world.

Another weird stuff, this time in Pinax. Looks like really young code, which hasn't been subjected to a thorough code review yet: friends_app/views.py. This is file has some weird indentation problems (some lines get indented more then they need to be). What about this piece of code:

authsub_token = request.session.get('authsub_token')
del request.session['authsub_token']

If 'authsub_token' is not in the session, (as it was my case, which made me discover this), you'll get an error on the second line. Of course, this bug highlights the more important problem, of the missing authsub_token, but what about the careless programming?

I'm not a Django developer, but I have a project that involves deploying a LFS site, and it took me a bit of time and effort to understand how to customize its templates and resources. This all may seem obvious to the any Django developer, but I'm not, so I think this info needs to be out there.

As a side note, at first I have reviewed Satchmo, as it seemed to have more traction in the Django community (being a Zope/Plone developer, I am more aware of the previous work done by the LFS developer, the EasyShop). The Satchmo installation, for the un-initiated, seems to be a nightmare of many different settings that needed to be tweaked, skeletons that don't work OOTB, at least in the way I have installed them. On top of all this, it seems to think that I also need to develop some sort of project that will integrate with the Satchmo shop (but this may be due to Django). What happened to treating your product as the end product? After a bit of effort, I had one instance running and ready to review it. Several days, in another Linux virtual machine (I've switched from VMWare to VirtualBox), I've tried to install Satchmo again, this time using djangorecipe. I didn't have the same patience, though.

A short look at LFS convinced me to at least give it a try: there is a buildout that can be installed easily (although there were a few dependencies that I had to install), the "backend" is polished compared to the default django admin views used by Satchmo, and, while it is simpler, less featured, than Satchmo, it does everything I need.

Onward to customizing the LFS: my goal is to be able to change templates (for example, the base layout) and publish a new logo.

I've created a new folder "mytheme" inside the buildout, with two subfolders: "templates" and "static". The path to the templates folder needs to be inserted in the "TEMPLATES_DIRS" variables from settings.py. To customize the base template, for example, I've created a "lfs/base.html" file inside the "templates" folder. Its content can be copied from the original in the lfs_theme folder. I've added a new line in the header section, pointing to a new CSS file that will customize the base LFS stylesheet.

<link rel="stylesheet" type="text/css" href="{{ MEDIA_URL }}mytheme/css/custom.css" />

As a side note, this templates folder can be used for overriding any other template: templates in Django are usually denoted by a relative path, and you need only to replicate this path inside our overriding templates folder.

Next, overriding and creating new resources: I have created a "static" folder inside "mytheme" and I've linked to it inside the lfs_project/media folder. Inside the "mytheme" folder I can now host images and files, in 2 new "css" and "img" subfolders. Back in my customized base.html, I have changed the logo path:

<img class="logo" src="{{ MEDIA_URL }}mytheme/img/logo.jpg" alt="Logo" />

That's about it: I got a custom CSS, the ability to customize templates and to point to new images. I'm happy for now.

As I have said in the beginning, this is all basic stuff. No harm done in pointing to the obvious though, as there seem to be many new developers flocking to Django, all with different level of expertise.

I'm starting to grow a strong dislike to it, enough to steer me off Python to JVM, with Jython or Scala. Case in point: its authors considered that it's appropriate to overload the Paragraph class from the platypus module, to make it accept a form of "xml". There's no switch to disable this behavior and its xml handling is something straight out of the 90's: no namespaces, no validation. What happens for example, when you're trying to make a paragraph with the text:

 "Some html documents contain <img> tags"

That's right, you'll encounter an error. The "<img>" fragment is interpreted as an img tag for the Paragraph, which will then complain that it's missing a src attribute. I wonder how can ReportLab be promoted as an enterprise solution and still display this behavior. The easiest solution, in my opinion, would be to create a new class that accepts properly formated xml, and RL's special tags need to be isolated in a new namespace. Fortunately, for the time being, I can do a replace for the "<" character with the &lt; entity.

Somebody at Google has screwed up in a laughable manner. Webmaster Tools just got a new feature: the ability to tell Google that you have moved a site to a new domain. I'm in the same situation with one site that I manage: www.caleidoscop.org.ro has been moved recently to www.caleidoscop.org. Although I have done my best (I still need to persuade nginx to generate 301 redirects instead of 302) to ease this transition and absolutely everything pointing to the .org.ro  is redirected to the .org address, Google is really slow in picking the change and the Webmaster Tools show links from caleidoscop.org.ro as the main source of links to the new .org domain. Unfortunately, as things stand now, it's impossible for me to use this feature: Google doesn't recognize *.org.ro as being a "root level domain", as can be seen in the next two screenshots. I wonder, org.uk domains have the same problems?

First, the bug

And this is how it's supposed to look

I've opened a new blog, at http://vimzone.pixelblaster.ro It runs on the latest Plone 3, with the NottreDamme skin and Scrawl as the blogging product. I'll probably install also the QuillsEnabled extension. There's not a lot of content on it, although it's already almost 2 months old. It's a place for me to scribble things I know about VIM and use or discover.

Ever since Plone 2.5 stopped being the hot new thing in the Plone world, the state of content internationalization support in Plone has been declining. Partly due to the difficulty of extending Zope 2 products, LinguaPlone has always been a hack, but at least back then it worked. These days I'm not heavily involved with Plone, but whenever I need to add LinguaPlone to a Plone 3 site, I cringe and weep, especially that those projects never allocate time for me to fix the obvious problems. Some of those problems, in current Plone 3.3 rc2 are:

• the flags are no longer grayed to indicate that a certain translation is unavailable
• the switchLanguage script doesn't work anymore, it won't redirect you to the translated content; the LanguageTool is broken (won't take VHM into consideration) and an update was released, but the versions.cfg file on plone.org hasn't been updated yet, so you need to add a [versions] section, with the correct specification for the Products.PloneLanguageTool
• the changeLanguage script was removed, in favor of doing a direct translation. Wops, now it's possible to translate into neutral, which gets you fast into an ugly traceback that you won't be able to get rid of.
• the language neutral behaviour for fields is broken. Example: I have a content type with the neutral fields, when translating a content item the content of those fields is erased. This is something that I'll have to fix.
  

On the upside, the translation management mechanism (after some hickups) has improved.

I don't have access to the Plone subversion. I already had some discussions with two of the developers that have done the latest changes to LP, but they either didn't understand the problems or didn't have time to do the fixes themselves. To make it easier for myself and stimulate my interest in the LP development, I have copied the latest LP trunk to Collective, in a location called Products.LinguaPlone-Unofficial. This version also has a check for the case when there's a translation into Neutral, so it should fix the error detailed above.

Update:

Another problems solved: I've wanted to be able to switch languages based on the subdomain (ro.site.com, en.site.com), which should be supported by the PloneLanguageTool, but didn't work for me. Digging down, I found the problem: plone.i18n.locales.cctld has a list that associates TLDs to languages, but most of the TLDs don't have a language associated to them! This issue should be at least highlighted in the documentation for PloneLanguageTool.

One more weird problem, which caused me to blame LP for even more problems than it has at fault: upgrading as site from Plone 3.2 to 3.3 rc2 broke the LP language selector, by making the default Plone one override the LP one (where it should be the other way). I was able to solve the problem by customizing the template for the LP language selector (with no change)

I'm checking out a large repository and svn keeps getting blocked, for some reason (bad network, maybe). When running update again, it complains that the checkout should be cleaned, but the cleanup operation doesn't succeed, with something like this:

tibi@xps:~/svn/vs/data$ svn cleanup                    
svn: In directory '.'                                                
svn: Error processing command 'modify-wcprop' in '.'                 
svn: 'setup.config' is not under version control

Searching on then net for a solution didn't yield anything, so the one that I have found was to empty the .svn/log file (by default is read only) and then to run svn up again.

I've upgraded CacheFu (Products.CacheSetup) to the latest 1.2 for some of the websites that I manage, in a Plone 2.5 cluster. One of them had a problem that I haven't been able to track previously, due to limited time: on a reinstall of CacheSetup, due to product upgrades, CacheFu couldn't be installed anymore. The traceback was something like:

this product has already been installed without Quickinstaller!failed:
Traceback (most recent call last):

  File "/home/zope/z29/Products/CMFQuickInstallerTool/QuickInstallerTool.py", line 330, in installProduct

  File "/home/zope/p25/parts/zope2/lib/python/Products/ExternalMethod/ExternalMethod.py", line 225, in __call__
    try: return f(*args, **kw)

  File "/home/zope/p25/eggs/Products.CacheSetup-1.2-py2.4.egg/Products/CacheSetup/Extensions/Install.py", line 35, in install
    policy_utils.addCachePolicies(self, out)

  File "/home/zope/p25/eggs/Products.CacheSetup-1.2-py2.4.egg/Products/CacheSetup/Extensions/policy_utils.py", line 72, in addCachePolicies
    p.addCacheRules(rules)

  File "/home/zope/p25/eggs/Products.CacheSetup-1.2-py2.4.egg/Products/CacheSetup/Extensions/policy_2.py", line 13, in addCacheRules
    rules.invokeFactory(id=id, type_name='PolicyHTTPCacheManagerCacheRule')

  File "/home/zope/z29/Products/CMFCore/PortalFolder.py", line 408, in invokeFactory

  File "/home/zope/z29/Products/CMFCore/TypesTool.py", line 934, in constructContent

  File "/home/zope/z29/Products/CMFCore/TypesTool.py", line 343, in constructInstance

  File "/home/zope/z29/Products/CMFCore/TypesTool.py", line 574, in _constructInstance

  File "", line 6, in addPolicyHTTPCacheManagerCacheRule

  File "/home/zope/p25/parts/zope2/lib/python/OFS/ObjectManager.py", line 301, in _setObject
    v = self._checkId(id)

  File "/home/zope/z29/Products/CMFCore/Skinnable.py", line 223, in _checkId

  File "/home/zope/p25/parts/zope2/lib/python/OFS/ObjectManager.py", line 95, in checkValidId
    raise BadRequest, (

BadRequest: The id "httpcache" is invalid - it is already in use.

One other weird thing are the paths in this traceback: /home/zope/z29 doesn't exist anymore, the database was moved from a different server. I think it's related to the persistent product entries in the Control_Panels, which can be cleared. Not a big problem. In the log, there was also an entry related to this traceback:

2009-02-28 17:01:18 CRITICAL txn.-1223480432 A storage error occurred during the second phase of the two-phase commit.  Resources may be in an inconsistent state

Now, the solution is really simple, but I needed to debug the policy_2 module to find this: 

rules.invokeFactory(id=id, type_name='PolicyHTTPCacheManagerCacheRule')

didn't work because there was a document in the root called "rules". Nobody expects the spamish Acquisition! And I didn't either... Lesson? Zope 2 was designed to be too smart for its own good, thus violating the KISS principle. Still love it, though.

While testing software to play music from a computer on my network, I have discovered Elisa, which is a wonderful piece of software built in Python and zope.interface. The package manager helpfully installed a python-zopeinterface package, which turned out to break one of the apps I've been working on (one of the packages that is used depends on a more recent version of zope.interface and breaks with a missing object import). Adding an explicit dependency on zope.interface>=3.5 didn't help either. The egg was installed and a reference to it was inserted in the generated script wrapper for the buildout's bin folder, but the system zope.interface was found.

The solution that I have found was to make sure the zope.interface dependency is listed in the last position in the install_requires section of setup.py. This has the effect of placing the zope.interface egg path first in the generated script, and thus solving the problem.

UPDATE: On another project I'm working on, this solution didn't work. Buildout would complain about a version conflict and would drop the building process. The solution was to setup a separate virtualenv bootstrapped with --no-site-packages and use the python from that virtualenv to bootstrap the buildout environment. I think it's a bug in zc.buildout, as it should have obeyed the versions section of buildout, plus the explicit dependency in install_requires of my package setup.py

One of my long-running projects involves a workflow where content is produced in a Plone site, with the data later extracted and processed in various ways (including scripting Scribus to layout this data in a book). Initially the site where the content was produced wasn't protected, so I could run a simple urllib script to download the content and process it using lxml. A recent change in the workflow security settings meant this script didn't work anymore and I had to remember how to login into a Plone site using urllib2. Some google searches found me nothing, but I remembered that the zope.testbrowser can be easily used to run a programatical browsing session, complete with cookies support. But trying to install zope.testbrowser standalone in a buildout didn't lend to too much success, due to some dependency problems (and even after I covered for those dependencies, it still broke somewhere in zope.app.testing).

The solution was to use just the mechanize package, on top of which zope.testbrowser is built. mechanize has a slightly different API (more modern) and doesn't do so much handholding as zope.testbrowser, but I only need to process one form. In the end my script looks something like this (the asxmllist page is just an xml page that returns a list of urls to the entities that I want to process):

import lxml.etree
import os
import os.path
import urllib
import mechanize

loginurl = "http://example.com/login_form"
listurl = "http://example.com/asxmllist"

def run():
    curdir = os.getcwd()
    datadir = os.path.join(curdir, 'data')
    if not os.path.exists(datadir):
        os.makedirs(datadir)
    
    b = mechanize.Browser()
    b.open(loginurl)
    b.select_form(nr=1)
    b['__ac_name'] = "username"
    b['__ac_password'] = "password"
    b.submit()
    b.open(listurl)
    etree = lxml.etree.parse(b.response())
    
    for entry in etree.xpath('//entry'):
        url = entry.get('url')
        print "Processing " + url
        e = lxml.etree.parse(b.open(url + '/asxml'))
        id = e.find('id').text
        print "Got entry " + id
        fpath = os.path.join(datadir, id + '.xml')
        f = open(fpath, 'w')
        xml = lxml.etree.tostring(e)
        f.write(xml)
        f.close()
        print "Saved " + fpath

if __name__ == "__main__":
    run()

I admit, I didn't knew this until now. In the following construction:

<div tal:content="somedict/keyname/someattr" />

"keyname" is taken as a string, it's the literal name of the key for the somedict mapping. To use a variable instead of the literal value of the key name, I used to do:

<div tal:content="python somedict[key].someattr" />

Browsing through the zope.app.catalog code, I saw that there's actually a way to use the TALES syntax:

<div tal:content="somedict/?key/someattr" />

I'm not sure that this works with TTW code in Zope 2 (I expect that it works with browser views), so I'll just have to try this next time I have the chance.

While developing an application and writing some migration code (using zope.app.generations), I had the need to reset the generation number recorded in the database for my application to a version lower than the current generation number (because my generation code didn't run properly and I didn't want to create bogus generation files). To solve this issue, in a pdb prompt I had to run:

(Pdb) db = self.request.publication.db
(Pdb) conn = db.open()
(Pdb) conn.root()['zope.app.generations']['myapp.generations'] = 0
(Pdb) import transaction
(Pdb) transaction.commit()
(Pdb) c

Not much to it, and this info can be easily obtained by reading the zope.app.generations source code.