I've upgraded CacheFu (Products.CacheSetup) to the latest 1.2 for some of the websites that I manage, in a Plone 2.5 cluster. One of them had a problem that I haven't been able to track previously, due to limited time: on a reinstall of CacheSetup, due to product upgrades, CacheFu couldn't be installed anymore. The traceback was something like:

this product has already been installed without Quickinstaller!failed:
Traceback (most recent call last):

  File "/home/zope/z29/Products/CMFQuickInstallerTool/QuickInstallerTool.py", line 330, in installProduct

  File "/home/zope/p25/parts/zope2/lib/python/Products/ExternalMethod/ExternalMethod.py", line 225, in __call__
    try: return f(*args, **kw)

  File "/home/zope/p25/eggs/Products.CacheSetup-1.2-py2.4.egg/Products/CacheSetup/Extensions/Install.py", line 35, in install
    policy_utils.addCachePolicies(self, out)

  File "/home/zope/p25/eggs/Products.CacheSetup-1.2-py2.4.egg/Products/CacheSetup/Extensions/policy_utils.py", line 72, in addCachePolicies
    p.addCacheRules(rules)

  File "/home/zope/p25/eggs/Products.CacheSetup-1.2-py2.4.egg/Products/CacheSetup/Extensions/policy_2.py", line 13, in addCacheRules
    rules.invokeFactory(id=id, type_name='PolicyHTTPCacheManagerCacheRule')

  File "/home/zope/z29/Products/CMFCore/PortalFolder.py", line 408, in invokeFactory

  File "/home/zope/z29/Products/CMFCore/TypesTool.py", line 934, in constructContent

  File "/home/zope/z29/Products/CMFCore/TypesTool.py", line 343, in constructInstance

  File "/home/zope/z29/Products/CMFCore/TypesTool.py", line 574, in _constructInstance

  File "", line 6, in addPolicyHTTPCacheManagerCacheRule

  File "/home/zope/p25/parts/zope2/lib/python/OFS/ObjectManager.py", line 301, in _setObject
    v = self._checkId(id)

  File "/home/zope/z29/Products/CMFCore/Skinnable.py", line 223, in _checkId

  File "/home/zope/p25/parts/zope2/lib/python/OFS/ObjectManager.py", line 95, in checkValidId
    raise BadRequest, (

BadRequest: The id "httpcache" is invalid - it is already in use.

One other weird thing are the paths in this traceback: /home/zope/z29 doesn't exist anymore, the database was moved from a different server. I think it's related to the persistent product entries in the Control_Panels, which can be cleared. Not a big problem. In the log, there was also an entry related to this traceback:

2009-02-28 17:01:18 CRITICAL txn.-1223480432 A storage error occurred during the second phase of the two-phase commit.  Resources may be in an inconsistent state

Now, the solution is really simple, but I needed to debug the policy_2 module to find this: 

rules.invokeFactory(id=id, type_name='PolicyHTTPCacheManagerCacheRule')

didn't work because there was a document in the root called "rules". Nobody expects the spamish Acquisition! And I didn't either... Lesson? Zope 2 was designed to be too smart for its own good, thus violating the KISS principle. Still love it, though.

While testing software to play music from a computer on my network, I have discovered Elisa, which is a wonderful piece of software built in Python and zope.interface. The package manager helpfully installed a python-zopeinterface package, which turned out to break one of the apps I've been working on (one of the packages that is used depends on a more recent version of zope.interface and breaks with a missing object import). Adding an explicit dependency on zope.interface>=3.5 didn't help either. The egg was installed and a reference to it was inserted in the generated script wrapper for the buildout's bin folder, but the system zope.interface was found.

The solution that I have found was to make sure the zope.interface dependency is listed in the last position in the install_requires section of setup.py. This has the effect of placing the zope.interface egg path first in the generated script, and thus solving the problem.

UPDATE: On another project I'm working on, this solution didn't work. Buildout would complain about a version conflict and would drop the building process. The solution was to setup a separate virtualenv bootstrapped with --no-site-packages and use the python from that virtualenv to bootstrap the buildout environment. I think it's a bug in zc.buildout, as it should have obeyed the versions section of buildout, plus the explicit dependency in install_requires of my package setup.py

One of my long-running projects involves a workflow where content is produced in a Plone site, with the data later extracted and processed in various ways (including scripting Scribus to layout this data in a book). Initially the site where the content was produced wasn't protected, so I could run a simple urllib script to download the content and process it using lxml. A recent change in the workflow security settings meant this script didn't work anymore and I had to remember how to login into a Plone site using urllib2. Some google searches found me nothing, but I remembered that the zope.testbrowser can be easily used to run a programatical browsing session, complete with cookies support. But trying to install zope.testbrowser standalone in a buildout didn't lend to too much success, due to some dependency problems (and even after I covered for those dependencies, it still broke somewhere in zope.app.testing).

The solution was to use just the mechanize package, on top of which zope.testbrowser is built. mechanize has a slightly different API (more modern) and doesn't do so much handholding as zope.testbrowser, but I only need to process one form. In the end my script looks something like this (the asxmllist page is just an xml page that returns a list of urls to the entities that I want to process):

import lxml.etree
import os
import os.path
import urllib
import mechanize

loginurl = "http://example.com/login_form"
listurl = "http://example.com/asxmllist"

def run():
    curdir = os.getcwd()
    datadir = os.path.join(curdir, 'data')
    if not os.path.exists(datadir):
        os.makedirs(datadir)
    
    b = mechanize.Browser()
    b.open(loginurl)
    b.select_form(nr=1)
    b['__ac_name'] = "username"
    b['__ac_password'] = "password"
    b.submit()
    b.open(listurl)
    etree = lxml.etree.parse(b.response())
    
    for entry in etree.xpath('//entry'):
        url = entry.get('url')
        print "Processing " + url
        e = lxml.etree.parse(b.open(url + '/asxml'))
        id = e.find('id').text
        print "Got entry " + id
        fpath = os.path.join(datadir, id + '.xml')
        f = open(fpath, 'w')
        xml = lxml.etree.tostring(e)
        f.write(xml)
        f.close()
        print "Saved " + fpath

if __name__ == "__main__":
    run()

I admit, I didn't knew this until now. In the following construction:

<div tal:content="somedict/keyname/someattr" />

"keyname" is taken as a string, it's the literal name of the key for the somedict mapping. To use a variable instead of the literal value of the key name, I used to do:

<div tal:content="python somedict[key].someattr" />

Browsing through the zope.app.catalog code, I saw that there's actually a way to use the TALES syntax:

<div tal:content="somedict/?key/someattr" />

I'm not sure that this works with TTW code in Zope 2 (I expect that it works with browser views), so I'll just have to try this next time I have the chance.

While developing an application and writing some migration code (using zope.app.generations), I had the need to reset the generation number recorded in the database for my application to a version lower than the current generation number (because my generation code didn't run properly and I didn't want to create bogus generation files). To solve this issue, in a pdb prompt I had to run:

(Pdb) db = self.request.publication.db
(Pdb) conn = db.open()
(Pdb) conn.root()['zope.app.generations']['myapp.generations'] = 0
(Pdb) import transaction
(Pdb) transaction.commit()
(Pdb) c

Not much to it, and this info can be easily obtained by reading the zope.app.generations source code.