#!/usr/bin/python

from zipfile import ZipFile
import lxml.etree
import os.path
import shutil
import sys

namespaces = {
        'u':"urn:oasis:names:tc:opendocument:xmlns:container",
        'xsi':"http://www.w3.org/2001/XMLSchema-instance",
        'opf':"http://www.idpf.org/2007/opf", 
        'dcterms':"http://purl.org/dc/terms/",
        'calibre':"http://calibre.kovidgoyal.net/2009/metadata",
        'dc':"http://purl.org/dc/elements/1.1/",
        }

def main():
    if len(sys.argv) != 2:
        print "Need a path."
        sys.exit(1)

    fpath = sys.argv[1]
    zip = ZipFile(fpath)
    meta = zip.read("META-INF/container.xml")
    e = lxml.etree.fromstring(meta)
    rootfile = e.xpath("/u:container/u:rootfiles/u:rootfile", 
            namespaces=namespaces)[0]
    path = rootfile.get('full-path')
    opf = zip.read(path)
    e = lxml.etree.fromstring(opf)
    title = e.xpath("//dc:title", namespaces=namespaces)[0].text
    author = e.xpath("//dc:creator", namespaces=namespaces)[0].text


    base = os.path.dirname(fpath)
    new_name = "%s - %s.epub" % (author, title)
    shutil.move(fpath, os.path.join(base, new_name))

if __name__ == "__main__":
    main()

(Pdb) pp type(ff).mro()
(<class 'plone.app.blob.subtypes.image.ExtensionBlobField'>,
 <class 'archetypes.schemaextender.field.TranslatableExtensionField'>,
 <class 'archetypes.schemaextender.field.BaseExtensionField'>,
 <class 'plone.app.blob.field.BlobField'>,
 <class 'Products.Archetypes.Field.ObjectField'>,
 <class 'Products.Archetypes.Field.Field'>,
 <class 'Products.Archetypes.Layer.DefaultLayerContainer'>,
 <class 'plone.app.blob.mixins.ImageFieldMixin'>,
 <class 'Products.Archetypes.Field.ImageField'>,
 <class 'Products.Archetypes.Field.FileField'>,
 <type 'ExtensionClass.Base'>,
 <type 'object'>)

 Credit goes to the original post where I found this.

While:
  Installing.
  Getting section zope2.
  Initializing section zope2.
  Installing recipe plone.recipe.zope2install.
Error: There is a version conflict.
We already have: setuptools 0.6c9

or, worse, this:

While:
  Installing.
  Getting section zope2.
  Initializing section zope2.
  Installing recipe plone.recipe.zope2install.
Error: There is a version conflict.
We already have: zc.buildout 1.5.2

Well, technically I know what happens: for example, zc.buildout is latest 2.0a1 now, but I've already installed 1.5.2 in my virtualenv (the bootstrap process failed hard, there are a tons of bugs there, I've had more failures in bootstrap then success, lately) and I had one product in my buildout which depended on zc.buildout, so it tried to pull the latest, only to get a version conflict.

I wish zc.buildout would behave more inteligently. Even specifying 

prefer-final = true

didn't do much to solve my problems. The only way to solve the problem was to add the following to the buildout.cfg file:

[buildout]
...
versions = versions

[versions]
zc.buildout =1.5.2
setuptools = 0.6c9

import cPickle

def export(self):
    pas = self.acl_users
    users = pas.source_users
    passwords = users._user_passwords
    result = dict(passwords)

    f = open('/tmp/out.blob', 'w')
    cPickle.dump(result, f)
    f.close()

    return "done"

def import_users(self):
    pas = self.acl_users
    users = pas.source_users
    f = open('/tmp/out.blob')
    res = cPickle.load(f)
    f.close()

    for uid, pwd in res.items():
        users.addUser(uid, uid, pwd)

    return "done"

In buildout.cfg, in the instance part definition, add:

zope-conf-additional =
    <environment>
        mylocation ${buildout:directory}/var/mylocation
    </environment>

Next, inside the product code I have:

from App.config import getConfiguration
import os

conf = getConfiguration()
dest = conf.environment['mylocation']
if not os.path.exists(dest):
    os.mkdir(dest)

There were 2 things that I had to research for this task: reading the global zope configuration (that's done with App.config.getConfiguration()) and the fact that you can't add arbitrary key/values in zope.conf and have to use the <environment> section.

Fortunately, Gloworm has been updated in svn trunk. To get the latest version you need to add it to sources (in buildout.cfg or develop.cfg):

[buildout]
eggs +=
     Products.Gloworm

[sources]
Products.Gloworm = svn https://weblion.psu.edu/svn/weblion/weblion/Products.Gloworm/trunk/

Next, run in the shell:

#bin/develop co Products.Gloworm
#bin/develop rb

This is what I have come up with (this code runs in a browser view):

_id = self.context.generateUniqueId("Document")
_id = self.context.invokeFactory(type_name=type_name, id=_id)
ob = self.context[_id]
ob.edit(
     description = "text...",
     subject     = ('tag1', 'tag2'),
     title       = "some title...",
 )
ob._renameAfterCreation(check_auto_id=True)
_id = ob.getId()

One thing I notice is that Plone 4 starts faster, comes with easily pluggable developer tools, and in general feels a lot more polished then any previous Plone releases. It's a good system to work with.

dolmen

Dolmen is an application development framework based on Grok and ZTK which also provides a CMS (Content Management System) out of the box. Dolmen is being made with four main objectives in mind: easily pluggable, rock solid and fast content type development, readability and speed.

dolmen.app.authentication

Users and group management in Dolmen

dolmen.app.breadcrumbs

Provides a breadcrumbs navigation for the Dolmen applications. It registers a viewlet to render the links.

dolmen.app.clipboard

Provides a useable "clipboard", that allows you to cut, copy and paste your objects.

dolmen.app.container

Is a collection of tools to work with containers in Dolmen applications.

dolmen.app.content

Provides out-of-the-box utilities for Dolmen applications content.

dolmen.app.layout

Provides ready-to-use components to get a fully functional and extensively pluggable User Interface for a Dolmen application

dolmen.app.metadatas

Forms and viewlets to edit ZopeDublinCore metadata

dolmen.app.search

Viewlets and utilities for permission-aware searching of objects in a Dolmen site.

dolmen.app.security

Roles and permissions for a Dolmen site

dolmen.app.site

The basic Dolmen objects that serve as roots of Dolmen sites

dolmen.app.viewselector

Allows basic management of alternate views

dolmen.authentication

Basic components for authentication

dolmen.beaker

Zope sessions implementation using beaker

dolmen.blob

A layer above zope.file using ZODB blobs as a storage facility. It offers a BlobFile content type and a BlobProperty property for complex schemas.

dolmen.builtins

A set of interfaces that apply to basic Python types, to better integrate them with ZCA

dolmen.content

Base classes and utilities to create content types

dolmen.field

Additional fields usable in schemas. At this moment there's just GlobalClass

dolmen.file

Allows you to manage and store files within the ZODB. It takes the core functionalities of zope.app.file, and simplifies them, using Grok for views and adapters registrations.

dolmen.forms.base

A package in charge of providing basic functionalities to work with zeam.form Forms.

dolmen.forms.crud

A package which helps developers create their C.R.U.D forms using Grok, zeam.form and dolmen.content. It provides a collection of base classes to add, edit, and access content. It innovates by providing adapters to customize the fields of a form.

dolmen.menu

Aims to provide the most flexible and explicit way to create and manage menus and their entries with Grok.

dolmen.queue

A simple layer on top of zc.async to provide queuing of tasks. Not ready?

dolmen.relations

Is a thin layer above zc.relation, allowing a simple and straightforward implementation of standalone relationships between objects.

dolmen.storage

Defines a clear high-level API to deal with pluggable storage components.

dolmen.thumbnailer

Is package specialized in Thumbnail generation. Using the dolmen.storage mechanisms, it allows a pluggable and flexible thumbnail storage.

dolmen.widget.file

A package that walks hand-in-hand with dolmen.file. It provides a useable and pluggable way to render the dolmen.file.FileField in a zeam.form Form.

dolmen.widget.image

A thin layer above dolmen.widget.file providing a widget suitable to fields implementing IImageField. It adds, thanks to dolmen.thumbnailer a preview of the uploaded image in both input and display mode.

dolmen.widget.tinymce

A package that provides a useable and pluggable way to render a text field as a WYSIWG editor in a zeam.form Form.

dolmen.workflow

Nothing here

megrok.icon

Allows registration of icons and associating them with content types

megrok.resourcemerger

Allows concatanation and packing of browser resources (css and js)

menhir.contenttype.document

An example document content type

menhir.contenttype.file

An example file content type

menhir.contenttype.folder

An example folder content type

menhir.contenttype.image

An example image content type

menhir.contenttype.photoalbum

An example photoalbum content type

menhir.contenttype.rstdocument

An example rstdocument content type

menhir.contenttype.user

An example user content type

menhir.library.tablesorter

Registers a jquery based library for HTML tables sorting

menhir.simple.comments

Simple commenting system with avatar integration

menhir.simple.livesearch

A viewlet that provides a livesearch box

menhir.simple.navtree

A viewlet providing a navigation tree

menhir.simple.tag

A tagging engine based on the lovely.tag

menhir.skin.lightblue

A complete skin for a Dolmen site.

menhir.skin.snappy

A skin for Snappy sites

snappy.site

The Snappy, a video sharing sample site

snappy.transform

Mimetype transform utilities. Not finished?

snappy.video.flasher

Utilities to mark files as Flash and allow to view them.

snappy.video.player

A video player for flash movies

snappy.video.transforms

Convert video files to flash movies and thumbnails

zeam.form.base

A form library designed to be grokish and simple

zeam.form.ztk

zope.schema integration for zeam.form. It provides widgets and default CRUD style actions.

dolmen-documentation

A few tutorials for Dolmen

dolmenproject

A Paste script extension that allows quick bootstrapping of new Dolmen projects

To download all the packages, I've ctrl+selected the git repositories names from http://gitweb.dolmen-project.org/, pasted them into a repositories.txt file and ran the following script:

import subprocess
import os

f = open('repositories.txt')
for line in f.readlines():
    git = line.strip()
    pkg = git
    if pkg.endswith('.git'):
        pkg = ".".join(pkg.split('.')[:-1])

    if os.path.exists(pkg):
        subprocess.check_call(['git', 'pull'], cwd=pkg)
    else:
        subprocess.check_call(['git', 'clone', 'git://devel.dolmen-project.org/' + git])

The issue is that the implementation of folders has changed from Plone 3 to 4 to use BTrees, which greatly improves performance. The problem is that, when viewing imported folders, I got the following traceback:

Traceback (innermost last):

    * Module ZPublisher.Publish, line 116, in publish
    * Module ZPublisher.BaseRequest, line 434, in traverse
    * Module Products.CMFCore.DynamicType, line 150, in __before_publishing_traverse__
    * Module Products.CMFDynamicViewFTI.fti, line 215, in queryMethodID
    * Module Products.CMFDynamicViewFTI.fti, line 182, in defaultView
    * Module Products.CMFPlone.PloneTool, line 840, in browserDefault
    * Module Products.CMFPlone.PloneTool, line 708, in getDefaultPage
    * Module Products.CMFPlone.utils, line 81, in getDefaultPage
    * Module plone.app.layout.navigation.defaultpage, line 32, in getDefaultPage
    * Module plone.app.layout.navigation.defaultpage, line 75, in getDefaultPage
    * Module Products.BTreeFolder2.BTreeFolder2, line 337, in has_key

AttributeError: 'NoneType' object has no attribute 'has_key' 

The solution was to call @@migrate-btrees on the imported folder, which fixes that folder and makes it conform to the latest implementation.

One final note, the default Plone buildout doesn't have a folder called "import" anywhere in the buildout, so one needs to be created inside the "client home folder", which is the folder of your plone and buildout instance, the one that hosts the bin, parts and var folders.

[myscript]
recipe = zc.regipe.egg
eggs = 
     myegg
     IMAPClient
python = python26

The python26 option is actually the name of a buildout part that configures the python executable path

[python26]
python = /usr/bin/python26

Now the problems. I've had various buildouts fail with a message "Cannot find egg myegg". After a bit of effort, we managed to trace the cause to this problem:

First, the python path in the [python26] part was incorect. Second, even if it pointed to the proper binary, the -devel packages for that python needed to be installed.

Well, now I know. Hopefully I'll remember it for the next time when I'll encounter the problem.

svn diff | kompare -

What it does is to take the output from svn diff and pipe it into Kompare, a merge/diff utility from the KDE Project. I can do this from the command line, straight from the directory that I'm in, and bang! I get a nice graphical overview, complete with the tree structure that I can navigate to see what I'm about to commit. Honestly, this little command gets me excited everytime I run it.

Now to the problem: ArchGenXML assumes (and hardcodes in its templates) the Products.* prefix for your package. I don't have any problems with it, but my employer uses a different package structure so for uniformity I need to follow their standards. 15 minutes of poking and changing through agx enabled me to change it so it would generate GS xml files with the proper namespace (based on a new model level TGV named "namespace" that I have created). After assessing the difficulty of the task and being under time pressure, I've decided to go the dumb route, which I'm documenting below:

My generation script is something like this:

./archgenxml -c archgenxml.cfg myproduct.zuml 

#rename Products.myproduct to ns.myproduct
find myproduct/* -type f -print | xargs sed -i 's/Products\.myproduct/ns\.myproduct/g'

#in the xml type profiles, replace myproduct by ns.myproduct
find myproduct/* -type f -print | xargs sed -i 's/>myproduct</>ns\.myproduct</g'

#in the profiles.zcml, rename the profile to ns.myproduct
find myproduct/* -type f -print | xargs sed -i 's/title=\"myproduct\"/title=\"ns\.myproduct\"/g'

Notice the sed lines, which change the source code to point to "ns.myproduct" instead of "Products.myproduct". Now there's just two more problems, which are actually AGX bugs:

• the FilesystemDirectoryViews registered for the skin are incorect, so you'll need to insert this in ns/myproduct/__init__.py

from Products.CMFCore import utils
from Globals import package_home
from os.path import dirname
ppath = utils.ProductsPath
utils.ProductsPath.append(dirname(package_home(product_globals)))
DirectoryView.registerDirectory('skins', product_globals)
utils.ProductsPath = ppath

• trying to get AGX to output the code inside a two level deep folder doesn't work (it crashes), so you'll need to move this script file and the model just above the "myproduct" folder, inside the "ns" namespace.

That's about it. Hopefully I'll find the time to fix the two bugs and add the namespace improvement in the next week. Still need to do the review for Plone for Education, which I have received for free from the publisher.

The latest Ubuntu version which I have installed (Lucid Lynx) comes with a package called python-pkg-resources, which packages pkg_resources, which used to be available only through the setuptools distribution. Buildout's bootstrap.py tries to guess if Setuptools or Distribute are installed by checking the availability of pkg_resources; by guessing wrong it all comes to a crash at the end.

I'm not very interested in debugging these types of problems anymore. Distribution/packaging tools should just work. I want to focus on my work, not debug the toolchain. No more corner cases or whatver. So I'm gonna compile separate Pythons in the future, especially when dealing with older Zope/Plones.

Yes, there are a couple of PyPi mirrors, but they only mirror files hosted by PyPi. The central PyPi site will probably have better performance and availability then what individual groups and developers can provide and it's always easier to mirror one single website than many, so there's no shame or loss of pride in using the PyPi to host your files. Please do so!