As such, I’ve been having to figure out a lot along the way and I thought I should share some of what I’ve learned to save others the trouble.  Plone 3.2 introduced some big improvements to the Plone build system recently that make it much easier to get started with and that aren’t in any books yet.  (My copy of Professional Plone Development by Martin Aspeli covers Plone 3, but not 3.2. Plone’s doc on how to upgrade to 3.2 only helped a little.  )

The PSU WebLion wiki has been very helpful.  They have a great description of what a Buildout is, among other helpful info.  That’s where I found the this blog entry on building out with Plone 3.2.x, which now packages Plone as Python eggs.

Here’s a minimal buildout procedure:

$ mkdir plone3.2_buildout
$ cd plone3.2_buildout
$ wget http://svn.zope.org/*checkout*/zc.buildout/trunk/bootstrap/bootstrap.py
$ cat > buildout.cfg
[buildout]
extends = http://dist.plone.org/release/3.2.1/versions.cfg
versions = versions
find-links = http://dist.plone.org/thirdparty
parts =
    zope2
    instance

[zope2]
recipe = plone.recipe.zope2install
url = ${versions:zope2-url}
fake-zope-eggs = true

[instance]
recipe = plone.recipe.zope2instance
zope2-location = ${zope2:location}
user = admin:admin
http-address = 8080
eggs =
    PIL
    Plone
^D
$ python2.4 bootstrap.py
$ bin/buildout

For any other Plone products you use, check to see if they’re in Pypi.  If so, you can just add them to the list of eggs in the instance.  If your favorite product isn’t in egg form, consider nudging the developers. Plone recommends all Products become eggs and it’s easy to do. I know hardly anything about Plone and was able to convert the ECQuiz product to an egg.  If you want that egg you can just add "Products.ECQuiz" under "Plone" in your instance:eggs list. (Tip for converters:  if you get "unbound prefix" from configure.zcml you probably need this line,

    xmlns:genericsetup="http://namespaces.zope.org/genericsetup" )

I’m also making edits to the ECQuiz code so it’s a little more complicated, but only barely.  I keep the code in src/ and I added the buildout.eggtractor extension to my buildout.cfg which scans the src directory for product eggs and automatically adds the appropriate values to the buildout.  Add it like so,

[buildout]
  extends = http://dist.plone.org/release/3.2.1/versions.cfg

  versions = versions

  #(...)

  extensions =
        buildout.eggtractor

If you get, "Error: Missing option: buildout:develop" you also need a "develop = " option to [buildout].  For more on buildout, try the Buildout Quick Reference card.

Now let’s say you want to check your new site into Subversion.  (This tip I got from Martin Aspeli’s slideshow on Extending and Customizing Plone 3.)  You don’t want to check your whole directory in because much of that is built out automatically.  Other developers don’t need it and likely won’t even work on their system because much of what is built out contains hard-coded paths to other files and executables.

So before checking all the files within your buildout, update the directory’s svn:ignore property to omit the stuff that doesn’t need to go into version control.  In your buildout directory run,

$ svn propset svn:ignore '
> eggs
> develop-eggs
> bin
> var
> parts
> .installed.cfg' .  # note period for current dir
$ svn add products bootstray.py buildout.cfg src README.txt
$ svn commit

Now someone else can check out that code and run "python bootstrap.py" to generate the bin directory and then "bin/buildout’ to build out all the eggs and code.

If you’re building more than one Plone site, you can save disk space and unnecessary downloads by configuring your Buildout to use a shared directory.

$ mkdir -p ~/.buildout/downloads
$ mkdir -p ~/.buildout/eggs
$ cat > ~/.buildout/default.cfg
[buildout]
  eggs-directory = ~/.buildout/eggs

  download-cache = ~/.buildout/downloads

That’s all I got for now.  I hope that’s helpful.

In an analysis I’m working on, I’m going to be munging the data all sorts of ways and the graphical statistics environment I’ve been using (JMP) is going to be painful for me to use. Particularly because it updates state and it’s hard to remember how I got there. I’ve had great success on a project recently using GNU Make to process the data from raw to beautiful and I want to try that here.

In my other project, the raw data is in XML so I got handy with Python and an XML processor. In this project, my raw data is in a MySQL database. I thought that would be as straightforward, but it hasn’t been. That may be due more to using a Mac this time around though (as opposed to Windows and Linux previously).

In any event, here is what I had to do to get the MySQL-python on my MacBook Pro. I hope this saves someone else the trouble. In the end, I made an .egg, so if you want to skip the rest you can just download that: MySQL_python-1.2.2-py2.5-macosx-10.5-i386.egg

First I had to get setuptools working. I noticed had two versions of Python installed, Leopard’s and MacPorts’, and decided to remove the MacPorts one to keep things clean. (Incidentally, I regret ever making use of MacPorts. I’ve much rather have built binaries ala Fink.) Once I had setuptools working I thought I could install the MySQL driver with simply,

easy_install MySQL-python

Unfortunately, that fails with a compile error,

In file included from /usr/local/mysql/include/mysql.h:47,                 from _mysql.c:40:/usr/include/sys/types.h:92: error: duplicate ‘unsigned’/usr/include/sys/types.h:92: error: two or more data types in declaration specifierserror: Setup script exited with error: command 'gcc' failed with exit status 1

Fortunately I found a blog that provided a workaround,
Install MySQL server from mysql.com
Download and unpack the MySQLdb package (download URL here)
Fix the path to mysql_config, either by editing setup-posix.py (editing mysql_config.path value) or adding /usr/local/mysql/bin/ to your PATH. (i.e. export PATH=$PATH:/usr/local/mysql/bin/)
In the folder run python setup.py clean and python setup.py build

At this point you’ll probably get this error,

/usr/include/sys/types.h:92: error: duplicate ‘unsigned’/usr/include/sys/types.h:92: error: two or more data types in declaration specifiers

To get around this, edit _mysql.c to change

#define uint unsigned int
to

#define /* uint unsigned int */
Then build again. You’ll probably see this warning,

ld: warning in build/temp.macosx-10.5-i386-2.5/_mysql.o, file is not of required architectureld: warning in /usr/local/mysql/lib/libmysqlclient_r.dylib, file is not of required architectureld: warning in /usr/local/mysql/lib/libmygcc.a, file is not of required architecture

I ignored them and haven’t encountered any problems. I’m on a 32-bit Intel Mac though. If you’re on 64-bit, this blog post may be able to help.

Run sudo python setup.py install to install.

Then I ran this code to test. Once I knew it was working, I tried to find a way to share the binary. I was able to build a binary .egg with python setup.py bdist_egg, which made MySQL_python-1.2.2-py2.5-macosx-10.5-i386.egg

I’d like to upload it to the PyPi record, having registered and set up my .pypirc file, but I don’t have permission. After writing this, I will contact to package index owner, Andy Dustman, and see if he can include it. If he does, then the simple command I was expecting at first should work for 32-bit Mac users now on.

easy_install MySQL-python

And until then, this negligibly longer command should work,

easy_install http://www.cs.cmu.edu/%7Etaleahma/projects/misc/MySQL_python-1.2.2-py2.5-macosx-10.5-i386.egg