This document discusses customizing and extending Plone 3 using buildout, eggs, GenericSetup profiles, and testing. Key points include: 1) Using buildout to manage dependencies and custom packages, including developing packages locally. 2) Creating a "policy product" that installs all customizations and dependencies in one step. 3) Customizing Plone through GenericSetup profiles using XML configuration files and custom import steps. 4) Writing tests to ensure customizations are properly installed and configured. 5) Customizing visual components like skins, views, and resources using layers and browser resources. Thorough testing and source control are emphasized.

1. Extending and customising
Plone 3
Martin Aspeli

2. A small aside

3. Agenda
Love the buildout
Embrace the egg
Policy: Repeatable repeatable repeatable!
Here a dependency, there a dependency
You love GenericSetup
Thou shalt write tests!
Visual customisation: A cacophony of layers and skins
Next steps …

4. Introduction
Zope 2 is great
But we want more!
Smaller, isolated packages are better
But we need better tools to manage them
Development and deployment are getting more
complex
Plan to keep control from the beginning
Through-the-web development is not cool anymore

5. Some terminology
Product – A Python package for Zope 2. Often lives in Products/ and is
subject to certain magic when Zope starts up.
Package – A more general term, but in the Zope world is typically refers to
something that is not in the Products.* name space.
Egg – A way to distribute packages. Supports automatic dependency
management: dependencies can be downloaded from PyPI/The Cheese Shop.
Buildout – A way to manage projects, with tools for building Zope instances,
downloading Plone, installing eggs and so on (these are known as “recipes”).
GenericSetup – A way to configure a Plone site using declarative XML files.
Policy product – A pattern whereby a project has a single product which,
upon installation, installs all dependencies and customisations in one step.

6. A buildout
First, get easy_install, ZopeSkel and paster
These can go in the global Python (most of the time)
Watch out for where they get installed
$ wget http://peak.telecommunity.com/dist/ez_setup.py
$ easy_install -U ZopeSkel
$ paster create --list-templates

7. A buildout
We will use the plone3_buildout template from
ZopeSkel
$ paster create -t plone3_buildout pc07
Answer the questions
the only one you have to answer is password
Setting debug mode to on is usually a good idea

8. A buildout
Let’s look at what we got:
buildout.cfg is the main configuration file
products/ can be used for old-style Products
src/ is used for custom eggs
var/ is used for Data.fs and logs

9. A buildout
Buildout will generate other directories:
bin/ contains executables
parts/ contains files managed by buildout
eggs/ contains downloaded eggs
develop-eggs/ contains egg-links to development
eggs

10. A buildout
Let’s build it
$ cd pc07
$ python bootstrap.py
$ ./bin/buildout
If you are on Windows
I’m sorry to hear that
But you can still get it to work - just read the
generated README.txt file and set up mingw32.
If Python 2.4 is not your default Python
Read README.txt

11. A buildout
Believe it or not, you now have a fully working Zope
and Plone installation
$ ./bin/instance fg
More importantly, we can manage other dependencies
and packages through buildout.cfg
We can also set up ZEO, Varnish, Deliverance, Apache
and so on using standard or custom recipes
To learn more about buildout, see
http://plone.org/documentation/tutorial/buildout

12. Subversion
Idiot developers do not use source control
We are not idiot developers
You don’t need a remote server to run Subversion
$ svnadmin create /repo
$ svn mkdir file:///repo/pc07/{,/trunk}
$ svn co file:///repo/pc07/trunk pc07 # our project
Of course, if you have a server, use that instead!
In this case, you can use svn+ssh or Apache and
mod_dav

13. Subversion
Check in only source code (not generated files or data)
$ svn ps svn:ignore '
> eggs
> develop-eggs
> bin
> var
> parts
> .installed.cfg' .
$ svn add products bootstrap.py buildout.cfg src README.txt
$ svn commit
Other developers can now use the same buildout
You can move it to a staging or live server

14. A policy product
This is a package that performs all the customisations
and installs all the dependencies to turn vanilla Plone
site into your application or site
Create a new egg
$ cd src/
$ paster create -t plone example.policy
Namespace is example, package is policy, Zope 2
product is True, Zip-safe is False

15. A policy product
This package is an egg!
setup.py declares dependencies, metadata
example/policy/ contains actual code
Let’s add to svn, but ignore the generated egg-info
$ svn add -N example.policy
$ cd example.policy
$ svn ps svn:ignore '*.egg-info' .
$ svn add setup.py docs example README.txt setup.cfg
$ svn commit

16. A policy product
We need to tell buildout about the package.
Declare it as a development egg (so buildout doesn’t
search the Cheese Shop for it!)
Add it to the egg “working set” for the Zope 2
instance
Install a ZCML slug so that Zope finds it at startup

17. A policy product
Edit buildout.cfg:
[buildout]
...
eggs =
elementtree
example.policy
develop =
src/example.policy
[instance]
...
zcml =
example.policy

18. A policy product
Re-run ./bin/buildout
$ ./bin/buildout -o
Develop: '/Users/optilude/tmp/eggs/pc07/src/example.policy'
...
We can run buildout in offline mode (-o) to make it a bit
faster
The egg is now ready. We can run the generated tests
just to check.
$ ./bin/instance test -s example.policy

19. Installing dependencies
Most sites use third party products
As a developer, you may want to use a particular
Python library (not necessarily a Zope product)
We would like to make the process of obtaining and
installing these as automated as possible

20. Installing dependencies
If it is an old-school product without a release, add it to
the top-level products/ directory (note lowercase “p”)
Do not add anything inside parts/ as buildout may
delete your changes
To track a package in Subversion, use svn:externals:
$ cd products
$ svn ps svn:externals 'RichDocument https://svn.plone.org/
svn/collective/RichDocument/trunk' .
$ svn up
$ svn commit

21. Installing dependencies
If it is an old-school product with a release, you can
have buildout download and configure it for you.
Edit buildout.cfg:
[productdistros]
...
urls =
http://plone.org/products/richdocument/releases/3.0.1/
RichDocument-3.0.1.tar.gz
nested-packages =
version-suffix-packages =
Then re-run ./bin/buildout
See http://plone.org/documentation/tutorial/buildout for
more about the other options

22. Installing dependencies
If your dependency is an egg in the Cheese Shop, add
it to setup.py in the policy product and re-run buildout:
install_requires=[ 'setuptools',
'plone.browserlayer>=1.0rc1,<1.1dev',],
Re-run ./bin/buildout to get the new package.
Be careful: Some of the newer Zope 3 eggs declare
parts of Zope 3.4 as transitive dependencies. This may
break Zope 2.10. For now, you must find a version
without copious dependencies.
To learn more about how setuptools handles versions:
http://peak.telecommunity.com/DevCenter/setuptools

23. Installing dependencies
If there is no release, put the egg in the src/ directory
Add it to the develop option in buildout.cfg
[buildout]
...
develop =
src/example.policy
src/some.package
You can still use the policy product’s setup.py to
declare it as a dependency
install_requires=[
'setuptools',
'some.package',
],

24. Installing dependencies
For a package outside Products.*, we must include it in
ZCML processing explicitly
Edit src/example.policy/example/policy/configure.zcml:
<configure
xmlns=quot;http://namespaces.zope.org/zopequot;
xmlns:five=quot;http://namespaces.zope.org/fivequot;
i18n_domain=quot;example.policyquot;>
<include package=quot;plone.browserlayerquot; />
...
</configure>

25. Installing dependencies
We should ensure that installable Zope 2 products are
installed when the policy product itself is installed
Create src/example.policy/example/policy/Extensions/
Install.py and add this boilerplate:
import transaction
from Products.CMFCore.utils import getToolByName
PRODUCT_DEPENDENCIES = ('RichDocument', 'plone.browerlayer') # Edit this list to add new dependencies
EXTENSION_PROFILES = () # ('example.policy:default',) # Our profile - we’ll activate this in a moment
def install(self, reinstall=False):
portal_quickinstaller = getToolByName(self, 'portal_quickinstaller')
portal_setup = getToolByName(self, 'portal_setup')
for product in PRODUCT_DEPENDENCIES:
if reinstall and portal_quickinstaller.isProductInstalled(product):
portal_quickinstaller.reinstallProducts([product])
transaction.savepoint()
elif not portal_quickinstaller.isProductInstalled(product):
portal_quickinstaller.installProduct(product)
transaction.savepoint()
for extension_id in EXTENSION_PROFILES:
portal_setup.runAllImportStepsFromProfile('profile-%s' % extension_id, purge_old=False)
product_name = extension_id.split(':')[0]
portal_quickinstaller.notifyInstalled(product_name)
transaction.savepoint()

26. Customisation with GS
GenericSetup lets us define customisations via XML
Two types of profiles:
Base profile: A complete configuration. Plone is
installed using one of these.
Extension profile: Bolts onto a base profile to amend
or change configuration. This is the most useful kind
for third party developers.
Managed via portal_setup, but portal_quickinstaller
knows how to install them too

27. Customisation with GS
First we must create and register a profile
$ mkdir -p example.policy/example/policy/profiles/default
Edit configure.zcml and add:
<configure
...
xmlns:genericsetup=quot;http://namespaces.zope.org/
genericsetupquot;>
<genericsetup:registerProfile
name=quot;defaultquot;
title=quot;Example policy productquot;
directory=quot;profiles/defaultquot;
description=quot;Install our example customisationsquot;
provides=quot;Products.GenericSetup.interfaces.EXTENSIONquot;
/>
...

28. Customisation with GS
If we didn’t have Extensions/Install.py, the quick-
installer would have found this profile and made it
available for installation
In the future, we will be able to declare profile
dependencies
Until then, we use Install.py to install dependencies and
then trigger our own profile
Now we can un-comment this line in Install.py:
EXTENSION_PROFILES = ('example.policy:default',)

29. Customisation with GS
Look at parts/plone/CMFPlone/profiles/default
This is Plone’s base profile. Copy any of these files into
your product’s extension profile and modify as
appropriate.
For example, create profiles/default/propertiestool.xml:
<object name=quot;portal_propertiesquot;>
<object name=quot;navtree_propertiesquot;>
<property name=quot;metaTypesNotToListquot; type=quot;linesquot; purge=quot;falsequot;>
<element value=quot;News Itemquot;/>
</property>
</object>
<object name=quot;site_propertiesquot;>
<property name=quot;allowAnonymousViewAboutquot; type=quot;booleanquot;>False</property>
</object>
</object>
When this is installed, portal_properties will be changed

30. Customisation with GS
Some useful handlers:
properties.xml – sets properties at the root of the site
propertiestools.xml – manages portal_properties
rolemap.xml – Creates roles and sets permissions at
the root of the site
actions.xml – Sets up actions in portal_actions
catalog.xml – Creates catalog indexes/metadata
workflows.xml – Maps and creates workflow
definitions (kept in the workflows/ folder)

31. Customisation with GS
It is not terribly hard to write your own GenericSetup
import/export handler
However, sometimes we just need to write some
Python code
For this, we can use the “import-various” hack
This registers a new handler, but instead of parsing an
XML file we simply execute some custom code

32. Customisation with GS
Create profiles/default/example.policy_various.txt. This
is a “flag” - it can be empty
Create profiles/default/import_steps.xml:
<import-steps>
<import-step id=quot;example.policy_variousquot;
version=quot;20071012-01quot;
handler=quot;example.policy.setuphandlers.importVariousquot;
title=quot;Various Example Settingsquot;>
<dependency step=quot;catalogquot;/>
<dependency step=quot;propertiestoolquot;/>
</import-step>
</import-steps>

33. Customisation with GS
Create setuphandlers.py:
def importVarious(context):
if context.readDataFile('example.policy_various.txt')
is None:
return
site = context.getSite()
# Now do something useful
We need to check for the flag - otherwise, the handler
could run for other profiles as well
Try to avoid using an import-various step if you can

34. Installation tests
Idiot developers don’t write tests
Remember - we’re not idiot developers
The most basic tests just check that our
customisations are in effect
There was a tests.py generated. That’s nice, but it’s a
bit simple for our needs. Get rid of it.

35. Installation tests
Create src/example.policy/example/policy/tests/, and
within it, __init__.py and then base.py:
from Products.Five import zcml
from Products.Five import fiveconfigure
from Testing import ZopeTestCase as ztc
from Products.PloneTestCase import PloneTestCase as ptc
from Products.PloneTestCase.layer import onsetup
ztc.installProduct('SimpleAttachment')
ztc.installProduct('RichDocument')
@onsetup
def setup_package():
import example.policy
zcml.load_config('configure.zcml', example.policy)
ztc.installPackage('plone.browserlayer')
ztc.installPackage('example.policy')
setup_package()
ptc.setupPloneSite(products=['example.policy'])
class ExamplePolicyTestCase(ptc.PloneTestCase):
quot;quot;quot;Common test base class
quot;quot;quot;
This sets up a Plone site for testing with our example

36. Installation tests
Now create tests/test_setup.py:
import unittest
from example.policy.tests.base import ExamplePolicyTestCase
from Products.CMFCore.utils import getToolByName
class TestSetup(ExamplePolicyTestCase):
def afterSetUp(self):
self.properties = getToolByName(self.portal, 'portal_properties')
self.types = getToolByName(self.portal, 'portal_types')
def test_metaTypesNotToList_set(self):
navtree_props = self.properties.navtree_properties
mtntl = navtree_props.getProperty('metaTypesNotToList')
self.failUnless('News Item' in mtntl)
def test_allowAnonymousViewAbout_set(self):
site_props = self.properties.site_properties
allow = site_props.getProperty('allowAnonymousViewAbout')
self.assertEquals(False, allow)
def test_rich_document_installed(self):
self.failUnless('RichDocument' in self.types.objectIds())
def test_suite():
suite = unittest.TestSuite()
suite.addTest(unittest.makeSuite(TestSetup))
return suite
This tests that we got the basic setup right

37. Installation tests
To run the tests, go to the root of the buildout and do:
$ ./bin/instance test -s example.policy
You should also try it through-the-web!

38. Visual customisation
Visual components include
Resources in layers found in portal_skins
Zope 3-style browser views
Zope 3-style viewlets
Zope 3-style browser resources (images, stylesheets)
Zope 3 resources are customised with the ‘layer’ ZCML
attribute
Resources in portal_skins are customised in ‘skin
layers’ - these have nothing to do with each other!

39. Visual customisation
In portal_skins in the ZMI, on the Properties tab, find
your skin
This lists a number of layers, which refer to folders
inside portal_skins itself
Bar custom, these folders are actually files on the file
system
Layers higher up take precedence when looking for a
template, image or other resource

40. Visual customisation
Edit example.policy/example/policy/__init__.py:
from Products.CMFCore.DirectoryView import
registerDirectory
GLOBALS = globals()
registerDirectory('skins', GLOBALS)
Add the folder skins/example_policy
Copy from parts/plone/CMFPlone/skins/* into here and
change as necessary
If the file has an associated .metadata file, get this too!

41. Visual customisation
To install the new layer, create profiles/default/skins.xml:
<object name=quot;portal_skinsquot;>
<object name=quot;example_policyquot;
meta_type=quot;Filesystem Directory Viewquot;
directory=quot;example.policy:skins/example_policyquot;/>
<skin-path name=quot;*quot;>
<layer name=quot;example_policyquot;
insert-after=quot;customquot;/>
</skin-path>
</object>
Install or re-install in Plone and look at portal_skins

42. Visual customisation
Zope 3 also has the concept of a layer
A Zope 3 layer is just an interface
All the browser ZCML directives can take a layer
If given, the view/viewlet/resource is only available
when this layer is in effect - this can also be used to
override a particular resource (by name)
With plone.theme, we can associated a layer with a
theme
With plone.browserlayer, we can install a layer for a
particular product.

43. Visual customisation
Create an interface in interfaces.py:
from zope.interface import Interface
class IExamplePolicy(Interface):
quot;quot;quot;This marker is used for Zope 3 layers
quot;quot;quot;
Then create profiles/default/browserlayer.xml:
<layers>
<layer name=quot;example.policy.layerquot;
interface=quot;example.policy.interfaces.IExamplePolicyquot; />
</layers>
With this, the IExamplePolicy layer is applied (as a
marker interface on the request) on each request in this
Plone site after the product has been installed.

44. Visual customisation
Let’s create a viewlet that is installed with this package
A viewlet is a small snippet of HTML that is inserted in
the page, inside a viewlet manager
Create browser/ and browser/__init__.py
In configure.zcml add:
<include package=quot;.browserquot; />

45. Visual customisation
Create browser/configure.zcml:
<configure
xmlns=quot;http://namespaces.zope.org/zopequot;
xmlns:browser=quot;http://namespaces.zope.org/browserquot;
i18n_domain=quot;example.policyquot;>
<browser:viewlet
name=quot;example.footerquot;
manager=quot;plone.app.layout.viewlets.interfaces.IPortalFooterquot;
class=quot;.footer.FooterViewletquot;
permission=quot;zope2.Viewquot;
layer=quot;..interfaces.IExamplePolicyquot;
/>
</configure>
Look at plone.app.layout.viewlets to find standard
viewlet managers and viewlets
This is in eggs/plone.app.layout-{version}.egg/...

46. Visual customisation
Create browser/footer.py:
from plone.app.layout.viewlets.common import ViewletBase
from Products.Five.browser.pagetemplatefile import
ViewPageTemplateFile
class FooterViewlet(ViewletBase):
quot;quot;quot;Create an extra footer
quot;quot;quot;
template = ViewPageTemplateFile('footer.pt')
def update(self):
super(FooterViewlet, self).update()
# calculate stuff here
def render(self):
return self.template()

47. Visual customisation
Create browser/footer.pt:
<div>
We hope you liked
<a tal:attributes=quot;href view/portal_urlquot;>our site</a>
</div>
Now restart Plone, install the product, and you should
see the viewlet
Use the /@@mange-viewlets view to move things
around
Then make that permanent with viewlets.xml

48. Visual customisation
Our viewlet had a unique name (and layer)
We can also override a browser resource by context
type (for attribute) or layer – using the same name
Here, we override a browser view
For simplicity, we specify a template only, and re-use
the same class
Some views have templates only
For a class-only view, the __call__() method is called – it
can render a template (or do nothing)

49. Visual customisation
Add this to browser/configure.zcml:
<include package=quot;plone.app.portletsquot; />
<browser:page
for=quot;Products.CMFCore.interfaces.ISiteRootquot;
name=quot;dashboardquot;
permission=quot;plone.app.portlets.ManageOwnPortletsquot;
class=quot;plone.app.layout.dashboard.dashboard.DashboardViewquot;
template=quot;dashboard.ptquot;
layer=quot;..interfaces.IExamplePolicyquot;
/>
We copied this from plone.app.layout.dashboard
We include plone.app.portlets first to ensure we have
the plone.app.portlets.ManageOwnPortlets permission

50. Visual customisation
Copy plone.app.layout.dashboard’s dashboard.pt to
browser/dashboard.pt.
Make a change, e.g. change title to:
<h1 class=quot;documentFirstHeadingquot;
i18n:translate=quot;heading_dashboardquot;>
<span tal:replace=quot;namequot; i18n:name=quot;user_namequot; />'s
really cool dashboard
</h1>

51. Next steps
Archetypes content
types
Forms and formlib
Themes
New workflows
Portlets
Other components
Deployment buildouts