Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
Journeys with Transmogrifier & 
friends 
or 
How not to get stuck in 
the Plone dark ages 
Daniel Jowett 
Jowett Enterpris...
About me 
● Software Professional since 1997 
● Background in C, Java, SQL, etc …. even P*P 
● Using Plone since 2008 on s...
Foundational Principles 
● Calvin & Hobbes 
● Calvin is an over-imaginative 6 year old 
● Hobbes is his toy tiger 
● A car...
Transmogrifier: ...what is it? 
● Transmogrifier – turns 
something into anything else 
like Calvin into a toad, or tiger ...
Transmogrifier Variants 
● Plain Transmogrifier – 2 stage process to export/import from Plone 
site via xml files (or csv ...
When to use Jsonmigrator 
● From old Plone versions, like 2.5, but even as far back as 1.0 in theory 
(but requires Python...
When NOT to use Transmogrifier 
● From Plone 4.x 
● Probably not from 3.3.6 with buildout 
● Without considering the alter...
Technology Stack 
● Transmogrifier: 
– plone.app.transmogrifier 
– collective.transmogrifier 
● collective.jsonify 
● coll...
Setting Up Jsonmigrator 
1) * Duplicate old Plone in your staging environment 
2) install collective.jsonify to this dupli...
Install collective.jsonify 
This is the really old way to install stuff & generally bad practise, but if 
you are using a ...
Add “External Methods” 
● Add an Extensions folder 
● Add json_methods.py 
from collective.jsonify import get_item 
from c...
Install collective.jsonmigrator 
● Add the highlighted lines to your buildout's: develop.cfg 
[sources] 
... 
collective.j...
Looking closer 
● Installing Collective.jsonify & the external methods gives a (fairly) 
complete json view of every piece...
Have a go! 
● Go to: http://<your-plone>:<port>/<instance>/@@jsonmigrator 
● Select one of the pre-loaded sample configura...
Demo
So what happened there? (I) 
● This was our “pipeline” 
[transmogrifier] 
pipeline = 
remotesource 
removeid 
strip-leadin...
So what happened there? (II) 
● Here are some pipeline section definitions: 
[constructor] 
blueprint = collective.transmo...
Real life isn't Blue Peter* 
So, even if it did work just now, it will probably break for you first time, 
so make some ad...
Benefits compared to Legacy Plone 
● A fresh start 
● A chance to use new Plone technologies like Diazo, Dexterity, new 
C...
Caveats 
● This doesn't export/import users (though see 
collective.blueprint.usersandgroups) 
● And it doesn't do portlet...
Credits 
● Transmogrifier – Martjin Pieters, Jarn 
● Jsonmigrator - Rok Garbas 
● RCS – letting me loose on this 
● Calvin...
Fin 
Daniel Jowett 
www.jowettenterprises.com 
daniel@jowettenterprises.com 
IRC nick: danima1 
Github user: djowett
Prochain SlideShare
Chargement dans…5
×

Journeys with Transmogrifier and friends or How not to get stuck in the Plone dark ages

946 vues

Publié le

Using JsonMigrator & Transmogrifier to upgrade Plone sites.
Presentation given at Plone Conference 2014 in Bristol

Publié dans : Logiciels
  • Soyez le premier à commenter

  • Soyez le premier à aimer ceci

Journeys with Transmogrifier and friends or How not to get stuck in the Plone dark ages

  1. 1. Journeys with Transmogrifier & friends or How not to get stuck in the Plone dark ages Daniel Jowett Jowett Enterprises Ltd
  2. 2. About me ● Software Professional since 1997 ● Background in C, Java, SQL, etc …. even P*P ● Using Plone since 2008 on starting self-employment ● Once upon a time... ● … I came to a Plone Conference in Bristol... ● … in 2010 ● And everyone seemed to be cleverer than me ● And now 4 years later? ... ● I know most of them are cleverer than me ● But I have done 4 or 5 projects with Transmogrifer / Jsonmigrator
  3. 3. Foundational Principles ● Calvin & Hobbes ● Calvin is an over-imaginative 6 year old ● Hobbes is his toy tiger ● A cardboard box can be anything ● But in this case it's a “Transmogrifier”
  4. 4. Transmogrifier: ...what is it? ● Transmogrifier – turns something into anything else like Calvin into a toad, or tiger ● Plone's Transmogrifier: a way to migrate content – typically between Plone versions – also from plain html sites & other CMS databases – Aims for and achieves a decent amount of code reuse ● JsonMigrator: Builds on top of Transmogrifier so you don't have to install it in the source Plone site
  5. 5. Transmogrifier Variants ● Plain Transmogrifier – 2 stage process to export/import from Plone site via xml files (or csv files) ● Funnelweb - Crawls and parses static sites for import, which are then 'pushed' into Plone ● Jsonmigrator – 1 stage process that crawls Json views of an old site ● Transmogrifier blueprints also exist that can read from SQL Dbs, so for example Drupal data We will be focussing on Jsonmigrator
  6. 6. When to use Jsonmigrator ● From old Plone versions, like 2.5, but even as far back as 1.0 in theory (but requires Python >= 2.2) ● Even from old Zope/CMF sites ● particularly when you have no buildout ● When changing content type framework from archetypes to dexterity ● To clean cruft out of an error prone old site ● When scared to upgrade because you don't know what might bite you later Note: This isn't the whole story - you still have to Upgrade Your Plone products & custom code
  7. 7. When NOT to use Transmogrifier ● From Plone 4.x ● Probably not from 3.3.6 with buildout ● Without considering the alternatives: – running Plone upgrade steps (plone.app.upgrade) – plone.app.contenttypes migration for AT to DX based types
  8. 8. Technology Stack ● Transmogrifier: – plone.app.transmogrifier – collective.transmogrifier ● collective.jsonify ● collective.jsonmigrator ● Products containing extra pipeline sections, such as: – transmogrify.dexterity – quintagroup.transmogrifier – transmogrify.print
  9. 9. Setting Up Jsonmigrator 1) * Duplicate old Plone in your staging environment 2) install collective.jsonify to this duplicate 3) Add “external methods” 4) * Create a brand new Plone 4.x instance (try 4.3.3 for starters) 5) * Install the updated or replacement products you need to the new instance 6) install collective.jsonmigrator & any special pipeline sections to new instance * These steps are outside the scope of this presentation – hopefully you've got an idea of what to do here (though step 5 may well not be trivial)
  10. 10. Install collective.jsonify This is the really old way to install stuff & generally bad practise, but if you are using a virtualenv (or dedicated python) it's passable. It's much easier if you're using buildout, but I'll assume that you're not! 1. Download the egg (my version in this case) to your Plone directory: wget https://github.com/djowett/collective.jsonify/archive/0.1.8b.zip 2. Unzip it unzip 0.1.8b.zip 3. Add it to your python (it will also pull in simplejson) cd collective.jsonify-0.1.8b/ ../bin/zopepy setup.py install … Adding collective.jsonify 0.1.8b to easy-install.pth file … … simplejson...
  11. 11. Add “External Methods” ● Add an Extensions folder ● Add json_methods.py from collective.jsonify import get_item from collective.jsonify import get_children from collective.jsonify import get_catalog_results ● Then run your Plone instance, go to the Zope root and create the three equivalently named External Methods, ie: “get_item”, “get_children” & “get_catalog_results”, eg:
  12. 12. Install collective.jsonmigrator ● Add the highlighted lines to your buildout's: develop.cfg [sources] ... collective.jsonmigrator = git https://github.com/djowett/collective.jsonmigrator.git [buildout] eggs += ... collective.jsonmigrator ● You might also want to add transmogrify pipelines like transmogrify.dexterity or quintagroup.transmogrifier to your eggs section ● Then run buildout & run up your instance Note: We check out the collective.jsonmigrator egg for development so that we can tweak the pipeline configuration when we need to
  13. 13. Looking closer ● Installing Collective.jsonify & the external methods gives a (fairly) complete json view of every piece of content on your old site ● For example... ● Collective.jsonmigrator can pick up these json data dictionarys, and using transmogrifier 'massage' them and create the corresponding objects on your new site
  14. 14. Have a go! ● Go to: http://<your-plone>:<port>/<instance>/@@jsonmigrator ● Select one of the pre-loaded sample configurations ● Select username, password & a small folder for your first run
  15. 15. Demo
  16. 16. So what happened there? (I) ● This was our “pipeline” [transmogrifier] pipeline = remotesource removeid strip-leading-path correct-zope-file-types fix_index_html_in_id fix_index_html_in_path print_seq_in_desc description_remove_seq tag_general_if_ccsnews_in_desc tag_service_if_ccsnews_not_in_desc description_remove_ccsnews find_path_depth repl_relative_urls_in_text get_hardcoded_urls_in_text print_hardcoded_url_post_replace get_siteImage_urls_in_text print_siteImage_urls fix_index_html_refs_in_text find_h1_title_in_text remove_h1_title_from_text constructor schema uid datafields fix_defaultPage # print_defaultpage browser-default # workflow-history get-last-workflow-state calc-workflow-transition publish_all_folders workflowupdater properties # permissions owner local_roles mimetype commit IMPORTING
  17. 17. So what happened there? (II) ● Here are some pipeline section definitions: [constructor] blueprint = collective.transmogrifier.sections.constructor # Zope Images & Files get incorrectly reported as Folders [correct-zope-file-types] blueprint = collective.transmogrifier.sections.manipulator keys = _classname destination = python:"_type" condition = python:(item['_type'] == 'Folder' and item.has_key('_classname') and item['_classname'] in ['Image', 'File']) [strip-leading-path] blueprint = collective.transmogrifier.sections.inserter key = string:_path value = python: item['_path'].replace('ccs/', '', 1) [tag_general_if_ccsnews_in_desc] blueprint = collective.transmogrifier.sections.inserter key = string:subject value = python: item['subject'] + ['general'] condition = python: item['_type'] == "News Item" and 'ccsnews' in item['description']
  18. 18. Real life isn't Blue Peter* So, even if it did work just now, it will probably break for you first time, so make some adjustments in the pipeline (<instance>/src/collective.jsonmigrator/collective/jsonmigrator/ config_plone3.cfg) ● Exclude objects from the import (particularly plone tools) ● Use print a lot (from transmogrify.print) ● Delete the import results (if you had any) & run again * Blue Peter was the staple TV diet for 1980's Kids in the UK, featuring the oft used phrase “here's one I made earlier” when things didn't work
  19. 19. Benefits compared to Legacy Plone ● A fresh start ● A chance to use new Plone technologies like Diazo, Dexterity, new Collections, plone.app.contenttypes. etc. etc. ● Moving from retired products to cutting edge equivalents ● The speed of Plone 4 & blob storage ● Using buildout to control your site
  20. 20. Caveats ● This doesn't export/import users (though see collective.blueprint.usersandgroups) ● And it doesn't do portlets (portlets can be exported by generic setup, but that doesn't include “deep” content portlets – you might be stuck with a manual portlet setup) ● Collective.jsonify ignores Plone's security model and is a security loophole – add this to a publicly accessible site at your own risk ● The docs can be a little misleading at times ● Many folk using this are probably working off checked out code, so egg releases are few & far between – you are going to have to get your hands dirty ● “Debugging is backwards” in transmogrifier
  21. 21. Credits ● Transmogrifier – Martjin Pieters, Jarn ● Jsonmigrator - Rok Garbas ● RCS – letting me loose on this ● Calvin & Hobbes – Bill Watterson – creating the original Transmogrifier Links ● http://www.jowettenterprises.com/blog/jsonmigrator ● http://collectivejsonify.readthedocs.org/ ● http://collectivejsonmigrator.readthedocs.org/
  22. 22. Fin Daniel Jowett www.jowettenterprises.com daniel@jowettenterprises.com IRC nick: danima1 Github user: djowett

×