If someone tells you that it is impossible to reimplement some piece of software to be compatible with a newer technology, that is most often not truth.
This is the story about porting RestrictedPython, one of the core packages of Zope to be compatible with Python 3.
3. PloneConf 2014 Bristol - Plone Roadmap Discussion
“Plone on Python 3
is impossible,
there are too
many blockers in Zope.
Especially RestrictedPython
& AccessControl”
3
4. Clarke's First Law – Arthur C. Clarke
“When a distinguished but elderly
scientist states that something is
possible, he is almost certainly right.
When he states that something is
impossible, he is very probably wrong.”
4
5. – 2015 - Start looking into RestrictedPython
“Why should RestrictedPython
be the major blocker for
Plone on Python 3?”
5
6. Hanno Schlichting - Zope Release Manager
„Every piece of Zope that was not
adopted by Plone is literally dead.“
6
7. „Debugging is twice as hard as writing
the code in the first place. Therefore, if
you write the code as cleverly as
possible, you are, by definition, not
smart enough to debug it.“
7
- Brian Kernighan
Almost no documentation
low test coverage
8. Python community - around 2000
„Where Zope leads,
Python follows“
8
Zope specific leftovers in the Python 2 Standard Library
Deprecated Python Standard Library Modules:
• compiler Package (no described upgrade path, almost no documentation)
9. Clarke's Second Law – Arthur C. Clarke
“The only way of discovering the
limits of the possible is to
venture a little way past them
into the impossible.”
9
10. Clarke's Third Law – Arthur C. Clarke
“Any sufficiently
advanced technology
is indistinguishable
from magic.”
10
13. A Workshop on Python at NIST
SPAM 1 - First PyCon (1994) - Topics
I've assembled a strawman list of topics for discussion. New topics
and comments on the topics listed are certainly welcome.
A.Requirements for a "Safe" Python interpreter
B. A standard GUI module interface definition for Python
C.The requirements for persistent objects in Python
D.A Python engineering graphs package
E. The standard Python WWW interface
F. Embedding Python in a WWW client
G.Technical information management using Python
H.Support for dynamic loading of foreign language modules in Python
I. Replacing make, rcs, and cvs with Python
J. An Electronic Data Interchange library for Python
K. Discussing the formation of a Python Consortium
13
14. Rich Hickey
“Most of the biggest problem
in software are problems of
misconception.”
14
19. • Python 2 Standard Library Module compiler and its Class ast
• Not fully documented
• No upgrade path for Python 3 described
• manual Byte-Code generator
• Interpreter specifics —> Only CPython 2 compatible
19
20. If it is not documented it is not usable
if it is not tested it did not work
if it is not checked into version control it did not exists
if it is not repeatable it is not science
• Compiler knowledge was necessary to port RestrictedPython
• I did have a degree in Informatics (Computer Science) and
learned how to build and manipulate compilers, so give it a try.
20
22. Kent Beck
“Any fool can write code that a
computer can understand.
Good programmers write code
that humans can understand.”
22
Reading &
understanding the Code
23. Louis Sreygley
“Without requirements or design,
programming is the art of adding
bugs to an empty text file.”
23
Documentation First +
Test Driven Development
24. • compiler.ast => ast.AST
• builtin compile() function since Python 2.6 accepts ast.AST as input
and compiles to Python Byte-Code —> Interpreter independence
• compile(source, filename, mode [, flags=ast.PyCF_ONLY_AST [, dont_inherit]])
or ast.parse() return ast.AST
With Python 2.6+ / 3.4+ RestrictedPython could be easier to implement
24
25. Got Access to Zope Foundation Repositories via Tres Seaver at Plone Symposium Tokyo 2015
“Access Required.”
25
26. >>> import this
The Zen of Python, by Tim Peters
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to
break the rules.
…
PEP 20 - 19. Aug. 2004 https://www.python.org/dev/peps/pep-0020/
26
Apply Python’s PEP8 and
Plone’s Coding Conventions
—> Make the Code more readable
27. – Richard Feynman
“The first principle is that you
must not fool yourself — and you
are the easiest person to fool.”
27
Writing Tests & raising test coverage
—> Functional Test of all Python Syntax cases
28. tox & .
• tox: local tests of multiple python versions
• pytest: modern Python testing framework, that allows:
• parameterisation (inputs and functions)
• skip_if
• assert
Talk: Modern Python Testing
28
35. My Wish
Make RestrictedPython more known
enable other Projects and Frameworks to use
a „Safe“ Python Interpreter through the Web
36. John Gall - Systemantics: How Systems Really Work and How They Fail
A complex system that works is invariably
found to have evolved from a simple system
that worked. A complex system designed from scratch
never works and cannot be patched up to make it work.
You have to start over with a working simple system.
36
37. Lessons Learned while Porting RestrictedPython
• Don’t take »„impossible“ to port / fix« statements serious
• Adopt modern tools and frameworks that helps:
• tox & pytest
—> Update Best Practices for Plone development
37
38. Some further Talk recommendations
• Zope on Python 3
Hanno Schlichting - Friday 14:30-15:15 - Auditori
• Modern Python Testing
Alexander Loechel - Thursday 14:30-15:15 - Sala d'actes
• subtemplates in bobtemplates.plone or on the way to plonecli
Maik Derstappen - Friday 15:25-15:55 - Sala d'actes
38
39. Clarke's Law of Revolutionary Ideas – Arthur C. Clarke
Every revolutionary idea
— in science, politics, art, or whatever —
seems to evoke three stages of reaction.
They may be summed up by the phrases:
1. "It's completely impossible — don't waste my time"
2. "It's possible, but it's not worth doing"
3. "I said it was a good idea all along"
39