Porting to Python 3

Porting to Python 3
Lennart Regebro @ PyCon Atlanta 2011

Sebastian, an albino Burmese python. Photo by Emmanuel “Tambako” Keller

Not going away

Stackoverflow questions about Python 3

Third-party projects

The number of projects on the CheeseShop

Check the status

http://py3ksupport.appspot.com/
Currently 34% out of the top 50 projects has Python 3 support

Only support Python 3
● For end-user software and systems
● Run 2to3 once
● Then fix the code until it works.

Separate trees
● For stable Python libraries
● Make a branch or copy
● Run 2to3 once
● Fix until it works
● Bug fix both trees

How to distribute
Include both src2/ and src3/ and have different
parameters for different Python versions:

import sys
from distutils.core import setup

if sys.version < '3':
package_dir = {'': 'src2'}
else:
package_dir = {'': 'src3'}

setup(name='foo',
version='1.0',
package_dir = package_dir)

Continuous conversion with 2to3
● If you need to support Python 2 and Python 3
and the code is changing a lot
● Keep developing in Python 2
● Convert to Python 3 for testing and install
● Compatibility hacks may be needed
● Distribute helps

Single code base; no conversion
● If 2to3 for some reason is infeasible
● Support Python 2 and Python 3 from the
same code without 2to3
● Loads of compatibility hacks
● Fun, but ugly!
● Adding Python 2 support to Python 3 is often
less work than the other way around
● Check out six

Try with 2to3 first!
● Run 2to3 to see if it changes much
● If in doubt use Distribute!

When?
● For end-user systems: When you feel like it
● Not a good thing to do the week before
deployment
● Your customers want Python 3!
● For libraries: As soon as you can
● I.e. when your dependencies runs on Python 3

Get rid of warnings
● Run under Python 2.7 with -3
● Fix all deprecation warnings

● [Use iterator methods on dicts]

Prepare for bytes
● Use separate variables for binary data
● Add 'b' and 't' to file flags

Use key instead of cmp

>>> def compare(a, b):
... return cmp(a[1:], b[1:])

>>> names = ['Adam', 'Donald', 'John']
>>> names.sort(cmp=compare)
>>> names
['Adam', 'John', 'Donald']

Slow!
● The cmp function compares pairs
● Averages (from Jarret Hardie):
● 4 items: 6 calls (1.5 per item)
● 40,000 items: 342,541 calls (8.56 per item)

New way

>>> def keyfunction(item):
... return item[1:]

>>> names.sort(key=keyfunction)
>>> names

Fast and simple!
● The key method is called exactly one time per item
● The key method is simpler to implement than a
cmp method

Way simpler

>>> names.sort(key=len)
>>> names

>>> names = ['Adam', 'Benno', 'april']
>>> names.sort(key=str.lower)
>>> names
['Adam', 'april', 'Benno']

Use rich comparison methods
● No support for __cmp__ in Python 3
● Use rich comparison operators instead:
__lt__, __le__, __eq__,
__ge__, __gt__, __ne__
● Not entirely trivial to implement
● The @totalordering recipe is WRONG!

My mixin
class ComparableMixin(object):
def _compare(self, other, method):
try:
return method(self._cmpkey(), other._cmpkey())
except (AttributeError, TypeError):
# _cmpkey not implemented, or return different type,
# so I can't compare with "other".
return NotImplemented

def __lt__(self, other):
return self._compare(other, lambda s, o: s < o)

def __le__(self, other):
return self._compare(other, lambda s, o: s <= o)

def __eq__(self, other):
return self._compare(other, lambda s, o: s == o)

def __ge__(self, other):
return self._compare(other, lambda s, o: s >= o)

def __gt__(self, other):
return self._compare(other, lambda s, o: s > o)

def __ne__(self, other):
return self._compare(other, lambda s, o: s != o)

Write tests

Increasing your test coverage
will simplify porting and
increase confidence in the port

Running 2to3 manually

2to3 -w .
2to3 -w -d .
2to3 -w -d README.txt

Using 2to3 with Distribute

setup(name='mystuff',
...
test_suite='mystuff.tests.test_suite',
use_2to3=True,)

Non-ported imports

>>> from urllib2 import urlparse

Replacing UserDict
● UserDict.IterableUserDict →
collections.UserDict
● UserDict.DictMixin →
collections.MutableMapping
● UserDict.UserDict →
collections.UserDict

Bytes
vs
Strings
vs
Unicode

Bytes

Used for binary data:

>>> file = open('maybe_a.gif', 'rb')
>>> file.read(6) == 'GIF89a'
False

>>> file.read(6)
b'GIF89a'

Byte literals
>>> type(b'GIF89a')
<class 'bytes'>

>>> b'GIF89a'[2]
70

An alias for str in Python 2.6 and 2.7:

>>> type(b'GIF89a')
<class 'str'>

>>> b'GIF89a'[2]
'F'

Python < 2.6

>>> import sys
>>> if sys.version < '3':
... def b(x):
... return x
... else:
... def b(x):
... return x.encode('latin-1')

>>> b('binaryx67data')

Still behaves differently

>>> data = open('a.gif', 'rb').read()
>>> data[2]

Python 2: 'F'
Python 3: 70

Manipulating binary data
● >= 2.6: Use bytearray
● <= 2.5: Helper methods to iterate and index

Reading from files
● Python 2: Always 8-bit strings
● Python 3: 8-bit bytes or Unicode strings

The io module

>>> import io
>>> infile = io.open('UTF-8.txt', 'rt',
encoding='UTF-8')

The codecs module

>>> import codecs
>>> infile = codecs.open('UTF-8.txt',
'rt', encoding='UTF-8')

stdlib reorganisation

try:
from StringIO import StringIO
except ImportError:
from io import StringIO

Workaround for except:
● Python 2:
except Exception, e:
● Python 3:
except Exception as e:
● Python 2.6 and 2.7 handles both
● Python 3 and Python <= 2.5:
except Exception:
e = sys.exc_info()[1]

Workaround for print()
● For simple print cases:
print(“Something”) works everywhere.
print(“Something”, “else”) doesn't.
● Use print_() from the six module
● Or make your own

The return of

Bytes
vs
Strings
vs
Unicode

Python < 2.6

>>> import sys
>>> if sys.version < '3':
... def u(x):
... return x.decode(
... 'unicode_escape')
... else:
... def u(x):
... return x

>>> u('Unicxf6de')
Unicöde

Intcompability

1L no longer works

>>> import sys
>>> if sys.version > '3':
… long = int

>>> long(1)
1

C Extensions
● No 2to3
● Most differences can be handled with
#ifndefs and macros
● There is talk about making a compatibility
header file.

Python 2 module init

PyMODINIT_FUNC
initspam(void)
{
(void) Py_InitModule("spam", SpamMethods);
}

Python 3 module init

static struct PyModuleDef spammodule = {
PyModuleDef_HEAD_INIT,
"spam", /* name of module */
spam_doc, /* module documentation */
-1, /* size of per-interpreter state*/
SpamMethods
};

PyMODINIT_FUNC
PyInit_spam(void)
{
return PyModule_Create(&spammodule);
}

Writing your own fixers
● If you must modify the API
● Can be very complex due to edge cases
● Requires a whole talk by itself

2 * 3 = Six
● A module to help with compatibility
● Several compatibility methods:
● b() and u()
● print_()
● etc...
● Tons of helpful constants:
● integer_types
● string_types
● etc...

Additional Resources
● http://docs.python.org/py3k/howto/pyporting.html
● http://wiki.python.org/moin/PortingToPy3k/
● python-porting@python.org
● “Porting to Python 3”
● I'll be sprinting, come and ask!

KEALAVV7

http://python3porting.com/

Porting to Python 3

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Porting to Python 3

Similaire à Porting to Python 3 (20)

Dernier

Dernier (20)

Porting to Python 3