Welcome to the PyPy Demo Evening
Laurence Tratt
2013-08-27
1 / 5 http://soft-dev.org/

Why are we here?

What do we do?
(l-r) Vasudevan, Bolz, Tratt, Barrett, Diekmann

What do we do?
• Aim: identify important challenges in software
development.

What do we do?
development.
• Strengths: language design and implementation.

What do we do?
development.
• Immediate beneﬁts: faster VMs.

What do we do?
development.
• Immediate beneﬁts: faster VMs.
• Long-term beneﬁts: language composition.

This evening
1 Carl Friedrich Bolz PyPy overview.
2 Lukas Diekmann Storage strategies.
3 Maciej Fijalkowski NumPy.
4 Armin Rigo Software Transactional Memory
(STM).
5 Edd Barrett Language composition using
meta-tracing.

Can you help?
• Contributors.

Can you help?
• Contributors.
• Resources.
• Software Freedom Conservancy

A Very Brief Introduction to PyPy
Carl Friedrich Bolz
PyPy Demo Evening, King’s College London,
August 27, 2013
Carl Friedrich Bolz A Very Brief Introduction to PyPy

CPython is slow
CPython 1-3 orders of magnitude slower than C
BinaryTrees
Dhrystone
FannkuchRedux
Fasta
Knucleotide
Mandelbrot
Nbody
RegexDNA
RevComp
Richards
SpectralNorm
0.1
1
10
100
1000
SlowerthanC,lowerisbetter
C
Java
Cpython

Reasons for Bad Performance
interpretation overhead
late binding
dispatching
boxing

Enter PyPy
a modern efﬁcient implementation of Python

Enter PyPy
a modern efﬁcient implementation of Python
open source, MIT license
written in Python itself, then bootstrapped to C
uses a tracing JIT compiler to produce machine code at
runtime

Performance of PyPy
signiﬁcantly faster than CPython, typically in the same order of
magnitude than C
BinaryTrees
Dhrystone
FannkuchRedux
Fasta
Knucleotide
Mandelbrot
Nbody
RegexDNA
RevComp
Richards
SpectralNorm
0.1
1
10
100
1000
C
Java
PyPy
CPython

Performance of PyPy
signiﬁcantly faster than CPython, typically in the same order of
magnitude than C
BinaryTrees
Dhrystone
FannkuchRedux
Fasta
Knucleotide
Mandelbrot
Nbody
RegexDNA
RevComp
Richards
SpectralNorm
0.1
1
10
100
1000
C
Java
PyPy
CPython
on average about 6.3 faster than CPython

Demo

Architecture
Python interpreter written in RPython
JIT-compiler automatically added via meta-tracing

Status
Python 2.7.3 support, 2.7.4 coming soon
beta-level support for Python 3.2, more coming eventually
pure Python code fully supported, please report as bug if
not

Status
Python 2.7.3 support, 2.7.4 coming soon
beta-level support for Python 3.2, more coming eventually
pure Python code fully supported, please report as bug if
not
C extension modules partially supported, if they are
well-behaved
they are slow
use cfﬁ (a ctypes replacement) instead

Questions?
PyPy is a fast JITted Python implementation
(if something is not fast, please report it as a bug)
open source under MIT license
http://pypy.org

Storage Strategies for Fast Containers
Lukas Diekmann
August, 27 2013

Collection strategies
introduced in PyPy 1.9
optimisation of collections for certain data types
improving speed
reducing memory

Idea
typical programs have homogeneously types collections
create optimised versions of collections for certain types
so far:
lists: ints, ﬂoats, strings/unicode
sets: ints, ﬂoats, strings/unicode
dicts: ints, strings/unicode

Optimisations
collections automatically change to most eﬃcient strategy
store elements more memory eﬃciently
fast elements access

Further optimisations
collection creation and initalisation
split(d), set([1,2,3])
type based operations:
contains, diﬀerence, issubset
special strategies
RangeListStrategy: calculates elements on the ﬂy
Tracing JIT interaction:
faster (low-level) comparisons, remove type checks

Results
paper at OOPSLA
on average ∼18% speedup
∼6% less memory usage
more info at http://soft-dev.org/pubs/

Numpy on PyPy
Maciej Fijałkowski
King’s College London
August 27 2013
ﬁjal Numpy on PyPy

Goals
fully compliant numpy replacement for PyPy
fast looped operations
fast vectorized operations

Why?
fast looping
single language

Model
some programs have numerical kernels that can
be written in C
some don’t
http://arxiv.org/abs/1301.1334
image manipulation demo
abstraction unfriendly

Status
fast looped operations
ok vectorized operations

Future goals
ﬁnish numpy
make it fast
make it compatible with more software
(matplotlib, scipy)

Funding
about $20k left
we likely need more
behind schedule, but not behind budget

Q&A
Thank you!

Software Transactional Memory on PyPy

Pseudo-Goal
“Kill the GIL”
GIL = Global Interpreter Lock

Real Goals
Multi-core programming
But reasonable multi-core programming
Using the recent model of Transactional Memory

PyPy-STM
An executable pypy-stm which uses internally Software
Transactional Memory
Optimistically run multiple threads in parallel
The only new feature is atomic:
with atomic:
piece of code...

Example of higher-level API
def work(...):
...
several more calls to:
transaction.add(work, ...)
...
Starts N threads, scheduling work() calls to them
Each work() is done in an atomic block
Multi-core, but as if all the work() are done sequentially

Status
Kind of working without the JIT
Roughly three times slower (you need four cores to see beneﬁts)
Working on the JIT support

Q&A
Thank you!
Budget of $10k left, likely more needed too

Unipycation
Combining Prolog and Python
Edd Barrett
August 27, 2013

Our Goal
The softdev team is exploring language compositions.
Ideally our compositions should be:
Easy to implement.
Transparent (as possible) to the user-programmer.
High performance.
Can meta-tracing help?

Unipycation: A Language Composition Experiment
PyPy + Pyrolog = Unipycation
Unipycation
Both interpreters implemented in RPython.
About 600 LoC of integration code.
A few months to develop.
Languages communicate via an API.
No syntax integration yet.

Why?
Explore composition of opposing paradigms.
Evaluate RPython as a language composition framework.
Performance/ease of development.
Composition with many realistic applications.
Example Scenario
Data acquisition by JSON/XML/Sqlite.
Easy in Python, not easy in Prolog.
Some kind of knowledge inference based upon data.
Perhaps not so easy in Python, trivial in Prolog.
Visualisation of Results.
Easy in Python, lack of library support in Prolog.

Example
Suppose we have a directed graph (London Underground?):
a c
b
d
e
d
g
And we need to ask questions like:
Where can I get to from ’b’ via at most 4 nodes and how?

This is Easy with Prolog
path.pl:
edge(a, c). edge(c, b). edge(c, d). edge(d, e).
edge(b, e). edge(c, f). edge(f, g). edge(e, g).
edge(g, b).
path(From , To , MaxLen , Nodes) :-
path(From , To , MaxLen , Nodes , 1).
path(Node , Node , _, [Node], _).
path(From , To , MaxLen , [From | Ahead ], Len) :-
Len < MaxLen , edge(From , Next),
Len1 is Len + 1,
path(Next , To , MaxLen , Ahead , Len1).
query: path(b, To, 4, Path).

Example: Python → Prolog
from uni import Engine
engine = Engine.from_file (" path.pl")
paths = engine.db.path.iter
for (to , nodes) in paths ("b", None , 4, None):
print ("To %s via %s" % (to , nodes))
To b via [’b’]
To e via [’b’, ’e’]
To g via [’b’, ’e’, ’g’]
To b via [’b’, ’e’, ’g’, ’b’]
Calling from Prolog to Python also possible. E.g.
python:somefunc(blah)

Performance

In Summary
Compositions are relatively easy to implement with RPython.
We were able to implement a fairly transparent API-like interface.
Performance promising.
Further Reading
https://bitbucket.org/vext01/pypy
http://soft-dev.org/

Python jobs booking site campsites parks

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Python jobs booking site campsites parks

Similaire à Python jobs booking site campsites parks (20)

Dernier

Dernier (20)

Python jobs booking site campsites parks