Talk by Didrik Pinte, MD at Enthought at Data Science London meetup 18/10/12

1. Numpy, the Python
foundation for number
crunching
Didrik Pinte, Enthought
London Data Science meetup
Monday 22 October 2012

2. Number crunching?
•High-level api
•Interactivity & visualization
•Performance
•Low-level access
Monday 22 October 2012

3. Evidence ?
Clyther
PyOpenGL
PyCuda
PyAlgoTrade
PyTrilinos
Joblib numpy-boost
petsc4py
Monday 22 October 2012

4. Evidence ?
Clyther
PyOpenGL
PyCuda
PyAlgoTrade
PyTrilinos
Joblib numpy-boost
petsc4py
Monday 22 October 2012

5. Why then?
•The API ...
•Simple but powerful memory model
•Open access to the data
Monday 22 October 2012

6. Array data structure
Monday 22 October 2012

7. Let’s look at the code!
•Examples:
– API / interactivity
– memory management with stride_tricks
– pikos
– extensions with talib
–(joblib, memmap/multiprocessing,
ipython //)
Monday 22 October 2012

8. API / interactivity
Monday 22 October 2012

9. Memory management
1 2 3 4 5 6 7 8 9 10 11 12
=
1 2 3 4
2 3 4 5
3 4 5 6
4 5 6 7
5 6 7 8
6 7 8 9
7 8 9 10
8 9 10 11
...
Monday 22 October 2012

10. Memory management
Shape 12,
Strides 8,
1 2 3 4 5 6 7 8 9 10 11 12
=
1 2 3 4
Shape 9,4, 2 3 4 5
Strides 8,8 3 4 5 6
4 5 6 7
5 6 7 8
6 7 8 9
7 8 9 10
8 9 10 11
...
Monday 22 October 2012

11. Memory management - pikos
Monday 22 October 2012

12. Memory management - chaco
Monday 22 October 2012

13. Low level access
%timeit talib.moving_average(adj_close, optInTimePeriod=5)
100000 loops, best of 3: 7.67 us per loop
%timeit as_strided(adj_close, shape=(len(adj_close)-4, 5), strides=(8, 8)).mean
(axis=1)
10000 loops, best of 3: 28.2 us per loop
Monday 22 October 2012

14. Conclusion
•It’s obvious, no?
Monday 22 October 2012

15. Q&A?
Monday 22 October 2012