SlideShare une entreprise Scribd logo
1  sur  27
Télécharger pour lire hors ligne
Faster Programming in
Python/Django
Subit Raj Pokharel
@ rajsubit,
rajsubit
First Code then optimize
Response Time Limits
● 0.1 seconds
○ limit for having the user feel that the system is reacting
instantaneously
● 1 seconds
○ limit for the user's flow of thought to stay uninterrupted
● 10 seconds
○ limit for keeping the user's attention focused on the dialogue
http://www.nngroup.com/articles/response-times-3-important-limits/
Time consuming function
New Relic
http://newrelic.com/
def find_recent_blog_post():
...
url = requests.get("http://blog.flipkarma.com/feed")
soup = BeautifulSoup.BeautifulSoup(url.text)
recent_post = soup.findAll('item')[:2]
...
A simple example
Analysis with Line Profiler
Line # Hits Time Per Hit % Time Line Contents
==============================================================
70 1 1112952 1112952.0 73.4 url = requests.get("http://blog.flipkarma.com/feed")
71 1 103695 103695.0 6.8 soup = BeautifulSoup.BeautifulSoup(url.text)
72 1 2186 2186.0 0.1 recent_post = soup.findAll('item')[:2]
https://github.com/rkern/line_profiler
$ pip install line_profiler
Optimization with Memcached
# Original code
url = requests.get("http://blog.flipkarma.com/feed")
soup = BeautifulSoup.BeautifulSoup(url.text)
recent_post = soup.findAll('item')[:2]
# Using Memcached
key = "flipkarma_blog_post"
cache_time = 10000
result = cache.get(key)
if result:
recent_post = result
else:
url = requests.get("http://blog.flipkarma.com/feed")
soup = BeautifulSoup.BeautifulSoup(url.text)
recent_post = soup.findAll('item')[:2]
cache.set(key, recent_post, cache_time)
http://memcached.org/
Line # Hits Time Per Hit % Time Line Contents
==============================================================
65 1 3 3.0 0.0 cache_time = 10000 # time to live in seconds
66 1 17973 17973.0 1.3 result = cache.get(cache_key)
67 1 4 4.0 0.0 if result:
68 1 31 31.0 0.0 parameters["result"] = result
69 1 3 3.0 0.0 if not result:
74 url = requests.get("http://blog.flipkarma.com/feed")
soup = BeautifulSoup.BeautifulSoup(url.text)
recent_post = soup.findAll('item')[:2]
cache.set(cache_key, recent_post, cache_time)
Result
New Relic
● analyze the application
● help understand the stories of application
data
● real-time business insights
http://newrelic.com/
Profiling
Python Tools:
● cProfile
● profile
● hotshot
Django Debug Toolbar
Line Profiler
Django Debug Toolbar
SQL Query
Query Time
Reducing Query
Query Time
Some Tips
If it's fast enough, don't optimise it.
Find the slowest step first.
Make the slowest operation faster
● Python functions have an overhead. Inline if possible
● List comprehensions are faster than for or if
Reduce the number of hits.
Perform each operation as rarely as
possible.
● Cache the result if speed is more important than memory
● Move slower operations inside if conditions
● assign operation to variable only if necessary
Reducing Queries
● Use of Class Based Views
● proper use of query sets
● select_related() and prefetch_related()
Change the algorithm.
This has the second-biggest impact on speed
The largest impact comes from
eliminating code.
Functionality is an asset. Code is a
liability.
Thank You

Contenu connexe

Similaire à Faster python/django programming

Garelic: Google Analytics as App Performance monitoring
Garelic: Google Analytics as App Performance monitoringGarelic: Google Analytics as App Performance monitoring
Garelic: Google Analytics as App Performance monitoring
Jano Suchal
 
Search Lucene
Search LuceneSearch Lucene
Search Lucene
Jeremy Coates
 
Performance Optimization of Rails Applications
Performance Optimization of Rails ApplicationsPerformance Optimization of Rails Applications
Performance Optimization of Rails Applications
Serge Smetana
 
Scrum Primer
Scrum PrimerScrum Primer
Scrum Primer
davelucey
 

Similaire à Faster python/django programming (20)

Faster Python Programs Through Optimization by Dr.-Ing Mike Muller
Faster Python Programs Through Optimization by Dr.-Ing Mike MullerFaster Python Programs Through Optimization by Dr.-Ing Mike Muller
Faster Python Programs Through Optimization by Dr.-Ing Mike Muller
 
PyGrunn2013 High Performance Web Applications with TurboGears
PyGrunn2013  High Performance Web Applications with TurboGearsPyGrunn2013  High Performance Web Applications with TurboGears
PyGrunn2013 High Performance Web Applications with TurboGears
 
Angular - Improve Runtime performance 2019
Angular - Improve Runtime performance 2019Angular - Improve Runtime performance 2019
Angular - Improve Runtime performance 2019
 
Creating a Smooth Development Workflow for High-Quality Modular Open-Source P...
Creating a Smooth Development Workflow for High-Quality Modular Open-Source P...Creating a Smooth Development Workflow for High-Quality Modular Open-Source P...
Creating a Smooth Development Workflow for High-Quality Modular Open-Source P...
 
Garelic: Google Analytics as App Performance monitoring
Garelic: Google Analytics as App Performance monitoringGarelic: Google Analytics as App Performance monitoring
Garelic: Google Analytics as App Performance monitoring
 
Journey through high performance django application
Journey through high performance django applicationJourney through high performance django application
Journey through high performance django application
 
[Srijan Wednesday Webinar] Easy Performance Wins for Your Rails App
[Srijan Wednesday Webinar] Easy Performance Wins for Your Rails App[Srijan Wednesday Webinar] Easy Performance Wins for Your Rails App
[Srijan Wednesday Webinar] Easy Performance Wins for Your Rails App
 
Search Lucene
Search LuceneSearch Lucene
Search Lucene
 
"Swoole: double troubles in c", Alexandr Vronskiy
"Swoole: double troubles in c", Alexandr Vronskiy"Swoole: double troubles in c", Alexandr Vronskiy
"Swoole: double troubles in c", Alexandr Vronskiy
 
PyCon AU 2012 - Debugging Live Python Web Applications
PyCon AU 2012 - Debugging Live Python Web ApplicationsPyCon AU 2012 - Debugging Live Python Web Applications
PyCon AU 2012 - Debugging Live Python Web Applications
 
Treasure Data Summer Internship 2016
Treasure Data Summer Internship 2016Treasure Data Summer Internship 2016
Treasure Data Summer Internship 2016
 
Performance Optimization of Rails Applications
Performance Optimization of Rails ApplicationsPerformance Optimization of Rails Applications
Performance Optimization of Rails Applications
 
Become a GC Hero
Become a GC HeroBecome a GC Hero
Become a GC Hero
 
Salesforce Apex Hours :- Hyper batch
Salesforce Apex Hours :- Hyper batchSalesforce Apex Hours :- Hyper batch
Salesforce Apex Hours :- Hyper batch
 
Design & Develop Batch Applications in Java/JEE
Design & Develop Batch Applications in Java/JEEDesign & Develop Batch Applications in Java/JEE
Design & Develop Batch Applications in Java/JEE
 
Scrum Primer
Scrum PrimerScrum Primer
Scrum Primer
 
EuroPython 2013 - Python3 TurboGears Training
EuroPython 2013 - Python3 TurboGears TrainingEuroPython 2013 - Python3 TurboGears Training
EuroPython 2013 - Python3 TurboGears Training
 
Building source code level profiler for C++.pdf
Building source code level profiler for C++.pdfBuilding source code level profiler for C++.pdf
Building source code level profiler for C++.pdf
 
Day 7 - Make it Fast
Day 7 - Make it FastDay 7 - Make it Fast
Day 7 - Make it Fast
 
jBPM5 Community Training Module 4: jBPM5 APIs Overview + Hands On
jBPM5 Community Training Module 4: jBPM5 APIs Overview + Hands OnjBPM5 Community Training Module 4: jBPM5 APIs Overview + Hands On
jBPM5 Community Training Module 4: jBPM5 APIs Overview + Hands On
 

Dernier

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@
 

Dernier (20)

Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
Ransomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdfRansomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdf
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 

Faster python/django programming

  • 1. Faster Programming in Python/Django Subit Raj Pokharel @ rajsubit, rajsubit
  • 2. First Code then optimize
  • 3. Response Time Limits ● 0.1 seconds ○ limit for having the user feel that the system is reacting instantaneously ● 1 seconds ○ limit for the user's flow of thought to stay uninterrupted ● 10 seconds ○ limit for keeping the user's attention focused on the dialogue http://www.nngroup.com/articles/response-times-3-important-limits/
  • 4. Time consuming function New Relic http://newrelic.com/
  • 5. def find_recent_blog_post(): ... url = requests.get("http://blog.flipkarma.com/feed") soup = BeautifulSoup.BeautifulSoup(url.text) recent_post = soup.findAll('item')[:2] ... A simple example
  • 6. Analysis with Line Profiler Line # Hits Time Per Hit % Time Line Contents ============================================================== 70 1 1112952 1112952.0 73.4 url = requests.get("http://blog.flipkarma.com/feed") 71 1 103695 103695.0 6.8 soup = BeautifulSoup.BeautifulSoup(url.text) 72 1 2186 2186.0 0.1 recent_post = soup.findAll('item')[:2] https://github.com/rkern/line_profiler $ pip install line_profiler
  • 7. Optimization with Memcached # Original code url = requests.get("http://blog.flipkarma.com/feed") soup = BeautifulSoup.BeautifulSoup(url.text) recent_post = soup.findAll('item')[:2] # Using Memcached key = "flipkarma_blog_post" cache_time = 10000 result = cache.get(key) if result: recent_post = result else: url = requests.get("http://blog.flipkarma.com/feed") soup = BeautifulSoup.BeautifulSoup(url.text) recent_post = soup.findAll('item')[:2] cache.set(key, recent_post, cache_time) http://memcached.org/
  • 8. Line # Hits Time Per Hit % Time Line Contents ============================================================== 65 1 3 3.0 0.0 cache_time = 10000 # time to live in seconds 66 1 17973 17973.0 1.3 result = cache.get(cache_key) 67 1 4 4.0 0.0 if result: 68 1 31 31.0 0.0 parameters["result"] = result 69 1 3 3.0 0.0 if not result: 74 url = requests.get("http://blog.flipkarma.com/feed") soup = BeautifulSoup.BeautifulSoup(url.text) recent_post = soup.findAll('item')[:2] cache.set(cache_key, recent_post, cache_time) Result
  • 9. New Relic ● analyze the application ● help understand the stories of application data ● real-time business insights http://newrelic.com/
  • 10.
  • 11. Profiling Python Tools: ● cProfile ● profile ● hotshot Django Debug Toolbar Line Profiler
  • 18. If it's fast enough, don't optimise it.
  • 19. Find the slowest step first.
  • 20. Make the slowest operation faster ● Python functions have an overhead. Inline if possible ● List comprehensions are faster than for or if
  • 21. Reduce the number of hits.
  • 22. Perform each operation as rarely as possible. ● Cache the result if speed is more important than memory ● Move slower operations inside if conditions ● assign operation to variable only if necessary
  • 23. Reducing Queries ● Use of Class Based Views ● proper use of query sets ● select_related() and prefetch_related()
  • 24. Change the algorithm. This has the second-biggest impact on speed
  • 25. The largest impact comes from eliminating code.
  • 26. Functionality is an asset. Code is a liability.