Session on scraping by Gerald Quisumbing for PyCon APAC 2017

1. Beginner’s Guide to Scraping
PYCON APAC 2017 by Gerald Quisumbing

2. What is webscraping?
Is the process of extracting information from the
web using automated network software
Defined by intent not by technology

3. The scraping process

4. When should you scrape?
No API available
(anti) https://blog.hartleybrody.com/web-scraping/
No legal / Robot.txt restrictions
http://blog.icreon.us/advise/web-scraping-legality
https://benbernardblog.com/web-scraping-and-
crawling-are-perfectly-legal-
right/

5. Workshop Requirements
Requests
Mechanize*
Beautiful Soup
Lxml
*Only Python 2.x

6. Workshop proper
https://github.com/gtq/beginner-scraping

7. Where to go from here?
● Go Python 3 for better Unicode handling
● Invest in learning XPATH
● Javascript processing
(Splash, PhantomJS, Selenium)
● Try scrapy for larger projects
(like django for scraping)
● Stay Legal
(Copyright, Respect robots file)

8. Need to get in touch?
● http://www.linkedin.com/in/gerald-quisumbing
● Python Philippines FB Group
https://www.slideshare.net/gquisumbing/beginners-guide-to-
scraping
Image Credits
• Designed by Creativeart / Freepik
• Designed by 4045 / Freepik
• Designed by nevarpp / 123RF