SF USGS Earthquakes via Twitter: How to make a simple Twitter bot to aggregate news.

1. @sfusgs Geo-tweet my quake… @ayman • yahoo! research http://en.wikipedia.org/wiki/File:San_francisco_1906_earthquake.jpg

2. Zync • Y! Messenger

3. FireEagle

4. FireEagle @ayman

5. http://bit.ly/statler

6. shake and rumble in 2GA

7. http://earthquake.usgs.gov/earthquakes/recenteqsus/Quakes/nc71336726.php

8. Click through maps…

9. A Want Ad

10. USGS lists every quake

11. http://earthquake.usgs.gov/earthquakes/catalogs/

12. http://earthquake.usgs.gov/earthquakes/catalogs/

13. Trust me, it is just XML

14. Yahoo! Search goes to YDN

15. Google Search goes to YDN

16. Bing goes to YDN

17. Using ElementTree

18. The flow was simple CRON this:: cUrl XML feed from USGS find local quakes bit.ly the quake permalink if not duplicate record this quake tweet this quake

19. def fetch_quakes(): def fetch_quakes(): api = bitly.Api(login=’ABCDEF', apikey=’12345678901234567890') rss = parse(urllib.urlopen(USGS_URL)).getroot() quakes = [] for element in rss.findall('{%s}entry' % USGS_NS): title = element.findtext('{%s}title' % USGS_NS) if (title.find("San Francisco Bay") > -1): point = element.findtext('{%s}point' % GEORSS_NS) points = point.split(None, 1) link = element.find('{%s}link' % USGS_NS) href = 'http://earthquakes.usgs.gov%s' % link.get('href') href = api.shorten(href) if (is_duplicate(href) is False): f = open(‘urls.txt', 'a') f.write('%s' % href) f.close() summary = element.findtext('{%s}summary' % USGS_NS) summary = summary.split("<br>")[1] summary = summary.replace(' at epicenter</p><p>', ' ') summary = strip_tags(summary) quakes.append({'title': element.findtext('{%s}title' % USGS_NS), 'link' : href, 'updated' : element.findtext('{%s}updated' % USGS_NS), 'long' : points[1], 'lat' : points[0], 'elev' : element.findtext('{%s}elev' % GEORSS_NS), 'summary' : summary}) return quakes

20. def is_duplicate(url): def is_duplicate(url): from subprocess import call if(os.path.exists(‘urls.txt') is False): call(['/usr/bin/touch', ‘urls.txt']) r = call(['/bin/fgrep', url, ‘urls.txt']) if (r == 0): return True else: return False

21. def send_tweet(t): def send_tweet(t): import httplib2 username = ’ABCDEFG' password = ’12345678901234567890' msg = ('%s %s %s' % (t['title'], t['link'], t['summary'])) http = httplib2.Http() http.add_credentials(username, password) response = http.request( "http://twitter.com/statuses/update.xml", "POST", urllib.urlencode({"status": msg, "lat": t['lat'], "long": t['long']}) ) SEE: http://code.google.com/p/httplib2/ ps: you shouldn’t store your passwords like that…this is just an illustration

22. fin

23. Crunched

24. USGS on Open Gov’t

25. Open Questions? How can other local sources be disseminated? How can mass media be filtered so it can be consumed in a similar manner? Can we collect information around the news? For example, find all the tweets that relate to a particular quake. Thanks, @ayman.