This document discusses the technologies used to build the Malaysian Bill Watcher application. It uses the Bottle microframework for the backend, Elasticsearch for search via the pyes Python client, and SQLAlchemy for database abstraction. Bootstrap is used for CSS and jQuery for JavaScript. Beautiful Soup is used for web scraping. The talk focuses on how Bottle templates and views work to return JSON and render templates, and how Elasticsearch is used to index and search scraped bill data via the indexer. Future plans include localization, new data sources, moving templates to Jinja, potentially migrating to MongoDB, and improving the scraping.

1. How We Use Bottle and
Elasticsearch
An Adventure in Sinar Project

2. A background of Malaysian Bill
Watcher
● The big idea, is to let citizen knows what is
happening in parliament, bills being debated
and pass
● No it doesn't know the bills that is not
debated yet.
● Its is scraper based, just because the
parliament site tend to be stand alone, and
nobody bother to go there.
● And cheat on twitter notification

3. The result
● Didn't quite worked out.
● That is not the point of this talk
● You can talk to us later on this

4. What we use?
● We use bottle micro framework.
● Elasticsearch via pyes
● sqlalchemy for db abstraction(seriously
thinking to move to mongodb instead, idea?)
● We use bootstrap for css and their js plugin
● with jquery
● Beaufifulsoup for scraping(that is our data
source)
We only going to cover pyes and bottle
(talk to me later for everything else)

5. Bottle
● A micro framework
● Similar to flask!!!
● Less feature though
● But it is OKAY.

6. Bottle a views
● https://github.com/Sinar/Malaysian-Bill-
Watcher/blob/master/billwatcher/pages.py
● https://github.com/Sinar/Malaysian-Bill-
Watcher/blob/master/billwatcher/pages.py

7. What happen?
● Bottle can return a dict, and it automatically
output json, if the object in dict is compatible
● Or with View Decorator, the dict can be feed
to a template

8. bottle the template
● https://github.com/Sinar/Malaysian-Bill-
Watcher/blob/master/billwatcher/views/list.tpl
●

9. What happen?
● Bottle read template from a dict/object
● like most templating language does.
● Have some logic like other, different syntax
same idea
● Don't have good inheritance though

10. elasticsearch!!!
● OK I lied, this is written in JAVA
● But hey, it got RESTful API
● and pyes did a lot of the stuff for us already

11. Before index
● before we do that
● we don't need to create a schema
● But we do it because we can control the
search priority

12. Indexer
● https://github.com/Sinar/Malaysian-Bill-
Watcher/blob/master/billwatcher/indexer.py

13. then we index
● First flatten your result
● convert to dict
● index

14. The indexer
● https://github.com/Sinar/Malaysian-Bill-
Watcher/blob/master/billwatcher/indexer.py
● https://github.com/Sinar/Malaysian-Bill-
Watcher/blob/master/billwatcher/loader.py

15. now we search!!!!
● There is some field inside the search result
● https://github.com/Sinar/Malaysian-Bill-
Watcher/blob/master/billwatcher/pages.py

16. The future
● Do localization
● Find new data source
● Move template to jinja
● Seriously thinking of moving to mongodb
● Seriously still trying to see a better way to
scrape websites