The document discusses transforming HTML from a third party service without a public API into a REST API. It recommends emulating a browser to issue HTTP requests and parse the resulting HTML. Tools like Wireshark, Chrome Inspector and Gargl can help analyze and record requests. The HTML can then be parsed using a library like JSoup and exposed through a framework to define API endpoints and verbs. Caching is also important to avoid overloading the third party service.
chapter 5.pptx: drainage and irrigation engineering
HTML to REST API thoughts
1. Cedric Gatay - c.gatay@code-troopers.com
or HTML to REST API guide
APIs for everyone
2. Cedric Gatay - c.gatay@code-troopers.com
Initial rant
• Want to build an awesome app
• On top of third party service
• No API, only HTML available
2
3. Cedric Gatay - c.gatay@code-troopers.com
Basic idea
• Emulate a browser to issue requests
• Parse resulting HTML
• Transform it into a REST API
3
4. Cedric Gatay - c.gatay@code-troopers.com
Prerequisites
• Analyze site to find endpoints
• mobile versions are great
• Reverse engineer server requests
• understand parameters might be tricky
4
6. Cedric Gatay - c.gatay@code-troopers.com
Tools
• Chrome Inspector to the rescue
6
7. Cedric Gatay - c.gatay@code-troopers.com
Tools
• Gargl : https://github.com/jodoglevy/gargl
• Record your requests
• Name them
• Parametrize them
• Generate Java/JavaScript code
7
8. Cedric Gatay - c.gatay@code-troopers.com
From HTML to API
• Gargl generates boilerplate for HTTP calls
• Refactor generated code
• Need to parse HTML
• Expose it through API
8
9. Cedric Gatay - c.gatay@code-troopers.com
Parse HTML
• JSoup : http://jsoup.org/
• simple parser
• allows to use css selectors
9
10. Cedric Gatay - c.gatay@code-troopers.com
Expose API
• Define verbs
• Use your preferred framework here
• restx : http://www.restx.io
• Test extensively
• parsing WILL break
10
11. Cedric Gatay - c.gatay@code-troopers.com
Cache API
• Do not spam base server
• Cache external calls as much as possible
• Store constant data
11