SlideShare a Scribd company logo
1 of 12
Download to read offline
Cedric Gatay - c.gatay@code-troopers.com
or HTML to REST API guide
APIs for everyone
Cedric Gatay - c.gatay@code-troopers.com
Initial rant
• Want to build an awesome app	

• On top of third party service	

• No API, only HTML available
2
Cedric Gatay - c.gatay@code-troopers.com
Basic idea
• Emulate a browser to issue requests	

• Parse resulting HTML	

• Transform it into a REST API
3
Cedric Gatay - c.gatay@code-troopers.com
Prerequisites
• Analyze site to find endpoints	

• mobile versions are great	

• Reverse engineer server requests	

• understand parameters might be tricky
4
Cedric Gatay - c.gatay@code-troopers.com
Tools
• Wireshark to analyze HTTP
5
Cedric Gatay - c.gatay@code-troopers.com
Tools
• Chrome Inspector to the rescue
6
Cedric Gatay - c.gatay@code-troopers.com
Tools
• Gargl : https://github.com/jodoglevy/gargl	

• Record your requests	

• Name them	

• Parametrize them	

• Generate Java/JavaScript code
7
Cedric Gatay - c.gatay@code-troopers.com
From HTML to API
• Gargl generates boilerplate for HTTP calls	

• Refactor generated code	

• Need to parse HTML	

• Expose it through API
8
Cedric Gatay - c.gatay@code-troopers.com
Parse HTML
• JSoup : http://jsoup.org/	

• simple parser	

• allows to use css selectors
9
Cedric Gatay - c.gatay@code-troopers.com
Expose API
• Define verbs	

• Use your preferred framework here	

• restx : http://www.restx.io	

• Test extensively	

• parsing WILL break
10
Cedric Gatay - c.gatay@code-troopers.com
Cache API
• Do not spam base server	

• Cache external calls as much as possible	

• Store constant data
11
Cedric Gatay - c.gatay@code-troopers.com
Questions ?
Thanks for listening
12

More Related Content

Similar to HTML to REST API thoughts

APIs_ An Introduction.pptx
APIs_ An Introduction.pptxAPIs_ An Introduction.pptx
APIs_ An Introduction.pptx
AkashThorat25
 
PaaS - google app engine
PaaS  - google app enginePaaS  - google app engine
PaaS - google app engine
J Singh
 
Documentation-driven development for Python web APIs v2
Documentation-driven development for Python web APIs v2Documentation-driven development for Python web APIs v2
Documentation-driven development for Python web APIs v2
José Haro Peralta
 

Similar to HTML to REST API thoughts (20)

De la bonne utilisation de OAuth2
De la bonne utilisation de OAuth2 De la bonne utilisation de OAuth2
De la bonne utilisation de OAuth2
 
Brushing skills on SignalR for ASP.NET developers
Brushing skills on SignalR for ASP.NET developersBrushing skills on SignalR for ASP.NET developers
Brushing skills on SignalR for ASP.NET developers
 
Engineering Student MuleSoft Meetup#3 - API Implementation using APIKIT route...
Engineering Student MuleSoft Meetup#3 - API Implementation using APIKIT route...Engineering Student MuleSoft Meetup#3 - API Implementation using APIKIT route...
Engineering Student MuleSoft Meetup#3 - API Implementation using APIKIT route...
 
Take a Groovy REST
Take a Groovy RESTTake a Groovy REST
Take a Groovy REST
 
Angular SEO
Angular SEO Angular SEO
Angular SEO
 
Introduction to GitHub Copilot
Introduction to GitHub CopilotIntroduction to GitHub Copilot
Introduction to GitHub Copilot
 
Datasets, APIs, and Web Scraping
Datasets, APIs, and Web ScrapingDatasets, APIs, and Web Scraping
Datasets, APIs, and Web Scraping
 
Hackolade Tutorial - part 12 - Create a REST API model
Hackolade Tutorial - part  12 - Create a REST API modelHackolade Tutorial - part  12 - Create a REST API model
Hackolade Tutorial - part 12 - Create a REST API model
 
Criando API's com HapiJS
Criando API's com HapiJSCriando API's com HapiJS
Criando API's com HapiJS
 
Web Scraping with Python
Web Scraping with PythonWeb Scraping with Python
Web Scraping with Python
 
Computer networking mcis 6163 project
Computer networking mcis 6163 projectComputer networking mcis 6163 project
Computer networking mcis 6163 project
 
APIs_ An Introduction.pptx
APIs_ An Introduction.pptxAPIs_ An Introduction.pptx
APIs_ An Introduction.pptx
 
Serverless authentication
Serverless authenticationServerless authentication
Serverless authentication
 
PaaS - google app engine
PaaS  - google app enginePaaS  - google app engine
PaaS - google app engine
 
Expanding APIs beyond the Web
Expanding APIs beyond the WebExpanding APIs beyond the Web
Expanding APIs beyond the Web
 
Webkit Chromium Contribution Process
Webkit Chromium Contribution ProcessWebkit Chromium Contribution Process
Webkit Chromium Contribution Process
 
Code for Startup MVP (Ruby on Rails) Session 1
Code for Startup MVP (Ruby on Rails) Session 1Code for Startup MVP (Ruby on Rails) Session 1
Code for Startup MVP (Ruby on Rails) Session 1
 
Building Awesome APIs with Lumen
Building Awesome APIs with LumenBuilding Awesome APIs with Lumen
Building Awesome APIs with Lumen
 
KSDG LT: 6分鐘node.JS就上手
KSDG LT: 6分鐘node.JS就上手KSDG LT: 6分鐘node.JS就上手
KSDG LT: 6分鐘node.JS就上手
 
Documentation-driven development for Python web APIs v2
Documentation-driven development for Python web APIs v2Documentation-driven development for Python web APIs v2
Documentation-driven development for Python web APIs v2
 

Recently uploaded

notes on Evolution Of Analytic Scalability.ppt
notes on Evolution Of Analytic Scalability.pptnotes on Evolution Of Analytic Scalability.ppt
notes on Evolution Of Analytic Scalability.ppt
MsecMca
 
Call Girls in Ramesh Nagar Delhi 💯 Call Us 🔝9953056974 🔝 Escort Service
Call Girls in Ramesh Nagar Delhi 💯 Call Us 🔝9953056974 🔝 Escort ServiceCall Girls in Ramesh Nagar Delhi 💯 Call Us 🔝9953056974 🔝 Escort Service
Call Girls in Ramesh Nagar Delhi 💯 Call Us 🔝9953056974 🔝 Escort Service
9953056974 Low Rate Call Girls In Saket, Delhi NCR
 
VIP Call Girls Palanpur 7001035870 Whatsapp Number, 24/07 Booking
VIP Call Girls Palanpur 7001035870 Whatsapp Number, 24/07 BookingVIP Call Girls Palanpur 7001035870 Whatsapp Number, 24/07 Booking
VIP Call Girls Palanpur 7001035870 Whatsapp Number, 24/07 Booking
dharasingh5698
 
Top Rated Call Girls In chittoor 📱 {7001035870} VIP Escorts chittoor
Top Rated Call Girls In chittoor 📱 {7001035870} VIP Escorts chittoorTop Rated Call Girls In chittoor 📱 {7001035870} VIP Escorts chittoor
Top Rated Call Girls In chittoor 📱 {7001035870} VIP Escorts chittoor
dharasingh5698
 

Recently uploaded (20)

notes on Evolution Of Analytic Scalability.ppt
notes on Evolution Of Analytic Scalability.pptnotes on Evolution Of Analytic Scalability.ppt
notes on Evolution Of Analytic Scalability.ppt
 
Booking open Available Pune Call Girls Pargaon 6297143586 Call Hot Indian Gi...
Booking open Available Pune Call Girls Pargaon  6297143586 Call Hot Indian Gi...Booking open Available Pune Call Girls Pargaon  6297143586 Call Hot Indian Gi...
Booking open Available Pune Call Girls Pargaon 6297143586 Call Hot Indian Gi...
 
Call Girls in Ramesh Nagar Delhi 💯 Call Us 🔝9953056974 🔝 Escort Service
Call Girls in Ramesh Nagar Delhi 💯 Call Us 🔝9953056974 🔝 Escort ServiceCall Girls in Ramesh Nagar Delhi 💯 Call Us 🔝9953056974 🔝 Escort Service
Call Girls in Ramesh Nagar Delhi 💯 Call Us 🔝9953056974 🔝 Escort Service
 
Call Girls Wakad Call Me 7737669865 Budget Friendly No Advance Booking
Call Girls Wakad Call Me 7737669865 Budget Friendly No Advance BookingCall Girls Wakad Call Me 7737669865 Budget Friendly No Advance Booking
Call Girls Wakad Call Me 7737669865 Budget Friendly No Advance Booking
 
Intro To Electric Vehicles PDF Notes.pdf
Intro To Electric Vehicles PDF Notes.pdfIntro To Electric Vehicles PDF Notes.pdf
Intro To Electric Vehicles PDF Notes.pdf
 
VIP Call Girls Palanpur 7001035870 Whatsapp Number, 24/07 Booking
VIP Call Girls Palanpur 7001035870 Whatsapp Number, 24/07 BookingVIP Call Girls Palanpur 7001035870 Whatsapp Number, 24/07 Booking
VIP Call Girls Palanpur 7001035870 Whatsapp Number, 24/07 Booking
 
Booking open Available Pune Call Girls Koregaon Park 6297143586 Call Hot Ind...
Booking open Available Pune Call Girls Koregaon Park  6297143586 Call Hot Ind...Booking open Available Pune Call Girls Koregaon Park  6297143586 Call Hot Ind...
Booking open Available Pune Call Girls Koregaon Park 6297143586 Call Hot Ind...
 
Call Girls Pimpri Chinchwad Call Me 7737669865 Budget Friendly No Advance Boo...
Call Girls Pimpri Chinchwad Call Me 7737669865 Budget Friendly No Advance Boo...Call Girls Pimpri Chinchwad Call Me 7737669865 Budget Friendly No Advance Boo...
Call Girls Pimpri Chinchwad Call Me 7737669865 Budget Friendly No Advance Boo...
 
Unit 2- Effective stress & Permeability.pdf
Unit 2- Effective stress & Permeability.pdfUnit 2- Effective stress & Permeability.pdf
Unit 2- Effective stress & Permeability.pdf
 
Generative AI or GenAI technology based PPT
Generative AI or GenAI technology based PPTGenerative AI or GenAI technology based PPT
Generative AI or GenAI technology based PPT
 
Block diagram reduction techniques in control systems.ppt
Block diagram reduction techniques in control systems.pptBlock diagram reduction techniques in control systems.ppt
Block diagram reduction techniques in control systems.ppt
 
Thermal Engineering -unit - III & IV.ppt
Thermal Engineering -unit - III & IV.pptThermal Engineering -unit - III & IV.ppt
Thermal Engineering -unit - III & IV.ppt
 
Top Rated Pune Call Girls Budhwar Peth ⟟ 6297143586 ⟟ Call Me For Genuine Se...
Top Rated  Pune Call Girls Budhwar Peth ⟟ 6297143586 ⟟ Call Me For Genuine Se...Top Rated  Pune Call Girls Budhwar Peth ⟟ 6297143586 ⟟ Call Me For Genuine Se...
Top Rated Pune Call Girls Budhwar Peth ⟟ 6297143586 ⟟ Call Me For Genuine Se...
 
Intze Overhead Water Tank Design by Working Stress - IS Method.pdf
Intze Overhead Water Tank  Design by Working Stress - IS Method.pdfIntze Overhead Water Tank  Design by Working Stress - IS Method.pdf
Intze Overhead Water Tank Design by Working Stress - IS Method.pdf
 
Top Rated Call Girls In chittoor 📱 {7001035870} VIP Escorts chittoor
Top Rated Call Girls In chittoor 📱 {7001035870} VIP Escorts chittoorTop Rated Call Girls In chittoor 📱 {7001035870} VIP Escorts chittoor
Top Rated Call Girls In chittoor 📱 {7001035870} VIP Escorts chittoor
 
data_management_and _data_science_cheat_sheet.pdf
data_management_and _data_science_cheat_sheet.pdfdata_management_and _data_science_cheat_sheet.pdf
data_management_and _data_science_cheat_sheet.pdf
 
CCS335 _ Neural Networks and Deep Learning Laboratory_Lab Complete Record
CCS335 _ Neural Networks and Deep Learning Laboratory_Lab Complete RecordCCS335 _ Neural Networks and Deep Learning Laboratory_Lab Complete Record
CCS335 _ Neural Networks and Deep Learning Laboratory_Lab Complete Record
 
(INDIRA) Call Girl Aurangabad Call Now 8617697112 Aurangabad Escorts 24x7
(INDIRA) Call Girl Aurangabad Call Now 8617697112 Aurangabad Escorts 24x7(INDIRA) Call Girl Aurangabad Call Now 8617697112 Aurangabad Escorts 24x7
(INDIRA) Call Girl Aurangabad Call Now 8617697112 Aurangabad Escorts 24x7
 
NFPA 5000 2024 standard .
NFPA 5000 2024 standard                                  .NFPA 5000 2024 standard                                  .
NFPA 5000 2024 standard .
 
chapter 5.pptx: drainage and irrigation engineering
chapter 5.pptx: drainage and irrigation engineeringchapter 5.pptx: drainage and irrigation engineering
chapter 5.pptx: drainage and irrigation engineering
 

HTML to REST API thoughts

  • 1. Cedric Gatay - c.gatay@code-troopers.com or HTML to REST API guide APIs for everyone
  • 2. Cedric Gatay - c.gatay@code-troopers.com Initial rant • Want to build an awesome app • On top of third party service • No API, only HTML available 2
  • 3. Cedric Gatay - c.gatay@code-troopers.com Basic idea • Emulate a browser to issue requests • Parse resulting HTML • Transform it into a REST API 3
  • 4. Cedric Gatay - c.gatay@code-troopers.com Prerequisites • Analyze site to find endpoints • mobile versions are great • Reverse engineer server requests • understand parameters might be tricky 4
  • 5. Cedric Gatay - c.gatay@code-troopers.com Tools • Wireshark to analyze HTTP 5
  • 6. Cedric Gatay - c.gatay@code-troopers.com Tools • Chrome Inspector to the rescue 6
  • 7. Cedric Gatay - c.gatay@code-troopers.com Tools • Gargl : https://github.com/jodoglevy/gargl • Record your requests • Name them • Parametrize them • Generate Java/JavaScript code 7
  • 8. Cedric Gatay - c.gatay@code-troopers.com From HTML to API • Gargl generates boilerplate for HTTP calls • Refactor generated code • Need to parse HTML • Expose it through API 8
  • 9. Cedric Gatay - c.gatay@code-troopers.com Parse HTML • JSoup : http://jsoup.org/ • simple parser • allows to use css selectors 9
  • 10. Cedric Gatay - c.gatay@code-troopers.com Expose API • Define verbs • Use your preferred framework here • restx : http://www.restx.io • Test extensively • parsing WILL break 10
  • 11. Cedric Gatay - c.gatay@code-troopers.com Cache API • Do not spam base server • Cache external calls as much as possible • Store constant data 11
  • 12. Cedric Gatay - c.gatay@code-troopers.com Questions ? Thanks for listening 12