Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Hacking the browser with puppeteer sharp .NET conf AR 2018

180 vues

Publié le

Hacking the browser with puppeteer sharp .NET conf AR 2018

Publié dans : Logiciels
  • Soyez le premier à commenter

Hacking the browser with puppeteer sharp .NET conf AR 2018

  1. 1. Hacking the Browser With Puppeteer-Sharp Darío Kondratiuk .NET Senior Developer - MultiTracks.com Author of Puppeteer-Sharp @kblok - @hardkoded www.hardkoded.com
  2. 2. Hacking the Browser With Puppeteer-Sharp Darío Kondratiuk .NET Senior Developer @ MultiTracks.com Author of Puppeteer-Sharp @hardkoded - @kblok www.hardkoded.com https://developer.mozilla.org/en-US/docs/Web/WebDriver October 4th, 5th & 6th 2018.NET Conf AR v2018
  3. 3. Headless Browsers https://developer.mozilla.org/en-US/docs/Web/WebDriver October 4th, 5th & 6th 2018.NET Conf AR v2018
  4. 4. Headless Browsers ● July 4, 2017 => Google Chrome 59 ○ chrome --headless --disable-gpu --print-to-pdf https://www.chromestatus.com/ https://developer.mozilla.org/en-US/docs/Web/WebDriver October 4th, 5th & 6th 2018.NET Conf AR v2018
  5. 5. https://developer.mozilla.org/en-US/docs/Web/WebDriver October 4th, 5th & 6th 2018.NET Conf AR v2018
  6. 6. https://developer.mozilla.org/en-US/docs/Web/WebDriver October 4th, 5th & 6th 2018.NET Conf AR v2018
  7. 7. Headless Browsers ● July 4, 2017 => Google Chrome 59 ○ chrome --headless --disable-gpu --print-to-pdf https://www.chromestatus.com/ ● August 8, 2017 => Firefox 55 ● August 16, 2017 => Puppeteer v0.9 ● January 12, 2018 => Puppeteer v1.0 ● March 1, 2018 => Puppeteer Sharp v0.1 ● April 27, 2018 => Edge DevTools Protocol v0.1 https://developer.mozilla.org/en-US/docs/Web/WebDriver October 4th, 5th & 6th 2018.NET Conf AR v2018
  8. 8. https://developer.mozilla.org/en-US/docs/Web/WebDriver October 4th, 5th & 6th 2018.NET Conf AR v2018
  9. 9. WebDriver vs Headless Browsers https://developer.mozilla.org/en-US/docs/Web/WebDriver October 4th, 5th & 6th 2018.NET Conf AR v2018
  10. 10. WebDriver WebDriver is a remote control interface that enables introspection and control of user agents. It provides a platform- and language-neutral wire protocol as a way for out- of-process programs to remotely instruct the behavior of web browsers. https://developer.mozilla.org/en-US/docs/Web/WebDriver October 4th, 5th & 6th 2018.NET Conf AR v2018
  11. 11. https://developer.mozilla.org/en-US/docs/Web/WebDriver https://developer.mozilla.org/en-US/docs/Web/WebDriver October 4th, 5th & 6th 2018.NET Conf AR v2018
  12. 12. https://developer.mozilla.org/en-US/docs/Web/WebDriver October 4th, 5th & 6th 2018.NET Conf AR v2018
  13. 13. Use Cases https://developer.mozilla.org/en-US/docs/Web/WebDriver October 4th, 5th & 6th 2018.NET Conf AR v2018
  14. 14. Screenshots https://developer.mozilla.org/en-US/docs/Web/WebDriver October 4th, 5th & 6th 2018.NET Conf AR v2018
  15. 15. https://developer.mozilla.org/en-US/docs/Web/WebDriver October 4th, 5th & 6th 2018.NET Conf AR v2018
  16. 16. https://developer.mozilla.org/en-US/docs/Web/WebDriver October 4th, 5th & 6th 2018.NET Conf AR v2018
  17. 17. [HttpGet("{owner}/{repo}")] public async Task<FileContentResult> Get(string owner, string repo) { var contributorsPage = $"https://github.com/{owner}/{repo}/graphs/contributors"; using (var browser = await Puppeteer.LaunchAsync(new LaunchOptions { Headless = false })) using (var page = await browser.NewPageAsync()) { await page.GoToAsync(contributorsPage); await page.WaitForSelectorAsync(".contrib-person"); var element = await page.QuerySelectorAsync("#contributors"); var image = await element.ScreenshotDataAsync(); return File(image, "image/png"); } }
  18. 18. PDFs https://developer.mozilla.org/en-US/docs/Web/WebDriver October 4th, 5th & 6th 2018.NET Conf AR v2018
  19. 19. [HttpGet("{owner}/{post}")] public async Task<FileContentResult> Get(string author, string post) { var contributorsPage = $"https://medium.com/{author}/{post}"; using (var browser = await Puppeteer.LaunchAsync(new LaunchOptions { Headless = true })) using (var page = await browser.NewPageAsync()) { await page.GoToAsync(contributorsPage); await page.WaitForSelectorAsync("HEADER"); await page.EvaluateExpressionAsync("document.querySelector('HEADER').remove();"); var pdf = await page.PdfDataAsync(); return File(pdf, "application/pdf"); } }
  20. 20. Web Scraping https://developer.mozilla.org/en-US/docs/Web/WebDriver October 4th, 5th & 6th 2018.NET Conf AR v2018
  21. 21. https://developer.mozilla.org/en-US/docs/Web/WebDriver October 4th, 5th & 6th 2018.NET Conf AR v2018
  22. 22. https://developer.mozilla.org/en-US/docs/Web/WebDriver October 4th, 5th & 6th 2018.NET Conf AR v2018
  23. 23. var url = "https://www.despegar.com.ar/shop/flights/results/roundtrip/BUE/MDZ/2018-12-01/2018-12-08/1"; using (var browser = await Puppeteer.LaunchAsync(options)) using (var page = await browser.NewPageAsync()) { await page.GoToAsync(url, WaitUntilNavigation.Networkidle0); await page.WaitForSelectorAsync("buy-button"); var bestPrice = await page.EvaluateFunctionAsync<string>(@"() => { var elements = document.querySelectorAll('.main-content .price-amount'); return elements.length ? elements[0].innerText : '0'; }"); Console.WriteLine($"Best price for Mendoza {bestPrice}"); await Task.Delay(60000); }
  24. 24. UI Testing https://developer.mozilla.org/en-US/docs/Web/WebDriver October 4th, 5th & 6th 2018.NET Conf AR v2018
  25. 25. [Fact] public async Task ShouldHonorThePrice() { //Previous Code var clickElement = await page.EvaluateExpressionHandleAsync(@" document.querySelectorAll('.main-content buy-button:first-child A')[0]") as ElementHandle; await clickElement.ClickAsync(); await page.WaitForSelectorAsync(".price-container .amount"); var checkoutPrice = await page.EvaluateExpressionAsync<string>(@" document.querySelectorAll('.price-container .amount')[0].innerText "); Assert.Equal(bestPrice, checkoutPrice); }
  26. 26. Task Automation https://developer.mozilla.org/en-US/docs/Web/WebDriver October 4th, 5th & 6th 2018.NET Conf AR v2018
  27. 27. https://developer.mozilla.org/en-US/docs/Web/WebDriver October 4th, 5th & 6th 2018.NET Conf AR v2018
  28. 28. DEMO https://developer.mozilla.org/en-US/docs/Web/WebDriver October 4th, 5th & 6th 2018.NET Conf AR v2018
  29. 29. More examples https://github.com/GoogleChromeLabs/puppeteer-examples https://developer.mozilla.org/en-US/docs/Web/WebDriver October 4th, 5th & 6th 2018.NET Conf AR v2018
  30. 30. Even more examples https://github.com/checkly/puppeteer-examples https://developer.mozilla.org/en-US/docs/Web/WebDriver October 4th, 5th & 6th 2018.NET Conf AR v2018
  31. 31. Don’ts ● DDoS Attacks ● Unethical Web Scraping ● Fake page loads ● Credential Stuffing https://developer.mozilla.org/en-US/docs/Web/WebDriver October 4th, 5th & 6th 2018.NET Conf AR v2018
  32. 32. Puppeteer the world! ● Puppeteer Recorder ● Rendertron ● Checkly ● Contributors! https://developer.mozilla.org/en-US/docs/Web/WebDriver October 4th, 5th & 6th 2018.NET Conf AR v2018
  33. 33. Give Back https://developer.mozilla.org/en-US/docs/Web/WebDriver October 4th, 5th & 6th 2018.NET Conf AR v2018
  34. 34. The power of a Star https://developer.mozilla.org/en-US/docs/Web/WebDriver October 4th, 5th & 6th 2018.NET Conf AR v2018
  35. 35. Thank you! https://developer.mozilla.org/en-US/docs/Web/WebDriver October 4th, 5th & 6th 2018.NET Conf AR v2018 @hardkoded - @kblok www.hardkoded.com

×