Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Artificial Intelligence – a buzzword, new era of IT or new threats?

61 vues

Publié le

In my presentation I will show you a couple of applications that use artificial intelligence in order to improve our security and how easily it is to use other AI to break it. You may like it or not, but natural language processing, deep learning, computer vision are being developed very rapidly and already have significant impact on your life, working behind the scenes of multiple services you use every day.
However, as a great man once said "with great power comes great responsibility", same with the AI - the risk of abuse appears. I will show you how to beat AI using rogue AI, how a crowd-sourced human intelligence can beat AI, or finally how a small, unnoticed by human change in the input data (constructed by AI of course) can severly impact the output of AI processing. I will focus on applications that improve our security not only in the cyber world (like CAPTCHA), but also in real life world (e.g. car safety systems).
Last, but not least, I will tell you how to prevent such abuses and why it is so important to understand how above-mentioned tools work.

Publié dans : Logiciels
  • Soyez le premier à commenter

  • Soyez le premier à aimer ceci

Artificial Intelligence – a buzzword, new era of IT or new threats?

  1. 1. www.securing.pl@drdr_zz Artificial Intelligence – a buzzword, new era of IT or new threats? Damian Rusinek InfoShare 2019
  2. 2. www.securing.pl@drdr_zz www.securing.pl@drdr_zz A story from the 19th century
  3. 3. www.securing.pl@drdr_zz www.securing.pl@drdr_zz • Intentions were good (bring peace to the world) „On the day when two army corps may mutually annihilate each other in a second, probably all civilized nations will recoil with horror and disband their troops.” • The reality was different „I intend to leave after my death a large fund for the promotion of the peace idea, but I am skeptical as to its results.” • The result – Nobel Prize A story from the 19th century
  4. 4. www.securing.pl@drdr_zz Back to the 21st century • Artificial Intelligence • Solves (very efficiently) problems that were unsolvable. • Will AI revolutionize IT?
  5. 5. www.securing.pl@drdr_zz www.securing.pl@drdr_zz Damian Rusinek • Will AI introduce new threats? • Will AI be the next dynamite? • Will AI become a powerful weapon od 21st century? The security perspective Security Researcher & Pentester Assistant Professor
  6. 6. www.securing.pl@drdr_zz www.securing.pl Rogue AI MY AI WILL BREAK YOUR AI
  7. 7. www.securing.pl@drdr_zz
  8. 8. www.securing.pl@drdr_zz • Verifier must know which images present cars • Simple solution: • A big database of manually categorized images • AI solution: • Use AI solution to recognize objects on images and categorize them • AI ready to use solutions: • Inception (GoogleNet) • AlexNet • ResNet • VGG AI behind reCAPTCHA
  9. 9. www.securing.pl@drdr_zz • Use existing image recognition solutions to solve CAPTCHA puzzles • Google Reverse Image Search, Clarifai, Alchemy, TDL, NeuralTalk, Caffe • Target • Google reCAPTCHA • Facebook CAPTCHA Rogue AI for reCAPTCHA
  10. 10. www.securing.pl@drdr_zz • Number of collected CAPTCHA image puzzles • 63 000 for Google reCAPTCHA • 200 for Facebook CAPTCHA • Results • Google reCAPTCHA – 70% (19 seconds) • Facebook CAPTCHA – 83% • With 40.000+ CAPTCHAs per day per host Rogue AI for reCAPTCHA
  11. 11. www.securing.pl@drdr_zz www.securing.pl Crowd-sourced human intelligence - AI will take care of your support AI CHAT BOT
  12. 12. www.securing.pl@drdr_zz • Goal • Automatic support agent • Uses AI to learn FAQ for new processes • Natural Language Processing • Experiment • Tay (abbr. Thinking about you) • A twitter account by Microsoft (@TayandYou) • Designed to mimic the language patterns of a 19-year-old American girl AI Chat Bot
  13. 13. www.securing.pl@drdr_zz • Learns from interacting with human users of Twitter • Threat • Knowledge from untrusted source • Anyone could teach Tay What can go wrong? Users posted incorrect and offensive tweets to Tay and made it… AI Chat Bot
  14. 14. www.securing.pl@drdr_zz • Tay became: • Racist • Nazi AI Chat Bot
  15. 15. www.securing.pl@drdr_zz • Taken down after 16 hours and 96 000 tweets • Lesson learned • Define the boundaries • Do not allow untrusted source to teach your AI • The next Tay – Zo • Twitter, Facebook and Skype • Does not talk about sensitive topics Are you ready for a Nazi in your support team? AI Chat Bot
  16. 16. www.securing.pl@drdr_zz www.securing.pl Small change (unnoticed by human) - Will you entrust your life to AI? AUTOMOTIVE AI
  17. 17. www.securing.pl@drdr_zz • Artificial Intelligence in Automotive • Rain sensor • AI recognizes rain drops on the windshield • Lane recognition • Autopilot keeps the car on the lane • Attack: • Funny – turn on the wipers • Scary – make the car to change lane to the opposite AI in automotive
  18. 18. www.securing.pl@drdr_zz • Tencent Keen Security Lab • Took out and analyze the autopilot component Lane detection attack
  19. 19. www.securing.pl@drdr_zz • Tencent Keen Security Lab • Took out and analyze the autopilot component Lane detection attack
  20. 20. www.securing.pl@drdr_zz • Attack scenario • Change the „input image” to fool AI. • Challenges • Find out how to change the image. • Change the physical world. „Most of the adversarial examples generated in digital domain are pixel level’s change, so it’s hard to deploy them in physical world.” AI in automotive
  21. 21. www.securing.pl@drdr_zz • Easy to get if you have access to the AI internals. Activation map Learning Deep Features for Discriminative Localization, Zhou et al., MIT
  22. 22. www.securing.pl@drdr_zz • Simple change in physical world • Can you see it? Successful lane detection attack
  23. 23. www.securing.pl@drdr_zz • Simple change in physical world • Can you see it? Successful lane detection attack
  24. 24. www.securing.pl@drdr_zz • What can you see? How hard is it to generate malicious input? Egyptian cat 78% (by alexnet) Assault rifle 93% (by alexnet)
  25. 25. www.securing.pl@drdr_zz How hard is it to generate malicious input? Pixels modified Pixels modified By more than 1% Pixels modified By more than 2%
  26. 26. www.securing.pl@drdr_zz DEMO How hard is it to generate malicious input? Access to the AI internals? Easy No access? Harder but possible
  27. 27. www.securing.pl@drdr_zz Porn alert! No need to change
  28. 28. www.securing.pl@drdr_zz Buzzword Back to the question
  29. 29. www.securing.pl@drdr_zz Buzzword New era of IT? Back to the question
  30. 30. www.securing.pl@drdr_zz Buzzword New era of IT? New security threats? Back to the question
  31. 31. www.securing.pl@drdr_zz Design • Threat modelling • Consider rogue AI as threat • Define boundaries AI security Development • No untrusted source teaching your AI • Generate malicious inputs and teach your AI Use • No critical decisions based on AI only • Monitor outputs from AI (be up to date) • Control boundaries Architecture Assessment System Testing System Monitoring
  32. 32. www.securing.pl@drdr_zz Thank you! Contact me: damian.rusinek@securing.pl Ready for ? We are! @drdr_zz