[JS Kongress 2016] KittyCam.js - Raspberry Pi Camera w/ Cat Facial Detection
•
1 j'aime•1,513 vues
The document describes a KittyCam project that uses a Raspberry Pi, camera, and PIR motion sensor to detect when a cat is in view of the camera and take a photo. It uses Node.js and several frameworks like Johnny-Five and KittyDar. When motion is detected, it takes a photo using Raspistill, detects if a cat face is present using KittyDar, and if so uploads the photo to Cloudinary and sends a text with the image link. It provides details on the hardware components, software stack including Node.js, and implementation of the various functions.
Recommandé
Contenu connexe
Tendances
Tendances (7)
En vedette
En vedette (14)
Similaire à [JS Kongress 2016] KittyCam.js - Raspberry Pi Camera w/ Cat Facial Detection
Similaire à [JS Kongress 2016] KittyCam.js - Raspberry Pi Camera w/ Cat Facial Detection (20)
Plus de Tomomi Imura
Plus de Tomomi Imura (20)
Dernier
Dernier (20)
[JS Kongress 2016] KittyCam.js - Raspberry Pi Camera w/ Cat Facial Detection
- 1. girlie_mac@ KittyCam.js Smile, you’re on camera!Smile, you’re on camera! Tomomi Imura
- 2. girlie_mac@ Hello, I’m Tomomi & I’m a: ● San Francisco Dweller ● Front-End Engineer ● Open Web + Tech Advocate ● N00b Hardware Hacker ● Sr. Developer Advocate at Nexmo ● Cat lady of InterWeb
- 3. girlie_mac@
- 5. girlie_mac@ Arduino ● MCU-based kit ● Open-Source hardware & software (IDE) ● The first developer-friendly boards
- 6. girlie_mac@ Sketch ● Language for Arduino ● Loosely based on C ● Wut, C? ● HALP!!1!!! #include <Servo.h> #include <Wire.h> #include <Firmata.h> #define I2C_WRITE B00000000 #define I2C_READ B00001000 #define I2C_READ_CONTINUOUSLY B00010000 #define I2C_STOP_READING B00011000 #define I2C_READ_WRITE_MODE_MASK B00011000 #define I2C_10BIT_ADDRESS_MODE_MASK B00100000 #define I2C_MAX_QUERIES 8 #define I2C_REGISTER_NOT_SPECIFIED -1 #define MINIMUM_SAMPLING_INTERVAL 1 int analogInputsToReport = 0; byte previousPINs[TOTAL_PORTS]; byte pinConfig[TOTAL_PINS]; byte portConfigInputs[TOTAL_PORTS]; int pinState[TOTAL_PINS]; unsigned long currentMillis;
- 7. girlie_mac@ Raspberry Pi ● $35 single-board computer ● Broadcom chip with ARM-compatible CPU & GPU ● Runs Linux ● More language choices: C, C++, Python...
- 8. girlie_mac@ But… I want to code in
- 9. girlie_mac@ Johnny-Five ● JavaScript robotics framework ● Works with Arduino-compatible Boards ● IO plugins for more platform supports ● http://johnny-five.io/
- 10. girlie_mac@ Awww, JavaScript all the way!
- 12. girlie_mac@ KittyCam Raspberry Pi camera with cat facial detection! ● Hardware: Raspberry Pi, camera, and PIR sensor ● Software: Node.js + J5 + More open-source goodies
- 14. girlie_mac@ kittyCam.js 1. Detect motion 2. Take a photo 3. Cat facial detection 4. Store the photo in cloud 5. Real-time view on web 6. SMS the photo link HELL, YEAH!
- 15. girlie_mac@
- 16. girlie_mac@ Raspberry Pi 3 PIR Motion Sensor Camera Module F/F Jumper Wires (x3)
- 17. girlie_mac@
- 18. girlie_mac@
- 19. girlie_mac@ 1. Raspberry Pi 3 2. 5MP Camera Board Module 3. Pyroelectric Infrared (PIR) motion sensor 4. Female/Female wires
- 20. girlie_mac@ Programming Raspberry Pi Pre-installed on Raspbian OS: C / C++
- 21. girlie_mac@ Programming RPi with Node.js
- 22. girlie_mac@
- 23. girlie_mac@ Installing Node ARM $ wget https://nodejs.org/dist/v4.4.5/node-v4.4.5-l inux-armv7l.tar.xz $ tar -xvf node-v4.4.5-linux-armv7l.tar.xz $ cd node-v4.4.5-linux-armv7l $ sudo cp -R * /usr/local/
- 24. girlie_mac@ kittyCam.js 1. Detect motion 2. Take a photo 3. Cat facial detection 4. Store the photo in cloud 5. Real-time view on web 6. SMS the photo link HELL, YEAH!
- 25. girlie_mac@ kittyCam.js Tech Stack 1. Detect motion w/ Johnny-Five IR.Motion obj 2. Take a photo w/ Raspistill, command line tool 3. Cat facial detection w/ KittyDar 4. Store the photo in Cloudinary 5. Publish & subscribe the url to display on web via PubNub 6. Send a text message via Nexmo
- 26. girlie_mac@ $ Raspistill canvas catDetect.js app.js kittyDar Motion detected take a photo Store the photo if cats are detected display photos on web browsers real-time anywhere Johnny-Five w/ raspi-io child process Returns url Notify with SMS via
- 27. girlie_mac@ Johnny-Five w/ Raspi-io const five = require('johnny-five'); const raspi = require('raspi-io'); let board = new five.Board({io: new raspi()}); board.on('ready', () => { console.log('board is ready'); ... });
- 28. girlie_mac@ Motion const five = require('johnny-five'); const raspi = require('raspi-io'); const board = new five.Board({io: new raspi()}); board.on('ready', function() { // Create a new `motion` hardware instance const motion = new five.Motion('P1-7'); ... }); a PIR is wired on pin 7 (GPIO 4) VCC Ground Data
- 29. girlie_mac@ Raspistill Command Line Tool $ raspistill -o myPhoto.jpg
- 30. girlie_mac@ PIR Sensor > Run Camera const child_process = require('child_process'); board.on('ready', () => { const motion = new five.Motion('P1-7'); motion.on('motionstart', () => { // Motion detected let filename = 'photo/image_'+i+'.jpg'; let args = ['-w', '320', '-h', '240', '-o', filename, '-t', '1']; let spawn = child_process.spawn('raspistill', args); spawn.on('exit', function() { console.log('A photo is saved as '+filename); ... motion detected! Take a photo! Spawns a new process w/ a given shell command
- 31. girlie_mac@ Processing Image spawn.on('exit', () => { let imgPath = __dirname + '/' + filename; // Child process: read the file and detect cats with KittyDar let args = [imgPath]; let fork = child_process.fork(__dirname+'/detectCatsFromPhoto.js'); fork.send(args); // the child process is completed fork.on('message', (base64) => { if(base64) { uploadToCloud(base64); // Send to cloud storage } }); ... Create another worker by running a new instance of V8 engine.
- 32. girlie_mac@ detectCatsFromPhoto.js const kittydar = require('kittydar'); const Canvas = require('canvas'); process.on('message', (m) => { fs.readFile(m[0], (err, data) => { ... let canvas = new Canvas(w, h); let ctx = canvas.getContext('2d'); ctx.drawImage(img, 0, 0, w, h, 0, 0, w, h); let cats = kittydar.detectCats(canvas); console.log('There are', cats.length, 'cats in this photo'); if(cats.length > 0) {base64Img = canvas.toDataURL();} process.send(base64Img); process.exit(0); }); Running in an another process a cat detected! KittyDar
- 33. girlie_mac@ KittyDar ● Open-source JavaScript cat facial detection written by Heather Arthur ● Takes a canvas obj & calculate the locations of cats in the image let cats = kittydar.detectCats(canvas);
- 34. girlie_mac@ KittyDar: Behind the Scene 1. Chops up the image up into many “windows” 2. Extracts data by measuring a set of gradients, from light & dark in order to find edges 3. Compares the direction of these edges to the edges found in known cat images Neural network (JSON w/ vector data) is pre-trained w/ thousands pics of cats & non-cats
- 37. girlie_mac@ Cat Facial Detection The annotation data sequence: (Number of points, default is 9), (Left Eye), (Right Eye), (Mouth), (Left Ear-1), (Left Ear-2), (Left Ear-3), (Right Ear-1), (Right Ear-2), and (Right Ear-3)
- 38. girlie_mac@ Cat Facial Data 9 247 94 294 92 273 127 213 81 207 29 247 53 286 50 320 20 322 74 00000561_012.jpg.cat00000561_012.jpg
- 39. girlie_mac@ Cat Facial Data Collection
- 41. girlie_mac@ Send the Pic w/ Kitty to Cloud const cloudinary = require('cloudinary'); // the child process is completed fork.on('message', (base64) => { if(base64) { cloudinary.uploader.upload(base64, (result) => { // Done! - Get the URL and do more stuff }); } else deletePhoto(imgPath); });
- 42. girlie_mac@ View Photos Real-time via Socket const pubnub = require(pubnub); Publish Subscribe
- 44. girlie_mac@ Send SMS via Nexmo const Nexmo = require(nexmo); let nexmo = new Nexmo({//config with API keys}); nexmo.message.sendSms( FROM_NUMBER, TO_NUMBER, ' '+ url, options, (err, responseData) => { if (err) console.log(err); else console.dir(responseData); });
- 45. girlie_mac@
- 46. girlie_mac@ QA Team Lead QA: Jamie Ginger Basil Alice Yugi Venom @kittenVenom Ozzy
- 48. girlie_mac@ Next Steps ● Upgrade Hardware ○ Raspberry Pi 3 ○ NoIR Night Vision Camera ● Upgrade Node (was 0.12) & all dependencies ● ES5 to ES6 ● More features (Maybe) ○ Cat Identification w/ RFID ○ Photo Booth w/ Filter effects & props
- 49. girlie_mac@ More Cat Projects w/ Node.js
- 50. girlie_mac@ https://http.cat Thanks Rogério Vicente for the .cat domain & the API! Previously: HTTP Status Cats
- 51. girlie_mac@ HTTP Status Cats Slack Bot Install it at: http://girliemac.com/ slack-httpstatuscats/ Tutorial on Medium.com: https://goo.gl/saOUl0
- 52. @girlie_mac Next Project? ● Selfie bot (à la Mannie the Selfie Cat) Mannie the Selfie Cat by @yoremahm on Instagram https://www.instagram.com/yoremahm/
- 54. girlie_mac@ Attribution: Emoji by Emoji-One (CC-BY 4.0)