JavaScript is finding its way further and further out of the browser. Only a couple of years ago, if someone had said they wanted to build robots only using JS you'd think they were crazy. Having tried it at the time those naysayers were correct - it was a disaster. Recently, particularly as a result of the nodebots project, JS Robotics has started to come of age and it's now possible to build simple robots using JavaScript for the majority of the stack - everything from control and sensing to motors to lights, AI and computer vision. This talk will give an overview of what's currently possible, where the current gotchas are, how to get started and have some interactive elements that can be played with during or after the session. Andrew is a creator & destroyer of things that combine mobile web, ubicomp and lots of data. Sometime programmer, interaction researcher & CTO @ JBA. Be Responsive meetup / Melbourne Geek Night Crossover night September 2014

1. Building droids with
JavaScript
MelbGeeks / Be Reponsive, September 2014
Andrew Fisher @ajfisher!
Hi! My name is Andrew Fisher and I’m an interaction researcher. Today I
want to talk to you about web connected hardware and building droids with
javascript for the next 30 minutes.

2. Video
Video of this talk is available at:!
!
http://vimeo.com/107036858
A Video of this talk is available at http://vimeo.com/107036858 if you want
to watch it there instead.

3. JS all the real things
flickr (cc) Quasimondo
Hardware for a long time was really hard. My initial interest in computing
really came from an interest in electronics. But the reality was that for a kid
growing up in the late 80s, early 90s, building serious hardware was
prohibitively expensive and required facilities that most kids didn’t have
access to in those days - even the idea of something like a hacker space or
a makerspace didn’t exist then.
!
Doing anything non-trivial in hardware was a very difficult undertaking. As
a result, I moved to software. As you’d expect, in the world of software, the
effort to reward ratio was far better than for hardware, especially for the
attention span of a teenager.
!
And this situation remained the case until the early 2000s.
!! Image: http://www.flickr.com/photos/quasimondo/5203908319

4. Hardware is more like software
(cc) Phill Farrugia
Until this came along - this is an arduino and it has changed the way
hardware is thought of. Hardware was no longer expensive - these were
designed at the time to be less than 20 euros and the designs were open
sourced, meaning anyone could make them. Today you can get them for
$8. Sophisticated hardware became accessible to students, artists, kids and
web people with little repercussion for blowing one up. With it, this
community has brought ideas around design, user experience, art and
software and architectural principles.
!
More recently, over the last 2 years, some great work has been done in the
node community getting JS to work with hardware like this - to the point
where working with hardware using javascript is now extremely easy.
! So tonight, I want to talk to you about that and how all of you can all start
working with hardware with JS and along the way we’ll bump into some
robots.
!
http://phillfarrugia.com/post/92908852193/spent-the-day-building-robots-
at-nodebotsau

5. TODO
1. Different ways to interface with hardware!
2. The JS / hardware stack!
3. Applications and examples
So to do this, we’ll look at.
!
How we can interact with hardware.
! Some code to show what a common JS hardware stack starts to look like
!
Then we’ll play with some examples.

6. Working with hardware
flickr (cc) Oskay
Let’s start by looking conceptually at how we develop with hardware. I think
about this at sort of three levels.
!
At the metal
At the device
And somewhere in between.
!
Image: http://www.flickr.com/photos/oskay/2310115216/

7. Working with the metal
flickr (cc) Wonderlane
Here we are usually writing code specifically for a chip or board at a very
low level. It usually means writing C or C++. You can get a bit of
abstraction with hardware libraries but even with good libraries you need to
write a lot of code.
!
You have insane limitations. It’s like going back to the 80s - an arduino for
example has TWO KILOBYTES of RAM.
!
Image: http://www.flickr.com/photos/wonderlane/3198166347

8. Working low level
flickr (cc) lisovy
Playing at the chip level gives you masses of control but your iteration
cycles take time and you’re constantly thinking about things like memory
and garbage collection and not frying a chip. Debugging is also generally a
nightmare.
!
Image: http://www.flickr.com/photos/lisovy/4677688431/

9. Device hacking
flickr (cc) Roo Reynolds
So At the other end of the spectrum there are devices that are already
made. This is hardware with APIs - maybe they are via the network, maybe
via an embedded web server or some sort of serial protocol.
!
However it works, you’re basically interacting with the hardware given to
you as a service.
!
You can build clever ways of working with the service but fundamentally
you can’t change the way the hardware works.
!
IMage: http://www.flickr.com/photos/rooreynolds/9350631793

10. AR.Drone
flickr (cc) neeravbhatt
A good example of this is the AR Drone - you can play with using a node
library to interact with the copter directly to do some really interesting
things but you can’t fundamentally change the hardware. Having said that,
people have done some fantastic things at this level.
! So there’s plenty of very interesting devices you can grab hold of and start
playing about with really easily.
!
http://www.flickr.com/photos/neeravbhatt/6885424870

11. Hardware in comfort
flickr (cc) Zack Hoeken
In between these extremes though is a new category, where you can work
with and change the hardware but you can still use high level languages to
do it. This is a great combo as it allows for rapid prototyping but also
makes hardware more accessible to non-embedded application designers -
people like us - web designers and developers to be able to tinker with.
!!!
Image: http://www.flickr.com/photos/hoeken/3519955473/

12. Sketching in hardware
flickr (cc) Camille Moussette
So this approach give you the ability to prototype rapidly as well as work
almost directly with the hardware. There are some limitations around some
hardware but to start off with and for some prototyping that doesn’t matter.
!
To show you how easy this is let’s make something here and now.
!
http://www.flickr.com/photos/9225693@N08/6051548279

13. First steps
We’re going to focus on the arduino.
!
This is an arduino - they cost about $30, they are awesome and come in
many different forms for different applications from small to large. They run
off USB or a battery so it’s hard to blow them up and even harder to
electrocute yourself. There is also huge amounts of community information
about them.
!
Now the arduino is way too under powered to run JavasScript so we’re going
to use our computer to do that. As such we just need to pass messages
down the wire to the arduino over the USB cable so for that we use a
protocol called Firmata.
! So Firmata allows you to tell the arduino to do things like turn a pin on or
off, take a reading etc. Firmata is a neat idea as it exposes nearly all of the
features of the arduino but via a protocol so now you can control it from
somewhere else that talks the firmata protocol…
!
Hang on - that sounds suspiciously like an API…

14. Hardware hello world
flickr (cc) pj_vanf
So we plug in an LED (this one is bigger than normal so you can see it
easily) and then we’ll need to write some code.
!
Go interactive version here to show connecting and turning an LED on and
off.
!I
’ve got a little bit of scaffolding here so I don’t have to type everything. But
you can see it just creates a board and connects it. Now it’s connected we
can tell the board we want to assign a pin as an output then we’ll tell it to
make the pin go HIGH which means send it some volts - we do that with the
digitalWrite command. Then we can make it go LOW and as you can see as I
do this it turns the LED on and off.
!
http://www.flickr.com/photos/vanf/5210360116
!

15. Web thing hello world
http://github.com/ajfisher/nbscaffold
So we’re all web devs here so let’s not stray too far away from our comfort
zone. Let’s wrap a web interface around this light so we can turn it on and
off with a click of a button on a page.
!
We can’t do this interactively very easily so I’ll show you some code.
!
This is a bit of overkill but we’re going to use express and web sockets so
it’s a bit more realtime but also it will lay the foundation for what I’ll show
you next.
!
We set up the socket messages on the server to switch things on and off .
On the client side all our HTML and JS is doing is just sending messages on
click so nothing too interesting.
!
And there we go - button click to turn a light on and off via a web browser.

16. The JS hardware stack
Arduino (Sensors and actuators)!
Firmata (Communications protocol)!
NodeJS (Application logic)!
WS/HTTP (Network & security protocols)!
Client (UI, Input, Visualisation)
So this is what the JS hardware stack looks like.
!
We’ve got an arduino with sensors and actuators. Firmata which provides
the communications protocol. The NodeJS application gives us application
logic and integration with other libraries.
!
Networking is provided over http or websockets and this can give us
security and encryption.
!F
inally the web page for the client gives as input methods, a data viz layer
as well as user interface.

17. Easy install
npm install firmata express socket.io
So this stack can be created with pretty much just this command plus an
arduino with firmata on it in about 2 minutes.
!
Which is pretty cool. And it’s robust enough now that it’s most likely going
to work the first time you try it.
!
Now it’s pretty cool to be able to connect a web page to a bit of hardware
right?

18. Hardware as objects
var led = new five.Led(13);!
led.blink(1000);!
led.stop();!
led.on()
Wouldn’t it be great if we could do something like this rather than telling
the pins to turn on and off?
!
Wouldn’t it be cool to have some sort of abstraction for LEDs which are
super common and be able to interact with them in standard ways?
!
Well we can, thanks to a project called Johnny Five that was kicked off about
2 years ago by a chap called Rick Waldron.

19. Johnny Five
Johnny Five uses the firmata protocol we were just using but provides
abstractions for common electronic devices. Things like LEDs, sensors,
motors, accelerometers - all sorts of things. The list keeps growing all the
time and there’s now over 60 committers to the project.
!
This means we can prototype really, really fast and build cool things.

20. Build stuff
This was a pulse oximeter that someone made with an LED, a light sensor
and some signal processing in JS

21. Build stuff
This was the beer-o-matic 2000 - a drinks dispenser that gave you beer
when light was on the hat.

22. Build stuff
A nerf gun attached to a build system to find and then shoot the developer
that broke a build.

23. Build stuff
And of course, robots.

24. Looking for droids?
flickr (cc) solo
So let’s look at a robot that is built with web tech.
!
Image: http://www.flickr.com/photos/donsolo/3768623542/

25. Nodebot
This is a bot I built from scratch - The core was just some metal and some
motors.
!
All of the control and interface for this is web tech. Node, HTML, CSS,
JavaScript

26. Nodebot architecture
Battery
Arduino RPi
4WD motor control
Sensors Firmata
NodeJS!
- Express!
- Johnny Five!
- Open CV!
- Socket IO
WiFi
Camera
4WD motor control
Architecturally this is what it looks like:
!
The arduino deals with the hardware using firmata.
!
A raspberry pi runs a node server and communicates to the arduino over a
serial connection.
!
The webcam provides a video feed.
!
A web interface shows the video and provide input control to drive the bot
using the device API and the gyro in your phone or tablet or keyboard on
your desktop.
!
And here it is here…

27. Go and make
Arduino.cc (start here for everything)!
Johnny Five (npm install johnny-five)!
Node ARDX (node-ardx.org)!
Minimal examples (github.com/ajfisher/nbs)!
SimpleBot (github.com/nodebotsau/simplebot)!
!
http://nodebotsau.github.io!
As the web starts extending its reach into more and more physical territory,
we need to translate our skills into that of physical design and engineering
along with it. Hardware is still ruled by traditional electronics engineering
and design - which mean we end up with interfaces on our hardware that
are appalling. The skills all of you have in this room are immensely
applicable to the physical world, experience design, interface design,
architecture, engineering, development.
! So I’d like to encourage all of you to to play around with some hardware
and take your skills physical.
!
The stack I’ve talked about is very solid, it works really well and will allow
you to experiment very efficiently. You can all do this and I’m sure build a
robot much better than this one I’ve got here.
!
Don’t be worried about playing with hardware - it’s loads of fun - just stay
away from mains electricity.
!
These links should get you started.
!
We run NodeBots events here in Melbourne and around Australia. The next
one is on October 1st at the Melbourne HackerSpace but go here and you
can find out more.

28. Building droids with
JavaScript
MelbGeeks / Be Reponsive, September 2014
Andrew Fisher @ajfisher!
So thanks a lot and if you have any questions now or after please let me
know.