node

Making robots to do our bidding with Node.js

Node powers almost every piece of user-facing software we develop and we enjoy using it every day. But we also like to stretch our imagination and find applications for node that are outside the mainstream.

Enter Nodebots Day 2014, an annual event held around the world that brings together people who love node and are excited to use it to interact with the physical world and make robots do their bidding.

Fluencia hosted the DC edition of this year’s event at our headquarters. This post will serve as both an introduction to nodebots and the johnny-five framework, and a summary of the highlights from the day for those who couldn’t make it. Let’s go!

Getting started with johnny-five

The johnny-five framework – named after the robot protagonist of the 1986 film Short Circuit – is an open-source library and npm module that provides an interface for writing JavaScript that the microcontroller can understand. It works out of the box with Arduino, but plugins are available for Raspberry Pi, Spark Core, and many others.

npm install johnny-five

Now, create your first Hello World! Make sure you have Node.js and npm installed, and follow the instructions on the johnny-five README to get up and running. Write a file called helloworld.js:

var five = require(‘johnny-five’), // Here’s Johnny!
    board = new five.Board();

board.on(“ready”, function() { 
  var led = new five.Led(13);     // LED connected to pin 13
  led.strobe();
});

Connect your Arduino to any USB port and run the program with node helloworld.js. In a few seconds the program will connect to your board and the LED will start blinking. Hello World!

But that’s not all. You also have access to your very own REPL (read-eval-print loop) session, which allows you to interact with the board directly from the command line. Inject the REPL within the board “ready” handler, for example:

this.repl.inject({
  led: led
});

Then you can run commands like led.on(), led.off(), and led.strobe() in real time.

We’re just getting started

This is all pretty cool already, but there’s much more you can do to harness the power of node for your bot.

Node is great for creating http servers quickly and easily. With the express framework, it’s even easier to write a simple server that listens for requests and do something when it receives a request. Imagine if we could remotely control our nodebot through a web browser from anywhere online.

Install the express framework with npm install express and use it to spin up a server and listen for GETs on the /turn route. For example,

var express = require('express');
var app = express();
...
app.get(‘/turn’, function(req, res) {
  servo.sweep();
});

app.listen(3000);

A program as simple as this allows you to control your robot by visiting http://localhost:3000/turn. This is awesome, but the app has to be running on the same machine that is connected to the Arduino to communicate with it. Not exactly remote control.

There is a solution to this, too. We can create a tunnel into localhost and expose the program to the internet. Several programs are out there to allow just such tunneling, and ngrok is exactly what I was looking for.

Install and run ./ngrok in the root folder of your project, and it will assign a randomly generated URL such as http://74d1d276.ngrok.com which tunnels directly into your localhost. Now anyone on the internet can visit this URL to get access to your locally running application. I wouldn’t recommend this for a production level app, but for testing and hobbying it works great!

Look at the awesome things we built

Here’s a sampling of some of the projects that came out of Nodebots Day DC 2014.

Morse code nodebot

This nodebot is a master morse code translator, taking in any string and converting it to a series of long and short buzzer beeps, by @mcwhittemore.

Button boxing game nodebot

Making use of only a few parts, this project used 7 LEDs, 2 pushbuttons, and a few wires and resistors, and quickly became the most addicting and exciting game of the day, by @daiweilu and @wind_memory.

Fork it and make your own!

Twitter nodebot

This nodebot connects to the Twitter Streaming API to blink and beep whenever anyone tweets @nodebotsday, by yours truly, @danielpazsoldan.

And many, many more!

There was an Xbox Kinect nodebot that mimics your arm movement, a nodebot that could tell you the traffic in your area, and there was the makings of a nodebot tank. But we just couldn’t capture all of the awesome projects.


We had a good turnout and everyone had a great time. The day was full of brilliant ideas, wonderful company, and the first steps toward JavaScript robot domination - a success in my book. Special thanks to Caroline Woods at Fluencia, and Matthew Whittemore and Chetan Shenoy at Social Tables for organizing a unique and awesome event.

What can you build using node and johnny-five? Show us what can you do, and see you at next year’s Nodebots Day!

The 4 Keys to 100% Uptime with Node.js

On Tuesday, I presented at Nova Node on achieving 100% uptime with Node.js. [Update: video of the talk is now available.] This topic is near and dear to our hearts: giving our users the best experience means keeping Fluencia and SpanishDict running all the time, every day. This takes work, and over the past few months we've had opportunities to dive deep into error handling/recovery and zero-downtime deployment techniques.

We've found that the four keys to (nearly) 100% uptime with Node.js are:

  1. Sensibly handle uncaught exceptions.
  2. Use domains to catch and contain errors.
  3. Manage processes with cluster.
  4. Gracefully terminate connections.

1. Sensibly handle uncaught exceptions.

Uncaught exceptions happen when an Error is thrown outsisde of a try/catch block, or an error event is emitted and nothing is listening for it. Node's default action following an uncaught exception is to exit (crash) the process. If the process is a server, this leads to downtime, so we want to avoid uncaught exceptions as much as possible, and sensibly handle them when they do happen.

Avoid them can be accomplished by writing solid code and testing it thoroughly, and by using domains.

2. Use domains to catch and contain errors.

Domains can catch asynchronous errors—for example, error events—which try/catch cannot do.

var d = require('domain').create();

d.on('error', function (err) {
  console.log("domain caught", err);
});

var f = d.bind(function() {
  console.log(domain.active === d); // <-- data-preserve-html-node="true" true
  throw new Error("uh-oh");
});

setTimeout(f, 100);

The above example would crash the Node process if function f (which throws an error) were not called in the context of a domain. Instead, the domain's error handler is called, giving you a chance to take some action: retry, abort, ignore, or even rethrow. What to do is dependent on context, but the point is that you can choose, rather than having the process crash.

When code is running in the context of a domain—that is, a domain is "active"—then domain.active contains a reference to it. New EventEmitters will automatically bind to the active domain, saving you the hassle of manually listening to their error events (though there may still be good reason to do so—but if you miss one, it will automatically go to the domain error handler).

domain.add will add an existing EventEmitter to a domain. Domains have more methods than add and bind; the full list of is available in the docs.

3. Manage processes with cluster.

Domains alone are not enough to achieve continuous uptime. Sometimes an individual process will need to go down whether due to a rethrown error or to deployment. Using node's cluster module, we can spawn a master process that manages one or more worker processes, distributing work (for example, incoming connections from clients) among them. The master process need never stop running, meaning the socket always accepts connections, and when any particular worker stops working, the master can replace it.

We can send a signal to the master process when we want to initiate a reloading of workers. For example, to deploy without downtime, we might give our master a symlink that points to worker code, then later update that symlink to point to a new version of the code, and send a signal to the master process that causes it to shut down existing workers and fork new workers. The new workers will come up running the new code, and the master process keeps running the entire time!

Coordination between master and workers is important. Workers must communicate their state, and master must know what action to take based on that. We've found recluster to be a useful wrapper around Node's native cluster module that provides worker state management.

4. Gracefully terminate connections.

When workers/servers need to shutdown, they must do so gracefully, as they may be serving any number of in-flight requests, and also have open TCP connections to many more clients. Any call to process.exit should occur inside the callback to the server.close method, which only fires once a server has closed all its connections.

To close existing connections, we can add middleware that checks whether an app is in a shutting down state, and if so, causes connections to be closed ASAP:

var afterErrorHook = function(err) { // <-- data-preserve-html-node="true" called after an unrecoverable error
  app.set("isShuttingDown", true); // <-- data-preserve-html-node="true" set state
  server.close(function() {
    process.exit(1);  // <-- data-preserve-html-node="true" all clear to exit
  });
}

var shutdownMiddle = function(req, res, next) {
  if(app.get("isShuttingDown") {  // <-- data-preserve-html-node="true" check state
    req.connection.setTimeout(1);  // <-- data-preserve-html-node="true" kill keep-alive
  }
  next();
}

To ensure all servers shutdown eventually, we can also call process.exit inside of a timer.

Back to handling uncaught exceptions.

The Node docs say not to keep running after an uncaught exception, for good reason:

An unhandled exception means your application — and by extension node.js itself — is in an undefined state. Blindly resuming means anything could happen. You have been warned.

http://nodejs.org/api/process.html#processeventuncaughtexception

Now we have the pieces in place to sensibly handle uncaught exceptions:

  • Log error.
  • Server stops accepting new connections.
  • Worker tells cluster master it's done.
  • Master forks a replacement worker.
  • Worker exits gracefully when all connections are closed, or after timeout.

If the exception was triggered by a request, it is not possible to respond to that request because the uncaught exception lacks the context to determine which request it was. But we should be able to handle any other in-flight requests, minimizing effective downtime.

Remember that while parts of Node are very stable, and it is in production use with greate success by many, many companies (including us!), other parts such as domains and the cluster module are still classified as unstable or experimental, and may change in the future.

This was a quick summary of what I covered in the talk. The slides below contain additional example code and various tips and other considerations. Hopefully they will help you to achieve (nearly) 100% uptime with your Node application!