Meet your Splunk Hackathon Winners: Splunk for Winston

One of the winners of Splunk’s first App Builder Contest at .conf2012 in Las Vegas, NV was Eric Helgeson.  Helgeson created and delivered a Splunk transport for Winston as a one-man team {team name: “Helgeson”, spirit animal “elephant”} and walked away tied for first place in the “Best Application” category.  An experienced Splunk user, Helgeson began exploring Node.js, the popular JavaScript platform for building fast, scalable network application, and stumbled upon Winston (named after a popular British meme), a multi-transport async logging library for Node.js.  Like any good hacker, Helgeson responded to the lack of a Splunk plugin for Winston by building it himself, choosing the inaugural Splunk hackathon at .conf2012 as the time and place to hack it together.

Helgeson started exploring the process of storing logs in JSON to ensure clear key-value pairs (a crucial logging best practice).  If you’re developing with Node.js, you can console.log and then pipe to a log file. Any objects sent into console.log are then automatically converted to JSON, which Splunk eats up.  Helgeson found he was able to easily parse out JSON structures with Splunk to do some cool reporting and analysis.  Once he built Splunk transport for Winston, he was pushing 1500 transactions-per-second to Splunk from his laptop, making working with Node.js and JSON logs fast and easy.

var winston = require('winston');
var options = {};
// Override logging options here
options.splunkHostname = 'node-server';
// No console, only Splunk!
winston.add(require('winston-splunk').splunk, options);
winston.remove(winston.transports.Console);

var http = require('http');
http.createServer(function (req, res) {
  // Log Request Headers
  winston.info("headers", req.headers);
  res.writeHead(200, {'Content-Type': 'text/plain'});

  // Serving up content
  res.end('Hello Splunk!\n');

  // Log other interesting elements from request json obj
  var logReq = {};
  logReq.bytesWritten = req.bytesWritten;
  logReq.url = req.url;
  logReq.method = req.method;
  logReq.statusCode = req.statusCode;
  winston.info("request", logReq);

  // Log interesting elements from response
  var logRes = {}
  logRes.statusCode = res.statusCode;
  logRes.headers = res._header;
  winston.info("response", logRes);
}).listen(1337, '0.0.0.0');

winston.info('Server running at http://127.0.0.1:1337/');

Helgeson’s work blew away the crowd and the judges.  Follow him on Twitter on @nulleric and check out his Splunk transport for Winston on GitHub (pull requests are welcome).