Splunk Inc. and Ford Motor Company collaborated to analyze real-time automotive data to gain insight into driving patterns and vehicle performance.
Using Ford OpenXC to gather data from connected vehicles, Splunk employees hit the streets of San Francisco in a Ford Focus Electric Vehicle and a gas-powered Ford Escape. The data was indexed, analyzed and visualized in Splunk® Enterprise and is now publicly available.
Check out the Connected Car dashboards and watch the video to see all the fun we had!
Want to know more about how we built the project? Keep reading for the technical deep dive.
OpenXC – what it is?
OpenXC is open source hardware/software that allows you to pull a wealth of data off your vehicle in real-time and extend vehicle capabilities via plugable modules.
In short, OpenXC is an API for your car.
If you know a bit about vehicle telemetry, you are probably wondering how this is different from ODB-II scanners. There are many differences, including number of data points collected, platforms supported and quality of the data. The OpenXC’s FAQ does a great job at addressing this in depth if you want the full read.
Getting the data from OpenXC into Splunk
This was incredibly easy. We used the OpenXC vehicle interface and followed the step-by-step instructions to get data streaming via Bluetooth from the OpenXC vehicle interface to our laptop.
OpenXC vehicle interface sends data in a JSON format. Sample OpenXC data:
{“name”:”accelerator_pedal_position”,”value”:0,”timestamp”:1365512404.143000}
In Splunk we configured a new data input to ingest this JSON data and put it in index=openxc. A few field extractions later and we were ready to start creating searches and dashboards.
Creating Searches and Dashboards
I will walk you through a few of the dashboards and try to highlight some of the important panels. If there is a dashboard or panel I don’t describe that you are interested in learning more about, leave a comment or send us a tweet at @Splunk4Good. For existing Splunk users analyzing this post with a watchful eye, you might notice the “tstats” search command being used. To make these dashboards a speedy as our drivers, we used the “tscollect” command to take searches from the raw data and create an accelerated dataset, then used “tstats” to analyze it. This data acceleration technology was introduced in Splunk 5.0 and have come to the forefront in the High Performance Analytics Store that Pivot / DataModel features makes use of in Splunk 6.
Electric Powered and Gas Powered Dashboards
These dashboards displays OpenXC data from the 2014 Ford Focus Electric and gas powered 2013 Ford Escape.
Distance panel – This panel utilizes a Single Value visualization. The OpenXC odometer signal is in kilometers by default, so we converted to miles. Here is the search driving that panel:
|tstats earliest(odo) as s_odo latest(odo) as e_odo from electricdata where driver=$driver$ | eval miles_driven=(e_odo-s_odo)*0.621 | fields miles_driven
MPG panel - This panel utilizes a Single Value visualization. We are using the OpenXC signals fuel_consumed_since_restart and odometer (we converted from the default of kilometer to miles). At first we were shocked by the low MPH (~5-7MPG). Upon further inspection we think our findings are legit given the very short distance (~2 miles), top speeds reached (~50-65 MPH) and city traffic conditions. Here is the search driving that panel:
| tstats earliest(odo) as s_odo latest(odo) as e_odo earliest(fuel_consumed_since_restart) as sfuel, latest(fuel_consumed_since_restart) as efuel from gasdata where driver=$driver$ | eval gallons_spent=(efuel-sfuel)*0.264 | eval miles_driven=(e_odo-s_odo)*0.621 | eval mpg=miles_driven/gallons_spent | fields mpg
Average Speed panel – This panel utilizes a Radial Gauge visualization. The OpenXC vehicle_speed signal is in kilometers by default, so we converted to miles. Here is the search driving that panel:
| tstats avg(speed) as Speed from electricdata where driver=$driver$ | eval Speed = (Speed*0.621)
Speed range map panel – This panel utilizes the Splunk Native Maps feature to plot the coordinates and speed. We are using the OpenXC latitude and longitude signals.
Here is the search driving that panel:
host=electric source=*erik* (name=latitude AND value=37*) OR (name=longitude AND value=-122*) OR (name=vehicle_speed) | eval {name}=value | transaction startswith=(name=vehicle_speed) | search latitude=* longitude=* | eval latitude=mvindex(latitude,0) | eval longitude=mvindex(longitude,0) | eval mpg_speed=(vehicle_speed*0.621) | bin span=10 mpg_speed | dedup latitude longitude sortby +_time | geostats latfield=latitude longfield=longitude count by mpg_speed maxzoomlevel=18
Average Speed and Acceleration over time panel - This panel utilizes an Area visualization to compare speed and acceleration. Negative acceleration is deceleration. We are using the OpenXC vehicle_speed signal (converted to miles as it is in kilometers by default). Acceleration–defined–is the rate at which the velocity of a body changes with time. The search retrieves data from the Splunk 6.0 “High Performance Analytics Store” using the “tstats” command, grouping by the _time field”. Next we use “autoregress” to setup the data so we can calculate a moving average. Further in the search string we use the formula to calculate acceleration by creating the “accel_mss” field (comparing current and previous speed and time with previous time). Speed and Acceleration are then charted over time. Here is the search driving that panel:
|tstats max(speed) as speed from electricdata where driver=$driver$ groupby _time span=1s| autoregress _time as prev_time p=1 | autoregress speed as prev_speed p=1 | eval accel_mss=(speed-prev_speed)*0.277/(_time-prev_time) | timechart max(speed) as Speed avg(eval(accel_mss*10)) as Acceleration | eval Speed = (Speed * 0.621)
Comparison Dashboard
Accelerator Pedal Position panel – This panel utilizes a column visualization. We are using the OpenXC accelerator_pedal_position signal (0-100%) and visualizing the weighted count per driver and car of times accelerator was in a given position.
|tstats max(accelerator_pedal_position) as accelerator_pedal_position from comparisondata groupby _time $split$ span=1s | search accelerator_pedal_position!=0 | eval accelerator_pedal_position = round(accelerator_pedal_position/5)*5| eventstats count by $split$ | eval $split$=case($split$ LIKE “%erik-electric%”, “Erik-Electric”,$split$ LIKE “%marklar-electric%”, “Marklar-Electric”,$split$ LIKE “%gerald-escape%”, “Gerald-Gas”,$split$ LIKE “%erik-escape%”, “Erik-Gas”,$split$ LIKE “%erik-escape%”, “Erik-Gas”,$split$ LIKE “%marklar-escape%”, “Marklar-Gas”,$split$ LIKE “erik”, “Erik”, $split$ LIKE “marklar”, “Marklar”, $split$ LIKE “gerald”, “Gerald”,$split$ LIKE “electric”, “Electric”, $split$ LIKE “escape”, “Gas” )|eval weight = 100/count | chart sum(weight) by accelerator_pedal_position $split$
Steering Wheel Angle panel – This panel utilizes a column visualization. We are using the OpenXC steering_wheel_angle signal (-600 to +600 degrees) and visualizing the weighted count per driver and car of times steering wheel was in a given position.
|tstats max(steering_wheel_angle) as steering_wheel_angle max(vehicle_speed) as vehicle_speed from comparisondata groupby _time $split$ span=1s| streamstats last(vehicle_speed) as vehicle_speed| search vehicle_speed!=0 | eval steering_wheel_angle = round(steering_wheel_angle/60)*60 | eventstats count by $split$ | eval $split$=case($split$ LIKE “%erik-electric%”, “Erik-Electric”,$split$ LIKE “%marklar-electric%”, “Marklar-Electric”,$split$ LIKE “%gerald-escape%”, “Gerald-Gas”,$split$ LIKE “%erik-escape%”, “Erik-Gas”,$split$ LIKE “%erik-escape%”, “Erik-Gas”,$split$ LIKE “%marklar-escape%”, “Marklar-Gas”,$split$ LIKE “erik”, “Erik”, $split$ LIKE “marklar”, “Marklar”, $split$ LIKE “gerald”, “Gerald”,$split$ LIKE “electric”, “Electric”, $split$ LIKE “escape”, “Gas” )| eval weight = 100/count | chart sum(weight) by steering_wheel_angle $split$
Transparency
Want the raw data and see if you can find new insights or bugs in our analysis? Great! Here is a zip of the gas-powered vehicle data. The electric data is from a developer build of the OpenXC Vehicle Interface and our friends at Ford have asked us to hold back on releasing the raw data just yet.
Thanks
This was one of the coolest Splunk4Good projects I have been able to pull off! First I have to thank Ford, especially TJ, for being excellent collaborators and also our consultant Jon Guzik who made this collaboration possible.
This project is part of Splunk4Good, Splunk’s corporate social responsibility program, and was entirely volunteer driven.
Thank you to the entire team that made this project a reality, but a very special thanks to the brilliant Michael Wilde, Chief of Awesome in the CTO’s Office. He worked tirelessly with the team to ensure the Connected Car Dashboards were…Awesome 