Splunk Plug-in for Eclipse

Developers can use the Splunk Plug-in for Eclipse for building applications that use and extend Splunk. Eclipse is the tool of choice for the over 10 million Java developers around the world, including the many at Splunk customers.

Java SDK template

The plug-in contains a project template for building a new Splunk SDK for Java application. This is ideal for building an application that searches against Splunk data or does automation. The project template includes snippets for performing common SDK tasks, as well as infrastructure for wiring up the application to log application data directly to Splunk utilizing popular log frameworks like Apache Log4J.

Below you can see a screenshot of using the plugin to create a new Splunk SDK for Java application.

Once I have given it the basic info for my app, I can click finish and see a skeletal application is generated for me.

The project comes loaded with a set of snippets for performing common tasks with the SDK. It is as easy as typing “spl + ctrl + space” in Eclipse to bring up a list of snippets you can select from.

Selecting “splconnect” for example will inject code for connecting to a splunk instance.

Using the new template makes it really easy to kick start your Java development with Splunk!

Modular Inputs

Additionally, the Plug-in includes support for creating modular inputs to extend Splunk. Modular inputs are useful for pulling in streams of data additional data sources such as other internal systems or public APIs like Google, Facebook and Twitter.

Using the modular input template removes a lot of the plumbing you would otherwise need to create a Java modular input. The template will even create a fully working starter implementation for you to play with. Below is a screenshot of creating a new input using the defaults, which includes creating the starter.

Once I click “Finish” you can see that the Splunk application, configuration and default implementation is created for me!  Once my app is ready to go, the plug-in even includes Ant integration for packaging up an SPL that I can publish to Splunk Apps or distribute internally. Right clicking on the build.xml brings up a menu to use this feature.

Selecting “Ant Build” takes care of everything for you. Below you can see the output of the successful build.

You can now grab the generated TestSplunkInput.spl and start sharing it right away. 

Custom Search Commands

One of the pieces of common feedback we hear from developers is that you love the support for creating custom commands in Python, but you wish that there was a more first class and simplified experience. Well we’ve heard you loud and clear! We’re happy to announce that today we’re introducing a new library for creating custom search commands within the SplunkSDK for Python. Using the library you will find it easy to author new commands and in a strongly typed and far less error-prone manner. The library also dramatically reduces the amount of code you need to write.

The new commands can be used for introducing complex algorithms like doing geo-fencing, or linear regression, for doing dynamic queries from public APIs (such as Social or Open Government APIs), or for addressing domain specific concerns like retrieving data from an internal system or applying custom business logic on top of Splunk data.

In the box you’ll find three types of base search commands you can derive from.

• StreamingCommand allows you to pipe data from another Splunk command and apply custom transformation logic or filtering. For example imagine you want to filter results based on some domain specific logic, streaming commands are perfect.
• ReportingCommand allows you to run map/reduce type operations. If you have a large set of data that you need to consolidate down or perform complex mathematical calculations on, then a ReportingCommand is what you want. 
• GeneratingCommand allows generating transient data dynamically which can be piped into other commands. You can use it to do adhoc queries from system. It is a great complement to our indexing features.

In future posts we’ll dive into more detail on how to actually build these.

Just for illustration, you can use a custom GeneratingCommand to retrieve listings in an adhoc manner dynamically from Yelp’s API based on a set of parameters such as the location and the type of business. Using the SDK, I created a new “yelp”  command for just this purpose.

Below you can see using the command to retrieve a list of sushi restaurants near our Splunk Seattle office in order by distance.

You can use it for more than finding just a good sushi meal, like finding local theaters.

You could even imagine going further and getting the local show times for each theater by using something like the Fandango API.

To implement this command, I created a YelpCommand class in Python which inherits from GeneratingCommand

Next I declaratively specified the options the command accepts using the Option decorator. Finally I implemented the Generate function, which in this case queries Yelp using the Python requests package, extracts the data I am interested in, and yields it back to Splunk.Once the data is returned from a custom search command, you can leverage the full power of Splunk to pipe the results to other commands for aggregating the data, creating charts, etc. You can even join existing events within Splunk to the custom search command to open up new ways of enhancing your data.

I’ll be blogging on how to build the Yelp command soon, but if you want it now, you’ll find it on Github!

Getting Started

To get the updated Python SDK with Search Commands or the new Eclipse tooling, head on over the dev.splunk.com. On the site you’ll find several walkthroughs on the new Eclipse plug-in as well as custom search commands. You can download either the SDK or tools, and in the SDK you’ll find custom search samples as well.

In the coming weeks our team will be blogging more in detail on how you can use these features including how to build custom commands like the Yelp command, as well as using the Eclipse plug-in.

So go check it out, and if you have any feedback on features you’d like to see, head on over to our UserVoice site at splunkdev.uservoice.com.