Modular Inputs and Scripted Inputs provide a great way to develop custom programs to collect and index virtually any kind of data that you can set your mind to.
But on whatever platform you have deployed Splunk on, you will also have a whole bevy of other inputs just waiting for you to tap into to get that data into Splunk .They would be the various programs that come with the platform and those that you have installed on your platform.v
This is actually why I created the Command Modular Input that I introduced in a recent blog, a means to as simply as possible leverage the power of your existing system programs and get this data into Splunk ,rather than having to go and create custom wrappers for each program you want to get data from.
Now the use cases are limitless really.And since I released this Modular Input on Splunkbase Apps, I’ve heard of customers, staff and partners using it for so many interesting uses cases, from a utility to rattle up a quick POC through to production Splunking.
So I have decided to start a blog series on these use cases, and this is episode 1.
Command Modular Input Use Case: Agentless monitoring of remote files
Use Case
You have files on a remote machine that you want to monitor without installing a remote agent.
Solution
The first thing I want to do is identify the system commands that will do this for me.
- Tail : streaming , appended file data is returned.
- Stat : non streaming , just a periodic poll of the file(s) attributes
- Cat : non streaming , to periodically poll the full file contents , or perhaps create a baseline image of the file contents.
The Command Modular Input is able to operate in streaming and non-streaming mode depending on the nature of the command output.
These commands will return raw command output. The Command Modular Input has the ability to plugin custom response handlers that can be used for data pre-proessing and custom formatting of output (ie: you might convert output to JSON).
So now that we have our commands , how can we execute these remotely ? Well , you can execute the commands remotely over SSH.
Here is an example of how you can configure a Command Modular Input stanza to execute a remote tail command over SSH.
And likewise for the other commands :
If you are wondering “hey , where is the password ? ” , well it’s generally a best practice to use shared keys as detailed in these links.
http://linuxers.org/howto/how-run-commands-remote-machine-using-ssh
http://linuxers.org/article/ssh-login-without-password
Here are some screenshots of what the raw indexed output from these remotely executed commands looks like :
Here is a screencast of the remote tailing in action. In this example I have a Command Modular Input stanza executing a tail command to a Raspberry Pi 7000 miles away on the other side of the world , streaming file appends back to Splunk and searching over this in realtime.