Modular Inputs are a great addition to Splunk Enterprise.  One of the things I really like about Modular Inputs is that they allow you to create inputs that “look and feel” as if they were part of the Splunk installation by providing a nice user interface for parameter input.

But, what if you need to encrypt a Modular Input value?  This could be a password, OAuth secret key, or some other confidential piece of information.  Traditional Splunk applications use setup.xml and the storage/passwords endpoint to accomplish this.  If you just need to encrypt an input value specific to the input (as opposed to the entire application), it may be cumbersome to the end user to first run through a setup.xml UI and then the Modular Input UI.  In this blog post, I will show you a technique to encrypt input values without going through a separate setup.xml process.

In this example, we will use a simple username/password combination.  This technique can apply to any field you want encrypted though.

The Technique

When our modular input code runs (which happens immediately after creating the input), the following will happen:

1. Retrieve the input parameters from inputs.conf for the modular input.
2. Check if the field we want to encrypt is clear text.
3. If the field is clear text, create an encrypted credential and mask the field in inputs.conf.
4. Decrypt the credential so that we can use the clear password in our code.

The Result

After installing the sample application code, go to Settings -> Data Inputs -> Splunk Modular Input Credential Example.

Create a new Input (this is a very simple example, but you can have as many fields as you want)

After clicking the “Next” button, the password is encrypted (creating a passwords.conf file in the local directory of the application) and masked in inputs.conf (in the same local directory).

Resulting local/inputs.conf

[splunk_modinput_cred_example://Testing123]
password = <nothing to see here>
username = Jason

Resulting local/passwords.conf

[credential::Jason:]
password = $1$oVfptNrGUg==

The code

First, we will set up some global variables to use anywhere in the code:

class MyScript(Script):
    # Define some global variables
    MASK           = "<nothing to see here>"
    APP            = __file__.split(os.sep)[-3]
    USERNAME       = None
    CLEAR_PASSWORD = None

The stream_events method is the entry point for the modular input code.  We check to see if the password is masked here and take action if it is not.

def stream_events(self, inputs, ew):
    self.input_name, self.input_items = inputs.inputs.popitem()
    session_key = self._input_definition.metadata["session_key"]
    username = self.input_items["username"]
    password = self.input_items['password']
    self.USERNAME = username
    try:
        # If the password is not masked, mask it.
        if password != self.MASK:
            self.encrypt_password(username, password, session_key)
            self.mask_password(session_key, username)
        self.CLEAR_PASSWORD = self.get_password(session_key, username)
    except Exception as e:
        ew.log("ERROR", "Error: %s" % str(e))
    ew.log("INFO", "USERNAME:%s CLEAR_PASSWORD:%s" % (self.USERNAME, self.CLEAR_PASSWORD))

Here is how we encrypt the password (or any confidential piece of information).

def encrypt_password(self, username, password, session_key):
    args = {'token':session_key}
    service = client.connect(**args)
    try:
        # If the credential already exists, delte it.
        for storage_password in service.storage_passwords:
            if storage_password.username == username:
                service.storage_passwords.delete(username=storage_password.username)
                break
        # Create the credential.
        service.storage_passwords.create(password, username)
    except Exception as e:
        raise Exception, "An error occurred updating credentials. Please ensure your user account has admin_all_objects and/or list_storage_passwords capabilities. Details: %s" % str(e)

And, here is the masking.

def mask_password(self, session_key, username):
    try:
        args = {'token':session_key}
        service = client.connect(**args)
        kind, input_name = self.input_name.split("://")
        item = service.inputs.__getitem__((input_name, kind))
        kwargs = {
            "username": username,
            "password": self.MASK
        }
        item.update(**kwargs).refresh()
    except Exception as e:
        raise Exception("Error updating inputs.conf: %s" % str(e))

Finally, the method to decrypt the credential.

def get_password(self, session_key, username):
    args = {'token':session_key}
    service = client.connect(**args)
    # Retrieve the password from the storage/passwords endpoint 
    for storage_password in service.storage_passwords:
        if storage_password.username == username:
            return storage_password.content.clear_password

Running the following search shows the output from the INFO log message in the stream_events method displaying the clear text username and password (do not do this in your actual code as this was done just to show that we did, in fact, decrypt the password):

index=_internal source="/applications/splunk/var/log/splunk/splunkd.log"

Putting it all together

A complete working example can be found on GitHub here -> https://github.com/JasonConger/TA_modinput_cred-example

Notes:

SDK – this example makes use of the Splunk Python SDK to abstract a lot of the REST API plumbing for you.

Capabilities  – in order for this example to work, the user creating the modular input needs to have the admin_all_objects capability.

Passwords.conf file – credentials are stored in a separate passwords.conf file starting with Splunk version 6.3.0

Distributed Environments – encrypting/decrypting relies on a splunk.secret key.  In a search head cluster, the captain replicates its splunk.secret file to all other cluster members during initial deployment of the cluster. Reference http://docs.splunk.com/Documentation/Splunk/latest/Security/Deploysecurepasswordsacrossmultipleservers