BOOK EXCERPT: When to use “transaction” and when to use “stats”

There are several ways to group events with the Search Processing Language (SPL). The most common approach uses either the transaction or stats command. But when should you use transaction and when should you use stats?

The rule of thumb: If you can use stats, use stats. It’s faster than transaction, especially in a distributed environment. With that speed, however, comes some limitations. You can only group events with stats if they have at least one common field value and if you require no other constraints. Typically, the raw event text is discarded.

Like stats, the transaction command can group events based on common field values, but it can also use more complex constraints such as total time span of the transaction, delays between events within the transaction, and required beginning and ending events. Unlike stats, transaction retains the raw event text and field values from the original events, but it does not compute any statistics over the grouped events, other than the duration (the delta of the _time field between oldest and newest events in the transaction) and the eventcount (the total number of events in the transaction).

The transaction command is most useful in two specific cases:

• When unique field values (also known as identifiers) are not sufficient to discriminate between discrete transactions. This is the
  case when an identifier might be reused, for example in web sessions identified by cookie/client IP. In this case, timespans or pauses should be used to segment the data into transactions. In other cases, when an identifier is reused, for example in DHCP logs, a particular message may identify the beginning or end of a transaction.
• When it is desirable to see the raw text of the events rather than an analysis on the constituent fields of the events.

Again, when neither of these cases is applicable, it is a better practice to use stats, as search performance for stats is generally better than transaction. Often there is a unique identifier, and stats can be used.

For example, to compute statistics on the duration of trades identified by the unique identifier trade_id, the following searches yield the same answer:

   ... | transaction trade_id
     | chart count by duration

   ... | stats range(_time) as duration by trade_id
     | chart count by duration

The second search is more efficient.

However, if trade_id values are reused but the last event of each trade is indicated by the text “END”, the only viable solution is:

   ... | transaction trade_id endswith=END
     | chart count by duration

If, instead of an end condition, trade_id values are not reused within 10 minutes, the most viable solution is:

   ... | transaction trade_id maxpause=10m
     | chart count by duration

Finally, a brief word about performance. No matter what search commands you use, it’s imperative for performance that you make the base search as specific as possible. Consider this search:

   sourcetype=x | transaction field=ip maxpause=15s | search
   ip=1.2.3.4

Here we are retrieving all events of sourcetype=x, building up transactions, and then throwing away any that don’t have an ip=1.2.3.4. If all your events have the same ip value, this search should be:

   sourcetype=x ip=1.2.3.4 | transaction field=ip maxpause=15s

This search retrieves only the events it needs to and is much more efficient. More about this is in “Finding Specific Transactions” later in this chapter.