Elastic runtime field example: Regular expressions (Regex)

Using regex in a runtime field can be very useful to get specific text within your source field and use it as value for your target runtime field. In most cases you will do this to extract a specific substring and use this as a keyword for your visualizations in Kibana. While this makes a lot of sense you need to be careful using Regex in a runtime field. The regex expressions can get very heavy to compute. This can reduce the performance significantly.

Remember that all the regex expression in a runtime will get executed for every document while querying. Elastic also allows to use dissect and grok patterns in runtime fields. Both are good alternatives in comparison to use plain regex. Using the regex pattern should be always the last option to choose.

If you want to execute a regex on a field that is mapped as text you need to follow these guidelines to make this available. However the conclusion is that you should always use keyword fields instead of text fields for the regex extraction.

def fieldname = "host.name";
if (!doc.containsKey(fieldname)) {
    return
}

def m = /([A-Za-z0-9]+)/.matcher(doc[fieldname].value);
if ( m.matches() ) {
    emit (m.group(1))
}

The first part is checking that the target field name exists. If you want to regex in a sparse field this prevents shard failures on search execution. The second part is defining the regex pattern and the field to extract to values from. If you use more complex regex scripts you may also need to change the parameter of m.group() as this is defining which part of the regex match should get emitted.

For a simple check that a regex pattern is present in longer string you can also use this shorthanded syntax.

if (doc['host.name'].value =~ /([A-Za-z0-9]+)/) { 
    emit("value detected");
} else {
    emit ("value not detected");
}

… or even shorter if you use the Boolean result of the comparison directly. Make sure to set the format of the field to Boolean.

emit (doc['host.name'].value =~ /([A-Za-z0-9]+)/)

About the author: Creator of the Elastic Content Share.