In this section we show runtime field examples that has been submitted by other users. Getting input from the Elastic Community is a great help. It enables Elasticsearch users all around the world to solve their needs more quickly. Thanks for all contributions that have been made.
Elastic Runtime field examples from Leaf
index_stats.index_date
This useful to extract all indices or data stream and find out the date of the index. This is useful for aggregating data usage.
if (doc['index_stats.index'].size() != 0) {
def index_name = doc['index_stats.index'].value;
if (index_name !=null) {
if (index_name =~ /[12][0-9]{3}./) { // match year pattern like 1903, 2021
Map my=grok('(%{GREEDYDATA:pattern}-%{INT:year}\\.%{INT:month}\\.%{INT:day})|(%{GREEDYDATA:pattern}-%{INT:year}\\.%{INT:month}\\.%{INT:day}-%{GREEDYDATA:junk})|(%{GREEDYDATA:pattern}-%{INT:year}-%{INT:month}-%{INT:day})').extract(index_name);
if (my !=null) {
emit(my.year+"."+my.month+"."+my.day);
}
}
}
}
index_stats.index_pattern
This useful to extract all indices or data stream and find out the index pattern the index. This is useful for aggregating data usage.
if (doc['index_stats.index'].size() != 0){
def index_name = doc['index_stats.index'].value;
if (index_name !=null) {
if (index_name =~ /[12][0-9]{3}./) { // match year pattern like 1903, 2021
Map my=grok('(%{GREEDYDATA:pattern}-%{INT:year}\\.%{INT:month}\\.%{INT:day})|(%{GREEDYDATA:pattern}-%{INT:year}\\.%{INT:month}\\.%{INT:day}-%{GREEDYDATA:junk2})|(%{GREEDYDATA:pattern}-%{INT:year}-%{INT:month}-%{INT:day})').extract(index_name);
if (my !=null){
emit(my.pattern);
}
}
} else {
emit(index_name);
}
}logger.category
You can use this to extract logger from applications to aggregate data in higher hierarchical level.
if (doc['logger.keyword'].size() != 0){
def fullname = doc['logger.keyword'].value;
def xpack="org.elasticsearch.xpack";
def http="org.elasticsearch.http";
def org="org.elasticsearch";
if (fullname !=null){
if (fullname =~ /org.elasticsearch.http/) {
def prefix = fullname.substring(http.length()+1);
int firstDotIndex = prefix.indexOf('.');
emit(prefix.substring(0,firstDotIndex));
}
else if (fullname =~ /org.elasticsearch.xpack/) {
def prefix = fullname.substring(xpack.length()+1);
int firstDotIndex = prefix.indexOf('.');
emit(prefix.substring(0,firstDotIndex));
}
else if (fullname =~ /org.elasticsearch/) {
def prefix = fullname.substring(org.length()+1);
int firstDotIndex = prefix.indexOf('.');
emit(prefix.substring(0,firstDotIndex));
}
}
}
version_for_sorting
This is because traditionally we have x.y.z as version number which is not sortable (where it treat as numerical sorting, so the order becomes 7.1.0, 7.10.1, 7.2.0 etc) and not queryable. One cannot say I want to check versions greater than x.y.z . But with 0x0y0z model, it can achieve both.
If you have permission to modify mapping of the index, you should change it to version type, then there will be no need to use runtime field here.
if (doc['version.keyword'].size() != 0){
def version = doc['version.keyword'].value;
if (version !=null){
int firstDot = version.indexOf('.');
int secondDot = version.indexOf('.', firstDot+1);
def major = version.substring(0,firstDot);
def minor = version.substring(firstDot+1,secondDot);
def patch = version.substring(secondDot+1);
while (major.length() < 2){
major="0"+major
}
while (minor.length() < 2){
minor="0"+minor
}
while (patch.length() < 2){
patch="0"+patch
}
emit(major+minor+patch);
}
}Elastic Runtime field examples from Graham
Sorted day of week
A day of the week that’s sortable in visualizations. This is an extension to the date math group of runtime fields. More date masks can be found in the Java docs.
ZonedDateTime input = doc['timestamp'].value;
String output = input.format(DateTimeFormatter.ofPattern('e')) + ' ' + input.format(DateTimeFormatter.ofPattern('E'));
emit(output);Return a static value when a condition is met
This runtime field can be useful for integrating event style information into visualizations. The example is using “record_score” from ML Anomalies to only show the anomalies over a certain score….but don’t show the score itself on the visualization. This runtime field accounts for empty values in the first conditional. You can take this example runtime field and easily adapt to any other numeric field. You also could add multiple categories and expose this field as a keyword field.
//Return a 1 when there is a specific value
if (doc['record_score'].size()==0) {
emit(0);
} else {
if (doc['record_score'].value > 0.5 ) {
emit(1);
} else {
emit(0);
}
}Elastic Runtime field examples from LeeDr
Parse specific value from a list
A field named `gh_labels.keyword` contains a list of text values.
For example: `pending_on_dev, kibana, Team:Control Plane/Ingress, cloud, assigned, closed:support, assign::cloud-dev`
But I want to do charts based on the `Team`. The steps are to create this “team” field as a keyword type;
1. check to make sure each doc contains the field
2. check that the field isn’t empty
3. check that the size isn’t 0 (this is probably redundant)
4. set variable labels equal to the value of the field
5. for each label, if it contains the string “Team:”, emit that string. In my case I want to strip “Team:” off so I get the substring starting at position 5. Return so we don’t spend any time comparing other strings in the list.
if (doc.containsKey('gh_labels.keyword')) {
if (!(doc['gh_labels.keyword'].empty)) {
if (!(doc['gh_labels.keyword'].size() == 0)) {
def labels = doc['gh_labels.keyword'];
for (int i = 0; i < labels.length; i++) {
if (labels[i].contains("Team:")) {
emit(labels[i].substring(5));
return;
}
}
}
}
}Elastic runtime field examples from Tre’ Seymour
Creating field that detects low test coverage of code called low_coverage
An easy way to see if code test-coverage is low for a given file.
def lines = doc["lines.total"].value;
def covered = doc["lines.covered"].value;
def threshold = lines / 2;
if (covered > threshold) {
emit(false);
} else {
emit(true);
}
Doeѕ your site have a contact page? I’m having trouble locating
it but, I’d like to send you an e-mail. I’ve got some ideas for your bⅼοg
you might be interested in hearing. Either wɑy, ցreat site and
I look forwаrd to seeing it improve over time.
You can find the contact me link in the footer of the page
This wasn’t explicitly stated, but I figured it out. If you want to create an ArrayList/ List / Keyword field, you need to repeatedly call emit().
Emit to populate a ‘list’ / keyword field – An example (using for loop)
def vals= doc[‘field_name.keyword’];
for (int i = 0; i < vals.length; i++) {
emit(labels[i]);
}
Hope someone finds this helpful!