Elasticsearch Query Language ES|QL
o
s
e d li
David Pilato - @dadoonet Developer | Evangelist
& s
m e d
Slide 2
A brief history of Elasticsearch’s analytical capabilities
2010
2013
2014
2015
2023
Elasticsearch 0.9
Elasticsearch < 0.90
Elasticsearch 1.0
Elasticsearch 2.0
Elasticsearch 8.11
Facets
Facet terms-stats
Aggregations
Pipeline aggregations
ES|QL
Slide 3
ES|QL
• Language • Engine • Visualization
Slide 4
o
e d li
ES|QL in action https://github.com/dadoonet/esql-demo
s
& s
m e d
Slide 5
PROJECTIONS
Each language client will offer a selection of projections relevant to that language ecosystem.
RESULT DATA
Ways to consume ES|QL results
Users can consume raw data directly from the server output in one of several formats.
DataFrame
Object / Dict
Cursor
For mapping domain objects within a client application
For incremental consumption of results, with implicit pagination
For data science and analytics; integration with frameworks like Pandas
Text
CSV
JSON
Human-readable format ideal for interactive work, CLIs, etc
Raw CSV data to load directly into spreadsheets and ETL processes
Structured response containing metadata and data in a 2D value array
Bring your own Custom projections built atop raw server output
Apache Arrow Dataframe IPC format
Slide 6
Object API https://github.com/dadoonet/elasticsearch-java-client-demo String query = “”” FROM persons | WHERE name == “David” | KEEP name | LIMIT 1 “”“; Iterable<Person> persons = client.esql() .query(ObjectsEsqlAdapter.of(Person.class), query); for (Person person : persons) { assertNull(person.getId()); assertNotNull(person.getName()); }
Slide 7
ResultSet JDBC API https://github.com/dadoonet/elasticsearch-java-client-demo String query = “”” FROM persons | WHERE name == “David” | KEEP name | LIMIT 1 “”“; try (ResultSet resultSet = client.esql() .query(ResultSetEsqlAdapter.INSTANCE, query)) { assertTrue(resultSet.next()); assertEquals(“David”, resultSet.getString(1)); }
Slide 8
POST /_query
8. 16
{ “query”: “”” from logs-* | stats x = ?function(?field) by ?breakdownField
A better dashboard experience with named parameters
| where x >= ?value “”“, “params”: [ {“function” : {“identifier” : “avg”}}, {“field” : {“identifier” : “network.bytes”}}, {“breakdownField” : {“identifier” : “agent.name”}}, {“value”: 1000} ] }
Slide 9
TD
B
Slide 10
TD
B
Slide 11
TD
B
Slide 12
17
Coming next
WHERE MATCH(actors, “Marlon*”) WHERE QSTR(“bytes:[1024 TO 2048]”)
Slide 13
18
Coming next
WHERE KQL(“bytes>=1024”)
Slide 14
TB joinType JOIN indexName (AS qualifier)? condition? joinType: LOOKUP | LEFT | RIGHT | INNER condition: ON identifier == identifier | USING identifier
JOINS!
INLINESTATS total_visits = COUNT() FROM employees | SORT emp_no | LOOKUP JOIN languages_lookup ON language_code | KEEP emp_no, language_name
● No need to create an enrich policy ● A drag and drop experience in the UI
D
Slide 15
Elasticsearch Query Language ES|QL
o
s
e d li
David Pilato - @dadoonet Developer | Evangelist
& s
m e d