🎹🎻🎸 Searching for similar music tracks 🎼🎶
A presentation at Geek Sessions Conf in in Faro, Portugal by David Pilato
🎹 Searching for similar musics 🎻🎸 David Pilato @pilato.fr @dadoonet
Elasticsearch You Know, for Search
These are not the droids you are looking for.
GET /_analyze { “char_filter”: [ “html_strip” ], “tokenizer”: “standard”, “filter”: [ “lowercase”, “stop”, “snowball” ], “text”: “These are <em>not</em> the droids you are looking for.” }
“char_filter”: “html_strip” These are <em>not</em> the droids you are looking for. These are not the droids you are looking for.
“tokenizer”: “standard” These are not the droids you are looking for. These are not the droids you are looking for
“filter”: “lowercase” These are not the droids you are looking for these are not the droids you are looking for
“filter”: “stop” These are not the droids you are looking for these are not the droids you are looking for droids you looking
“filter”: “snowball” These are not the droids you are looking for these are not the droids you are looking for droids you droid you looking look
These are <em>not</em> the droids you are looking for. { “tokens”: [{ “token”: “droid”, “start_offset”: 27, “end_offset”: 33, “type”: “<ALPHANUM>”, “position”: 4 },{ “token”: “you”, “start_offset”: 34, “end_offset”: 37, “type”: “<ALPHANUM>”, “position”: 5 }, { “token”: “look”, “start_offset”: 42, “end_offset”: 49, “type”: “<ALPHANUM>”, “position”: 7 }]}
Semantic search ≠ Literal matches
Elasticsearch You Know, for Search
Elasticsearch You Know, for Vector Search
What is a Vector ?
Example: 1-dimensional vector Character Vector [ 1 ] ] Realistic
[ Embeddings represent your data Cartoon 1
represent different data aspects Human Character Vector [ 1, 1 Realistic Cartoon ] ] Machine
[ Multiple dimensions 1, 0
is grouped together Human Character Vector [ 1.0, 1.0 1.0, 0.0 Realistic Cartoon [ 1.0, 0.8 1.0, 1.0 [ 1.0, 1.0 ] ] ] ] ]
Machine
[ [ Similar data
Vector search ranks objects by similarity (~relevance) to the query Human Rank Query 1 Realistic Cartoon 2 3 4 5 Machine Result
How do you index vectors ?
Architecture of Vector Search
dense_vector field type PUT ecommerce { “mappings”: { “properties”: { “description”: { “type”: “text” } “desc_embedding”: { “type”: “dense_vector” } } } }
Data Ingestion and Embedding Generation POST /ecommerce/_doc { “_id”:”product-1234”, “product_name”:”Summer Dress”, “description”:”Our best-selling…”, “Price”: 118, “color”:”blue”, “fabric”:”cotton”, “fabric”:”cotton” “desc_embedding”:[0.452,0.3242,…], } “desc_embedding”:[0.452,0.3242,…] } “img_embedding”:[0.012,0.0,…] } Source data POST /ecommerce/_doc
How do you search vectors ?
Architecture of Vector Search
knn query GET /ecommerce/_search { “query” : { “bool”: { “must”: [{ “knn”: { “field”: “desc_embbeding”, “query_vector”: [0.123, 0.244,…] } }], “filter”: { “term”: { “department”: “women” } } } } }, “size”: 10
Architecture of Vector Search
But how does it really work?
Similarity Human q cos(θ) = d1 d2 Realistic θ q⃗ × d ⃗ | q⃗ | × | d |⃗ _score = 1 + cos(θ) 2
Similarity: cosine (cosine) θ Similar vectors θ close to 0 cos(θ) close to 1 1+1 _score = =1 2 θ Orthogonal vectors θ close to 90° cos(θ) close to 0 1+0 _score = = 0.5 2 θ Opposite vectors θ close to 180° cos(θ) close to -1 1−1 _score = =0 2
https://djdadoo.pilato.fr/
https://github.com/dadoonet/music-search/
🎹 Searching for similar musics 🎻🎸 David Pilato @pilato.fr @dadoonet Sli de s &d em o
In this session, we will use the principles of vector search to find pieces of music that are (maybe) similar to others. To do this, we will review the principles of generating embeddings to represent any type of data, whether textual or binary.
Resources
The following resources were mentioned during the presentation or are useful additional information.
-
Event page
-
Demo source code
The Jupyter notebook
-
Podcast
This space is where I’m publishing my DJ sets
Buzz and feedback
Here’s what was said about this presentation on social media.
