Both Opensearch and Elasticsearch support semantic search. However, there are key differences, plus Elasticsearch has limited capability on the Basic license, allowing full semantic search features on the Platinum license and above.
Additionally, there are other vendors with semantic search solutions, such as vector databases, all-in-one search platform services, and Solr (another great open source option based on Lucene just like Elasticsearch and Opensearch). We won’t go over them specifically here, but will provide links to them at the end.
This table gives a brief overview of the similarities and differences in Elasticsearch vs Opensearch semantic search features. More detailed comparisons are provided below.
|Description of Feature
|Elasticsearch (Basic License)
|Elasticsearch (Platinum License)
|Embedding Field Type
|A mapping field type for indexing vector embeddings
|A type of similarity query
|Combine BM25 and kNN Results
|Can combine BM25 and KNN in one result set
|Yes, with hybrid query in v2.10+
|Ingestion Pipeline Inference Processing
|Allows transforming data into vector embeddings within search engine indexing
|Yes, limited to the default model 'lang_ident_model_1'
|Yes, via Neural Search
|Custom Transformer Models in Ingestion Pipeline
|Enables use of custom third party transformers, such as SBERT
|Reciprocal Rank Fusion
|A way to rerank multiple result sets into one based on rank
|Built-in Pre-trained Model (ELSER)
|Out-of-the-box model for attaining semantic vectors
Table last updated: 12-19-2023
If you use Elasticsearch or Opensearch for your data retrieval you’ve probably wondered if you should buy into the hype of AI powered search, and if you do, how would you even go about implementing it? Even then, the options and official documentation can be overwhelming!
In this article, we’ll define some terms, compare Elasticsearch and Opensearch implementations for hybrid search, as well as list some of the other options for AI powered search.
Our focus is to compare features to help you decide which solution to go with, so we won’t provide in-depth examples of how to implement hybrid search. However, we provide tons of helpful links to documentation and tutorials along the way.
Why Hybrid Search?
AI powered search usually comes in a couple of flavors: semantic search and personalized search. We’re focusing on semantic search. When combined with traditional text-based keyword search we get hybrid search.
How to Hybrid Search
Let’s walk through the basic steps for achieving hybrid search. Regardless of provider/implementation hybrid search looks like this:
- Ingest raw data
a. Analyze as sparse vector for BM25
b. Transform into dense vector embedding for kNN
- Search via text query
a. Analyze input for BM25 matching, find matches
b. Transform input for vector searching, find matches
- Combine results
a. Implement some sort of re-ranking algorithm to combine result sets
Let’s break each of these steps down and compare Elasticsearch and Opensearch’s implementations.
Sparse Vector Analysis
Elasticsearch and Opensearch have pretty similar processes for ingesting (aka “indexing) data for sparse vector retrieval, which is what is used for BM25. You’re probably familiar with creating a mapping file for an index, specifying what kind of analyzer to use, and maybe even creating custom analyzers using different token filters and tokenization. (If not, check out these articles to learn more about it).
Dense Vector Analysis
You can either create your own ingestion pipeline to transform data into vector embeddings, or take advantage of built-in inference processors in Elasticsearch and Opensearch.
Elasticsearch has a great page describing how to implement semantic search. There are basically two main ways to go about it: using the built in ELSER (Elastic Learned Sparse EncodeR) model, or specifying a custom model and using Elasticsearch’s
dense_vector field type.
Dense Vector Models
Elasticsearch launched the
dense_vector field type2 in version 7 (though starting in 7.3 it is only available in the x-pack distribution, meaning Elasticsearch 7.3 and up hosted on AWS Opensearch won’t have it – those installations will need to use the Opensearch k-NN plugin).
dense_vector field is the way to go if you want to perform your own vector embedding transformations via an external process (ie, a data pipeline that encodes documents into vectors via SBERT3 sentence transformers or some other readily available model) or with a custom model deployed to Elasticsearch to use in an inference processor in an ingest pipeline.
Gigasearch has a great blog post describing how to define a
dense_vector field and set up an ingestion pipeline using an inference processor (and implementing kNN and hybrid search) here4.
If you want to explore using a custom third party transformer without using an ingestion pipeline, checkout this Semantic Hybrid Search Google Colab notebook5. It shows how to use a transformer from hugging face to create your embeddings for your index, as well as transforming text queries into embeddings at search time in order to use an exact kNN search using script score.
If you don’t want or need the ability to use a custom model for your embeddings (and you’re on the Platinum license or higher), Elasticsearch provides the ELSER model6, which is an out-of-domain model (meaning no fine-tuning required) that uses Elasticsearch’s
rank_features field type7 (instead of
dense_vector) under the hood to store embeddings and search.
Elasticsearch has pretty good documentation on how to set up an index to use ELSER for ingestion and searching here8.
Opensearch uses just the one field type for storing vectors –
knn_vector9. However, before v2.5, vector embeddings needed to be generated externally (ie, a data pipeline to transform text to embeddings before indexing). In v2.5, the Neural Search plugin10 was introduced, and it was released to General Availability in v2.9.
The “Create and Store the Embeddings” section of the Semantic Hybrid Search Google Colab notebook5 could be used to index to an Opensearch
knn_vector field just as easily as Elasticsearch’s
Neural Search plugin
The Neural Search plugin enables features very similar to the inference processors in the Elasticsearch ingest pipeline. You can specify a transformer model in an ingest pipeline that will generate the
knn_vector embeddings for you at index time.
For an example of how to set up an ingestion pipeline with a custom transformer checkout this blog post11 from Opensearch.
Approximate kNN search is used to optimize for latency on large datasets (tens of thousands of docs or more). As expected, the algorithms used are a little less precise than Exact kNN search, which uses a script, and is better for smaller datasets, or when a pre-filter is used to narrow down the documents to execute kNN search against.
Both Elasticsearch and Opensearch offer Approximate kNN and Exact kNN search capabilities. Elasticsearch also offers the ELSER model approach, which is slightly different. In addition, both allow you to specify either a vector or text string as the query input value.
Simply use the
knn object on the
_search endpoint as shown in the documentation12.
Instead of a kNN or script score query, Elasticsearch provides a
text_expansion query to use for ELSER queries. Again, the tutorial on ELSER8 has great examples.
Simply use the
knn query type as shown in the documentation14.
Similar to Elasticsearch, Exact k-NN requires using a script score query, but instead of specifying a vector method, you actually just specify
"knn_score" as the source for the script, and
"knn" as the language. See the documentation15 for more info.
Though there doesn’t seem to be explicit documentation stating whether Neural search uses approximate or exact k-NN search, we know it can be combined with the different types of k-NN filtering defined here16.
Given this, we can assume that neural search most likely performs approximate kNN search (since it’s most efficient) unless a pre-filter script score is applied (in which case exact kNN search would be used), or if Efficient kNN filtering is applied (in which case, the algorithm decides whether to use approximate or exact k-NN search).
The advantage of neural search over either k-NN search is the ability to use a transformer model to convert input text to vector embedding at query time, rather than pre-computing the vector embedding before generating the Opensearch query.
The default method for combining kNN and regular search results17 is to simply specify a query section and knn section to the
_search endpoint, with boosts for each section. The boosts will be multiplied by the corresponding section score, and then added together.
score = query_boost * query_score + knn_boost * knn_score
The ELSER method uses a boolean query to combine scores18. Boosts are used similarly to the default combination method.
Reciprocal Rank Fusion has been shown to yield more relevant results than either other combination methods for hybrid search. Elasticsearch enables it via the
rank object on the
You can also use script score to combine results as shown in Semantic Hybrid Search Google Colab notebook5. (This is especially useful when using exact kNN search).
The documentation for Neural Search10 has a pretty clear example of how to combine bm25 with Neural Search scores using script score, very similar to the Elasticsearch approach.
In v2.10, Opensearch introduced the
normalization-processor, which allows the user to control how bm25 and neural query results are combined in a hybrid query.
Conclusion: Which One Should You Pick?
Given the similarities, whichever search provider you’re currently using will probably be that one that requires the least work to get hybrid semantic search running. That being said, here are some considerations that might prompt you to pick one over the other:
- If you don’t want to deal with language transformer models at all, and want the simplest, quickest out-of-the-box solution – Elasticsearch’s ELSER solution is probably the way to go.
- If you highly value fully open source projects, you should go with Opensearch, since Elasticsearch is no longer fully open since version 7.10.
- If you want the ability to combine results with Reciprocal Rank Fusion within the search engine (and not have to build out a function in your search api to implement rrf), Openserach doesn’t yet provide this, nor is it guaranteed that it will (combining scores via normalization processor is available starting v2.1020), so you should probably go with Elasticsearch.
- If you use Elasticsearch Basic license and can’t or won’t upgrade to the Platinum license, it might be worth considering switching to Opensearch to get more functionality without extra licensing cost.
- If you don’t currently have a search installation with either Elasticsearch or Opensearch, you can’t go wrong with either. Elasticsearch seems to be a little ahead in its released features to support semantic search, but Opensearch is not far behind. The difference in licensing/pricing is likely to be the biggest factor that won’t change any time soon, whereas the released features are changing very quickly. You might also consider a vector database or one of the other options below if you’re greenfielding a search solution.
Other Options for AI Powered Search
There are quite a few different options for vendors/providers when it comes to implementing AI powered search. While we focused on Elasticsearch and Opensearch in this article, it’s worth calling out vector database solutions, as they tend to have better performance for storing and retrieving vectors.
A vector database might be a good solution if you’re implementing a green-field search solution and don’t have to worry about migrating from an existing system.
Additionally, there are more and more platform as a service solutions offering search powered by AI, so we’ve listed a few of those options as well for those who would rather be more hands off.
Open Source Lucene Based
Solr (fully open source)
AI Search Platforms
Sources and Links
Hybrid Search performs better than BM25 and Vector Search alone (Sources)
Gigasearch Tutorial - Semantic Hybrid Search Google Colab notebook
Elasticsearch Tutorial - Semantic Search with ELSER
Opensearch Tutorial - Similar Document Search