Save up to 30% in storage and up to 90% in memory per index! No change needed except upgrading to the latest version of the Elasticstack
With the newest version of the Elasticstack, the Devs have targeted one of the major pain-points that are coming with Elastic: the scalability in regards of memory and fast, expensive storage. Especially clusters with hundreds or even thousands of indices are notorious for query timeouts and will take profit of an upgrade.
The comparison: 7.13.4 vs. 7.16.3
I ran a test with two VMs. One machine had Elasticsearch 7.13.4 installed, the other 7.16.3. Both machines had 1 virtual CPU and 8000MB memory. OS is Ubuntu 18.04.6 LTS. Both machines had an index “testdata” with around 8 million documents. The performance test was done with an “update_by_query”, where a field was splitted into an array and stored as a new field over each document of the index.
Conclusions from the test
- the task was done by version 7.16 (22mins) a lot faster than by 7.13 (1 hour)
- Memory Usage of Lucene “Version Map” as well as of Index Memory was in average loosely a tenth that 7.16 needed compared to 7.13
- 7.16 needs around 12GB storage while processing the task, 7.13 needs up to 18 GB.
- The latency is mostly 0.34ms, although 7.16 had 2 nasty spikes. Maybe there could be space for improvements
- Request time seems to be slightly higher for 7.16 (800ms) vs 7.13 (750ms)
For more information, please consult the Elastic presentation at https://www.elastic.co
Index stats for 7.16
Index stats for 7.13
