Elasticsearch is a highly scalable data store that allows you to ingest, search and aggregate millions of records with ease. Recently a customer of mine was running into some performance issues with their multi-node deployment. The cluster was required to ingest around 90,000 1kb records per second and their configuration looked like this:

  • Two physical servers with 40 cores, 256GB RAM and 8x 1TB HDDs
  • Each server ran 7 nodes on the host OS
  • Each node was allocated 24GB RAM (maximum should be 32GB)
  • Each node was allocated 5 cores (see Processors setting)
  • Each node was allocated one

Read more