Context:
With 7.x.x Elastic decreased the default shard count per index from 5 to 1. As well as introduced a limit of a maximum of 1000 open shards per node.
We reconsider the default shardCount in the Optimize configuration (currently being 5) as well as whether there are certain indexes that may need different defaults than others.

E.g. with the introduction of camunda events OPT-2955 we introduced per definition indexes for which a lower shardcount compared to the overall processInstance index count would be feasible. As for those indexes query performance is not that crucial + they are read sequentially anyway so search queries are not really a use case there. This also applies to the event index.

AT:

• default shard count config is reevaluated
• documentation is updated to better explain the purpose of configurable shard size
• different use case indexes are identified and if considered reasonable a different default shardCount is introduced to be used, potential different use cases are e.g.:
  • the event index, camunda activity event indices which are written and read sequentially
  • event trace & Count indexes are not that crucial to yield the best query performance so a shard size of 1 could be fine for them
  • every index except the decision/process instance indices usually does not contain much data, the shardSize could easily get reduced to 1 for others