We currently have an Elasticsearch compatibility test that runs all of our tests against different versions, including an AWS instance. However, the AWS instance is liable to problems and failures aren't uncommon. When this happens, it can be hard to detect real problems in other parallel runs against non-AWS Elasticsearch instances. For this reason, we should extract the AWS tests into a dedicated job.

ATs:

• The ES compatibility job no longer tests against AWS
• A new job tests only against AWS Elasticsearch