Import fails if suspended state of a processInstance is imported before the activity instance

• given:
  • I have an engine with a lot of data
  • I have a process instance P1 that produced the activities A1 and A2
  • I start Optimize and the import is running
• when:
  • P1 is imported and persisted to Elasticsearch
  • A1 and A2 are fetched from the engine afterward
• then:
  • the import fails with the exception:

    12:21:47.281 [ImportJobExecutor-pool-58] DEBUG o.c.o.s.e.r.EventSequenceCountReader - Fetching event sequences containing both event types: [EventTypeDto(group=Process_PolicyOrigination_Combi_ProductRequest_v1, source=camunda, eventName=EndEvent_RequestAccepted, eventLabel=null)] and [EventTypeDto(group=Process_PolicyOrigination_Combi_ProductRequest_v1, source=camunda, eventName=EndEvent_RequestAccepted, eventLabel=null)]
    12:21:47.285 [ImportJobExecutor-pool-58] DEBUG o.c.o.s.e.r.EventSequenceCountReader - Fetching event sequences containing both event types: [EventTypeDto(group=Process_PolicyOrigination_Combi_ProductRequest_v1, source=camunda, eventName=EndEvent_RequestAccepted, eventLabel=null)] and [EventTypeDto(group=Process_PolicyOrigination_Combi_ProductRequest_v1, source=camunda, eventName=EndEvent_RequestAccepted, eventLabel=null)]
    12:21:47.287 [ImportJobExecutor-pool-74] ERROR o.c.o.s.e.j.i.CompletedActivityInstanceElasticsearchImportJob - Error while executing import to elasticsearch
    org.camunda.optimize.service.exceptions.OptimizeRuntimeException: There were failures while performing bulk on Completed activity instances with message: failure in bulk execution:
    [11905]: index [optimize-nh1816-acc-process-instance_v5], type [_doc], id [6a19b5ed-b6bf-11ea-8c40-528611fdf5f7], message [ElasticsearchException[Elasticsearch exception [type=illegal_argument_exception, reason=failed to execute script]]; nested: ElasticsearchException[Elasticsearch exception [type=script_exception, reason=runtime error]]; nested: ElasticsearchException[Elasticsearch exception [type=null_pointer_exception, reason=Cannot invoke "Object.getClass()" because "receiver" is null]];]
    at org.camunda.optimize.service.es.writer.ElasticsearchWriterUtil.doBulkRequest(ElasticsearchWriterUtil.java:230)
    at org.camunda.optimize.service.es.writer.ElasticsearchWriterUtil.lambda$executeImportRequestsAsBulk$3(ElasticsearchWriterUtil.java:128)
    at java.util.HashMap.forEach(HashMap.java:1289)
    at org.camunda.optimize.service.es.writer.ElasticsearchWriterUtil.executeImportRequestsAsBulk(ElasticsearchWriterUtil.java:108)
    at org.camunda.optimize.service.es.job.importing.CompletedActivityInstanceElasticsearchImportJob.persistEntities(CompletedActivityInstanceElasticsearchImportJob.java:36)
    at org.camunda.optimize.service.es.job.ElasticsearchImportJob.executeImport(ElasticsearchImportJob.java:56)
    at org.camunda.optimize.service.es.job.ElasticsearchImportJob.run(ElasticsearchImportJob.java:37)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
    12:21:47.288 [ImportJobExecutor-pool-58] DEBUG o.c.o.s.e.r.EventSequenceCountReader - Fetching event sequences containing both event types: [EventTypeDto(group=Process_PolicyOrigination_Combi_ProductRequest_v1, source=camunda, eventName=SendTask_SendCalculatedProducts_end, eventLabel=null)] and [EventTypeDto(group=Process_PolicyOrigination_Combi_ProductRequest_v1, source=camunda, eventName=SendTask_SendCalculatedProducts_end, eventLabel=null)]
    12:21:47.291 [ImportJobExecutor-pool-58] DEBUG o.c.o.s.e.r.EventSequenceCountReader - Fetching event sequences containing both event types: [EventTypeDto(group=Process_PolicyOrigination_Combi_ProductRequest_v1, source=camunda, eventName=SendTask_SendCalculatedProducts_end, eventLabel=null)] and [EventTypeDto(group=Process_PolicyOrigination_Combi_ProductRequest_v1, source=camunda, eventName=SendTask_SendCalculatedProducts_end, eventLabel=null)]
    12:21:47.295 [ImportJobExecutor-pool-58] DEBUG o.c.o.s.e.r.EventSequenceCountReader - Fetching event sequences containing both event types: [EventTypeDto(group=Process_PolicyOrigination_Combi_ProductRequest_v1, source=camunda, eventName=SendTask_SendCalculatedProducts_end, eventLabel=null)] and [EventTypeDto(group=Process_PolicyOrigination_Combi_ProductRequest_v1, source=camunda, eventName=SendTask_SendCalculatedProducts_end, eventLabel=null)]
    12:21:47.298 [ImportJobExecutor-pool-58] DEBUG o.c.o.s.e.r.EventSequenceCountReader - Fetching event sequences containing both event types: [EventTypeDto(group=Process_PolicyOrigination_Combi_ProductRequest_v1, source=camunda, eventName=EndEvent_RequestAccepted, eventLabel=null)] and [EventTypeDto(group=Process_PolicyOrigination_Combi_ProductRequest_v1, source=camunda, eventName=EndEvent_RequestAccepted, eventLabel=null)]
    12:21:47.305 [ImportJobExecutor-pool-58] DEBUG o.c.o.s.e.r.EventSequenceCountReader - Fetching event sequences containing both event types: [EventTypeDto(group=Process_PolicyOrigination_Combi_ProductRequest_v1, source=camunda, eventName=EndEvent_RequestAccepted, eventLabel=null)] and [EventTypeDto(group=Process_PolicyOrigination_Combi_ProductRequest_v1, source=camunda, eventName=EndEvent_RequestAccepted, eventLabel=null)]
    

• expected:
  • the import works for all import permutations

Hints:

• We already have import permutation tests for user task data. We should have that for process instance, activity instance, and variable updates as well.
• Having a quick glance at the variable upsert script revealed that the variable import might have the same issue.
• To be sure we should also check the permutation process instance <-> user task import