Uploaded image for project: 'Camunda Optimize'
  1. Camunda Optimize
  2. OPT-3651

Deduplicate eventTrace events on write

    XMLWordPrintable

    Details

    • Type: Task
    • Status: Done
    • Priority: L3 - Default
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.1.0-alpha2, 3.1.0
    • Component/s: backend
    • Labels:
      None

      Description

      Context:
      We observed duplicate events being stored in an eventTrace due to concurrent writes that included the same events. These then caused a side-effect on the subsequent eventTrace sorting resulting in this exception:

      20:24:05.323 [ImportJobExecutor-pool-2] ERROR o.c.o.s.e.j.i.EventCountAndTracesImportJob - Error while executing import to elasticsearch
      java.lang.IllegalArgumentException: Comparison method violates its general contract!
      	at java.util.TimSort.mergeHi(TimSort.java:899)
      	at java.util.TimSort.mergeAt(TimSort.java:516)
      	at java.util.TimSort.mergeForceCollapse(TimSort.java:457)
      	at java.util.TimSort.sort(TimSort.java:254)
      	at java.util.Arrays.sort(Arrays.java:1512)
      	at java.util.ArrayList.sort(ArrayList.java:1462)
      	at org.camunda.optimize.service.EventTraceStateService.sortTracedEvents(EventTraceStateService.java:88)
      	at org.camunda.optimize.service.EventTraceStateService.lambda$updateTracesAndCountsForEvents$0(EventTraceStateService.java:72)
      	at java.util.stream.ReferencePipeline$11$1.accept(ReferencePipeline.java:372)
      	at java.util.HashMap$ValueSpliterator.forEachRemaining(HashMap.java:1628)
      	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
      	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
      	at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
      	at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
      	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
      	at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
      	at org.camunda.optimize.service.EventTraceStateService.updateTracesAndCountsForEvents(EventTraceStateService.java:74)
      	at org.camunda.optimize.service.es.job.importing.EventCountAndTracesImportJob.persistEntities(EventCountAndTracesImportJob.java:25)
      	at org.camunda.optimize.service.es.job.ElasticsearchImportJob.executeImport(ElasticsearchImportJob.java:56)
      	at org.camunda.optimize.service.es.job.ElasticsearchImportJob.run(ElasticsearchImportJob.java:37)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      	at java.lang.Thread.run(Thread.java:748)
      

      AT:

      • we should deduplicate identical events when writing eventTraces using an upsert script (like also done e.g. in CompletedActivityInstanceWriter)

        Attachments

          Activity

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            sebastian.bathke Sebastian Bathke
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: