-
Sub-task
-
Resolution: Done
-
L3 - Default
-
None
-
None
-
Not defined
Context:
See parent task:
The lower process instance count appears to be caused by the maximum page size limitation when our importers fetch data. When fetching, we query for all entities with timestamp equal or larger than the last imported entity timestamp. We also set a the maxResults to return to 10.000 (from ENGINE_IMPORT_PROCESS_INSTANCE_MAX_PAGE_SIZE config). Due to how we manipulate the startdates of process instances, we sometimes have more than 10k instances with the same timestamp. In this case, some instances will be missed by the import.
Since we don't need the date adjustments for this dataset, we would like to add a flag to our data generation with which we can specify whether or not dates should be adjusted. For the generation of this large dataset, we can then turn off the adjustment of startDate data.
After this is done we should regenerate the large dataset.
AT:
- A parameter is added to our data generation to indicate whether the dates of instances should be adjusted
- The large datast is regenerated with this parameter set to false