Uploaded image for project: 'Camunda Optimize'
  1. Camunda Optimize
  2. OPT-6049

Zeebe Data Generation Improvements

XMLWordPrintable

    • Icon: Task Task
    • Resolution: Done
    • Icon: L3 - Default L3 - Default
    • 3.9.0, 3.9.0-alpha3
    • None
    • backend
    • None
    • Not defined

      Context:

      With OPT-5953, we have added a basic zeebe data generator job to generate data we can use in import performance tests. We can iterate on this generator to improve it by:

      • ability to create different data snapshots, eg one 1M instance one for import performance tests and one 100M instance dataset for query performance tests
        • ability to select name/filter of snapshot as job parameter done
      • consider adding manipulation of when instances are exported to create zeebe record indices spread over several days
      • consider adding the job worker count as a configurable parameter done
      • consider adding more variety to deployed instances/definitions
      • extract methods shared between zeebe data generation and import performance into shared jenkins lib (eg ES snapshot related code)
      • switch to stable node pool to avoid issues with longer running jobs done

      AT:

      • tbd depending on which of the above are to be implemented, might also extract tasks into separate tickets

        This is the controller panel for Smart Panels app

          1.
          		 Add parameter for jobworker count to zeebe data generation Sub-task Done Unassigned
          2.
          		 Add ability to store/create multiple different snapshots Sub-task Done Unassigned

            [OPT-6049] Zeebe Data Generation Improvements

            Helene Waechtler created issue -
            Helene Waechtler made changes -
            Labels Original: current_release
            Helene Waechtler made changes -
            Description Original: *Context:*

            With OPT-5953, we have added a basic zeebe data generator job to generate data we can use in import performance tests. We can iterate on this generator to improve it by:
             * consider adding manipulation of when instances are exported to create zeebe record indices spread over several days
             * consider adding the job worker count as a configurable parameter
             * consider adding more variety to deployed instances/definitions

            *AT:*
             * tbd depending on which of the above are to be implemented             		New: *Context:*

            With OPT-5953, we have added a basic zeebe data generator job to generate data we can use in import performance tests. We can iterate on this generator to improve it by:
             * consider adding manipulation of when instances are exported to create zeebe record indices spread over several days
             * consider adding the job worker count as a configurable parameter
             * consider adding more variety to deployed instances/definitions
             * extract methods shared between zeebe data generatio and import performance into shared jenkins lib (eg ES snapshot related code)

            *AT:*
             * tbd depending on which of the above are to be implemented
            Helene Waechtler made changes -
            Description Original: *Context:*

            With OPT-5953, we have added a basic zeebe data generator job to generate data we can use in import performance tests. We can iterate on this generator to improve it by:
             * consider adding manipulation of when instances are exported to create zeebe record indices spread over several days
             * consider adding the job worker count as a configurable parameter
             * consider adding more variety to deployed instances/definitions
             * extract methods shared between zeebe data generatio and import performance into shared jenkins lib (eg ES snapshot related code)

            *AT:*
             * tbd depending on which of the above are to be implemented             		New: *Context:*

            With OPT-5953, we have added a basic zeebe data generator job to generate data we can use in import performance tests. We can iterate on this generator to improve it by:
             * ability to create different data snapshots, eg one 1M instance one for import performance tests and one 100M instance dataset for query performance tests
             * consider adding manipulation of when instances are exported to create zeebe record indices spread over several days
             * consider adding the job worker count as a configurable parameter
             * consider adding more variety to deployed instances/definitions
             * extract methods shared between zeebe data generation and import performance into shared jenkins lib (eg ES snapshot related code)

            *AT:*
             * tbd depending on which of the above are to be implemented, might also extract tasks into separate tickets
            Helene Waechtler made changes -
            Description Original: *Context:*

            With OPT-5953, we have added a basic zeebe data generator job to generate data we can use in import performance tests. We can iterate on this generator to improve it by:
             * ability to create different data snapshots, eg one 1M instance one for import performance tests and one 100M instance dataset for query performance tests
             * consider adding manipulation of when instances are exported to create zeebe record indices spread over several days
             * consider adding the job worker count as a configurable parameter
             * consider adding more variety to deployed instances/definitions
             * extract methods shared between zeebe data generation and import performance into shared jenkins lib (eg ES snapshot related code)

            *AT:*
             * tbd depending on which of the above are to be implemented, might also extract tasks into separate tickets             		New: *Context:*

            With OPT-5953, we have added a basic zeebe data generator job to generate data we can use in import performance tests. We can iterate on this generator to improve it by:
             * ability to create different data snapshots, eg one 1M instance one for import performance tests and one 100M instance dataset for query performance tests
             ** ability to select name/filter of snapshot as job parameter
             * consider adding manipulation of when instances are exported to create zeebe record indices spread over several days
             * consider adding the job worker count as a configurable parameter
             * consider adding more variety to deployed instances/definitions
             * extract methods shared between zeebe data generation and import performance into shared jenkins lib (eg ES snapshot related code)

            *AT:*
             * tbd depending on which of the above are to be implemented, might also extract tasks into separate tickets
            Helene Waechtler made changes -
            Labels New: current_release
            Helene Waechtler made changes -
            Description Original: *Context:*

            With OPT-5953, we have added a basic zeebe data generator job to generate data we can use in import performance tests. We can iterate on this generator to improve it by:
             * ability to create different data snapshots, eg one 1M instance one for import performance tests and one 100M instance dataset for query performance tests
             ** ability to select name/filter of snapshot as job parameter
             * consider adding manipulation of when instances are exported to create zeebe record indices spread over several days
             * consider adding the job worker count as a configurable parameter
             * consider adding more variety to deployed instances/definitions
             * extract methods shared between zeebe data generation and import performance into shared jenkins lib (eg ES snapshot related code)

            *AT:*
             * tbd depending on which of the above are to be implemented, might also extract tasks into separate tickets             		New: *Context:*

            With OPT-5953, we have added a basic zeebe data generator job to generate data we can use in import performance tests. We can iterate on this generator to improve it by:
             * ability to create different data snapshots, eg one 1M instance one for import performance tests and one 100M instance dataset for query performance tests
             ** ability to select name/filter of snapshot as job parameter
             * consider adding manipulation of when instances are exported to create zeebe record indices spread over several days
             * consider adding the job worker count as a configurable parameter
             * consider adding more variety to deployed instances/definitions
             * extract methods shared between zeebe data generation and import performance into shared jenkins lib (eg ES snapshot related code)
             * switch to stable node to avoid issues with longer running jobs

            *AT:*
             * tbd depending on which of the above are to be implemented, might also extract tasks into separate tickets
            Helene Waechtler made changes -
            Description Original: *Context:*

            With OPT-5953, we have added a basic zeebe data generator job to generate data we can use in import performance tests. We can iterate on this generator to improve it by:
             * ability to create different data snapshots, eg one 1M instance one for import performance tests and one 100M instance dataset for query performance tests
             ** ability to select name/filter of snapshot as job parameter
             * consider adding manipulation of when instances are exported to create zeebe record indices spread over several days
             * consider adding the job worker count as a configurable parameter
             * consider adding more variety to deployed instances/definitions
             * extract methods shared between zeebe data generation and import performance into shared jenkins lib (eg ES snapshot related code)
             * switch to stable node to avoid issues with longer running jobs

            *AT:*
             * tbd depending on which of the above are to be implemented, might also extract tasks into separate tickets             		New: *Context:*

            With OPT-5953, we have added a basic zeebe data generator job to generate data we can use in import performance tests. We can iterate on this generator to improve it by:
             * ability to create different data snapshots, eg one 1M instance one for import performance tests and one 100M instance dataset for query performance tests
             ** ability to select name/filter of snapshot as job parameter
             * consider adding manipulation of when instances are exported to create zeebe record indices spread over several days
             * consider adding the job worker count as a configurable parameter
             * consider adding more variety to deployed instances/definitions
             * extract methods shared between zeebe data generation and import performance into shared jenkins lib (eg ES snapshot related code)
             * switch to stable node pool to avoid issues with longer running jobs

            *AT:*
             * tbd depending on which of the above are to be implemented, might also extract tasks into separate tickets
            Helene Waechtler made changes -
            Description Original: *Context:*

            With OPT-5953, we have added a basic zeebe data generator job to generate data we can use in import performance tests. We can iterate on this generator to improve it by:
             * ability to create different data snapshots, eg one 1M instance one for import performance tests and one 100M instance dataset for query performance tests
             ** ability to select name/filter of snapshot as job parameter
             * consider adding manipulation of when instances are exported to create zeebe record indices spread over several days
             * consider adding the job worker count as a configurable parameter
             * consider adding more variety to deployed instances/definitions
             * extract methods shared between zeebe data generation and import performance into shared jenkins lib (eg ES snapshot related code)
             * switch to stable node pool to avoid issues with longer running jobs

            *AT:*
             * tbd depending on which of the above are to be implemented, might also extract tasks into separate tickets             		New: *Context:*

            With OPT-5953, we have added a basic zeebe data generator job to generate data we can use in import performance tests. We can iterate on this generator to improve it by:
             * ability to create different data snapshots, eg one 1M instance one for import performance tests and one 100M instance dataset for query performance tests
             ** ability to select name/filter of snapshot as job parameter
             * consider adding manipulation of when instances are exported to create zeebe record indices spread over several days
             * -consider adding the job worker count as a configurable parameter- done
             * consider adding more variety to deployed instances/definitions
             * extract methods shared between zeebe data generation and import performance into shared jenkins lib (eg ES snapshot related code)
             * switch to stable node pool to avoid issues with longer running jobs

            *AT:*
             * tbd depending on which of the above are to be implemented, might also extract tasks into separate tickets
            Helene Waechtler made changes -
            Description Original: *Context:*

            With OPT-5953, we have added a basic zeebe data generator job to generate data we can use in import performance tests. We can iterate on this generator to improve it by:
             * ability to create different data snapshots, eg one 1M instance one for import performance tests and one 100M instance dataset for query performance tests
             ** ability to select name/filter of snapshot as job parameter
             * consider adding manipulation of when instances are exported to create zeebe record indices spread over several days
             * -consider adding the job worker count as a configurable parameter- done
             * consider adding more variety to deployed instances/definitions
             * extract methods shared between zeebe data generation and import performance into shared jenkins lib (eg ES snapshot related code)
             * switch to stable node pool to avoid issues with longer running jobs

            *AT:*
             * tbd depending on which of the above are to be implemented, might also extract tasks into separate tickets             		New: *Context:*

            With OPT-5953, we have added a basic zeebe data generator job to generate data we can use in import performance tests. We can iterate on this generator to improve it by:
             * ability to create different data snapshots, eg one 1M instance one for import performance tests and one 100M instance dataset for query performance tests
             ** ability to select name/filter of snapshot as job parameter
             * consider adding manipulation of when instances are exported to create zeebe record indices spread over several days
             * -consider adding the job worker count as a configurable parameter- done
             * consider adding more variety to deployed instances/definitions
             * extract methods shared between zeebe data generation and import performance into shared jenkins lib (eg ES snapshot related code)
             * -switch to stable node pool to avoid issues with longer running jobs- done

            *AT:*
             * tbd depending on which of the above are to be implemented, might also extract tasks into separate tickets

              Unassigned Unassigned
              helene.waechtler Helene Waechtler
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: