Uploaded image for project: 'camunda BPM'
  1. camunda BPM
  2. CAM-5284

I can use a long error message with an External Task

    • Icon: Feature Request Feature Request
    • Resolution: Fixed
    • Icon: L3 - Default L3 - Default
    • 7.6.0, 7.6.0-alpha3
    • None
    • engine

      While working on a new model, our service tasks generated errors with backtraces which were too large to be stored in the various message related fields in the database. The engine crashed very ungracefully when that happened. Our fix was to change the table definitions so that instead of making message fields to have type `varchar(4000)`, they're defined as `text`.

      I'm not sure if that is really the best fix. It might be better to have a limit and enforce it by the engine. In any case, running into a database error causes trouble that requires an engine restart, which should be avoided.

      I tried to attach our changed `sql/create/postgres_engine_7.4.1-ee.sql` file for your consideration, but JIRA gave me the error message "No project could be found with id '10330'. Something on JIRA's end seems to be broken for file uploads. The only change that I made was the column type change as described above.

        This is the controller panel for Smart Panels app

            [CAM-5284] I can use a long error message with an External Task

            Hi Daniel,

            I am unable to reproduce the complete crash, but if I encounter it again, I will let you know. It seemed to be a follow-on problem to the original issue caused by the lack of space in a column. I am currently working with a modified schema that does not have a size restriction on the various message fields (which I prefer to truncating the message anyway), so it is unlikely that I'm going to run into this by accident now.

            We can live with using a non-standard schema for now, yet this is only a temporary solution and we'd hope that this bug be fixed for external tasks so that we can safely upgrade our database schema from your scripts.

            Hans Hübner added a comment - Hi Daniel, I am unable to reproduce the complete crash, but if I encounter it again, I will let you know. It seemed to be a follow-on problem to the original issue caused by the lack of space in a column. I am currently working with a modified schema that does not have a size restriction on the various message fields (which I prefer to truncating the message anyway), so it is unlikely that I'm going to run into this by accident now. We can live with using a non-standard schema for now, yet this is only a temporary solution and we'd hope that this bug be fixed for external tasks so that we can safely upgrade our database schema from your scripts.

            Hi Hans,

            we can validate the max-length of strings in the Java code instead of only in the DB. We will discuss internally whether if and when we will do this.

            Concerning the

            When the first incident occurred, the engine completely lost its mind and all subsequent database operations failed."

            The "org.postgresql.util.PSQLException: ERROR: value too long for type character varying(255)" will only rollback the current transaction. It will not have any effects on subsequent transactions. What could have happened is that some other component like your custom infrastructure for performing external tasks retried submitting the same error message which is too long repeatedly. After the fix I proposed above you would then still see many exceptions but ProcessEngineExceptions instead of database errors.

            How did you fetch and complete the external tasks?

            Does that make sense to you.

            All the best,
            Daniel

            Daniel Meyer added a comment - Hi Hans, we can validate the max-length of strings in the Java code instead of only in the DB. We will discuss internally whether if and when we will do this. Concerning the When the first incident occurred, the engine completely lost its mind and all subsequent database operations failed." The "org.postgresql.util.PSQLException: ERROR: value too long for type character varying(255)" will only rollback the current transaction. It will not have any effects on subsequent transactions. What could have happened is that some other component like your custom infrastructure for performing external tasks retried submitting the same error message which is too long repeatedly. After the fix I proposed above you would then still see many exceptions but ProcessEngineExceptions instead of database errors. How did you fetch and complete the external tasks? Does that make sense to you. All the best, Daniel

            Hans Hübner added a comment - - edited

            Hi Daniel,

            it is unfortunate that we have not kept the original error message when the engine completely crashed. The message was rather scary and not the same as the ones that we've been seeing for the overly long column values. Sorry about this.

            With respect to external task fetching and completion, we use a sequence like this:

            GET /external-task to select a topic that we want to work on
            POST /external-task/fetchAndLock to actually fetch and lock the task that we've selected
            GET /process-instance/<process-instance-id>/variables to get the list of variables of the process (we don't know them in advance so we can't supply the list in the fetchAndLock call)
            POST /external-task/<external-task-id>/complete to complete the task

            The reason why we're using additional GETs is that we use the topic name as configuration parameter for the external task. Our external task handler executes shell commands, and the topic begins with "shell" and then has the command to execute appended. Likewise, as we do not know what variables the shell command wants to look at, we are not sending any variables in the fetchAndLock call, but rather fetch all process variables and make them available to the command in the environment.

            We'd prefer to be able to annotate the model with task parameters (i.e. have "shell" be the topic name, put the command to execute into a named external task parameter and maybe also be able to have other task parameters that'd allow us to choose one of several executors for the task), but that does not seem to be possible both from a modeler and from a model perspective at this point.

            Thanks,
            Hans

            Hans Hübner added a comment - - edited Hi Daniel, it is unfortunate that we have not kept the original error message when the engine completely crashed. The message was rather scary and not the same as the ones that we've been seeing for the overly long column values. Sorry about this. With respect to external task fetching and completion, we use a sequence like this: GET /external-task to select a topic that we want to work on POST /external-task/fetchAndLock to actually fetch and lock the task that we've selected GET /process-instance/<process-instance-id>/variables to get the list of variables of the process (we don't know them in advance so we can't supply the list in the fetchAndLock call) POST /external-task/<external-task-id>/complete to complete the task The reason why we're using additional GETs is that we use the topic name as configuration parameter for the external task. Our external task handler executes shell commands, and the topic begins with "shell" and then has the command to execute appended. Likewise, as we do not know what variables the shell command wants to look at, we are not sending any variables in the fetchAndLock call, but rather fetch all process variables and make them available to the command in the environment. We'd prefer to be able to annotate the model with task parameters (i.e. have "shell" be the topic name, put the command to execute into a named external task parameter and maybe also be able to have other task parameters that'd allow us to choose one of several executors for the task), but that does not seem to be possible both from a modeler and from a model perspective at this point. Thanks, Hans

            Hello,

            are there any plans to change the `varchar(4000)`s in the Postgres schema to `text`? We're currently doing that manually, and I see no reason why one would want error messages and other textual information to be cut at an arbitrary size.

            Thanks,
            Hans

            Hans Hübner added a comment - Hello, are there any plans to change the `varchar(4000)`s in the Postgres schema to `text`? We're currently doing that manually, and I see no reason why one would want error messages and other textual information to be cut at an arbitrary size. Thanks, Hans

            Matthijs added a comment -

            Good morning Hans,

            Thanks for your input on this issue.

            A short question: would you like us to move this issue to our support project?
            The difference between the Support project and the Camunda BPM project is that when raising issues in the Camunda BPM project, they are not subject to the agreed SLAs and they can be viewed by all users. In contrast, issues raised in the Support project can only be seen by your authorized support contacts and us. You can find more information in our documentation.
            If you would like us to move this issue, please let us know.

            Thank you and best regards,
            Mat

            Matthijs added a comment - Good morning Hans, Thanks for your input on this issue. A short question: would you like us to move this issue to our support project? The difference between the Support project and the Camunda BPM project is that when raising issues in the Camunda BPM project, they are not subject to the agreed SLAs and they can be viewed by all users. In contrast, issues raised in the Support project can only be seen by your authorized support contacts and us. You can find more information in our documentation . If you would like us to move this issue, please let us know. Thank you and best regards, Mat

            Hans Hübner added a comment -

            Hi Mat,

            I frankly don't care so much where this request is being tracked, and it is also not an urgent matter in that we're currently changing the database schema creation files manually before creating the Camunda BPM database. It is simply an annoyance and something that will be disturbing in production, when either the engine crashes because it wants to write an overlong message or when someone tries to diagnose a problem, finding that the error message has been cut after 4000 characters.

            Thanks,
            Hans

            Hans Hübner added a comment - Hi Mat, I frankly don't care so much where this request is being tracked, and it is also not an urgent matter in that we're currently changing the database schema creation files manually before creating the Camunda BPM database. It is simply an annoyance and something that will be disturbing in production, when either the engine crashes because it wants to write an overlong message or when someone tries to diagnose a problem, finding that the error message has been cut after 4000 characters. Thanks, Hans

            Matthijs added a comment -

            Good morning Hans,

            we have raised a separate issue in our Support project and have linked it to this issue: SUPPORT-2426. We will take a deeper look into this in the context of our product support and will respond in the Support issue.

            Thank you and best regards,
            Mat

            Matthijs added a comment - Good morning Hans, we have raised a separate issue in our Support project and have linked it to this issue: SUPPORT-2426. We will take a deeper look into this in the context of our product support and will respond in the Support issue. Thank you and best regards, Mat

            Askar added a comment -

            multi tenancy test is missing

            Askar added a comment - multi tenancy test is missing

            Askar added a comment -

            added missing test

            Askar added a comment - added missing test

            Hans Hübner added a comment -

            I don't get to see much of what you're doing, but I would like to point out that by "long error message" i mean something which can include a complete Java stack trace, and that easily amounts to a few kilobytes.

            Hans Hübner added a comment - I don't get to see much of what you're doing, but I would like to point out that by "long error message" i mean something which can include a complete Java stack trace, and that easily amounts to a few kilobytes.

              Unassigned Unassigned
              hans.huebner@lambdawerk.com Hans Hübner
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

                Created:
                Updated:
                Resolved: