Uploaded image for project: 'camunda BPM'
  1. camunda BPM
  2. CAM-11897

DMN date conversion issue in multithreading mode

    • Icon: Bug Report Bug Report
    • Resolution: Fixed
    • Icon: L3 - Default L3 - Default
    • 7.14.0, 7.14.0-alpha1
    • None
    • dmn-engine
    • None

      There is a date conversion bug in DMN engine in multithreading environment

      PoC: https://bitbucket.org/capitallabs/dmn-issue/src/master/

      There are two tests: MultiThreading, that uses parallelStream and SingleThreading, that uses stream.

      SingleThreading is always passed
      MultiThreading is always failed

       

        This is the controller panel for Smart Panels app

            [CAM-11897] DMN date conversion issue in multithreading mode

            Hey Dmitry,

            After finishing the 7.13 release, I reviewed the PR you submitted again. It would be useful if you provided some more information on why this issue occurs here (explanation, a stack trace), and why your solution works.

            I also added some suggestions for code additions in the PR.

            Best,
            Nikola

            Nikola Koevski added a comment - Hey Dmitry , After finishing the 7.13 release, I reviewed the PR you submitted again. It would be useful if you provided some more information on why this issue occurs here (explanation, a stack trace), and why your solution works. I also added some suggestions for code additions in the PR. Best, Nikola

            Hi @Nikola Koevski,

            the issue is connected with a way how DMN is being prepared for execution and execution itself.
            there are objects initializations on prepare step, including org.camunda.bpm.dmn.engine.impl.type.DateDataTypeTransformer in case of Date data type existing in DMN. This transformer class creates java.text.SimpleDateFormat instance for further consumption. Keep in mind that java.text.SimpleDateFormat is not thread-safe according to SimpleDateFormat doc page

            Synchronization

            Date formats are not synchronized. It is recommended to create separate format instances for each thread. If multiple threads access a format concurrently, it must be synchronized externally.
             

            then we can execute the rule concurrently. in this case we are getting undefined behavior due to missing synchronization while date formatting.

            Possible solutions:
            1) add synchronization. bad idea due to performance
            2) avoid of using java.text.SimpleDateFormat as a part of org.camunda.bpm.dmn.engine.impl.type.DateDataTypeTransformer member.
            3) avoid of using java.text.SimpleDateFormat at all by using java 8 Date Time API (DateTimeFormatter)

            I was not sure about Camunda Team plans of java versions. I would prefer to use 3) but chose 2)

            If you guys can consider that it's OK to use java 8 Date Time API I would rewrite the PR for using it

            Dmitry Molotchko added a comment - Hi @Nikola Koevski, the issue is connected with a way how DMN is being prepared for execution and execution itself. there are objects initializations on prepare step, including org.camunda.bpm.dmn.engine.impl.type.DateDataTypeTransformer in case of Date data type existing in DMN. This transformer class creates java.text.SimpleDateFormat instance for further consumption. Keep in mind that java.text.SimpleDateFormat is not thread-safe according to  SimpleDateFormat doc page Synchronization Date formats are not synchronized. It is recommended to create separate format instances for each thread. If multiple threads access a format concurrently, it must be synchronized externally.   then we can execute the rule concurrently. in this case we are getting undefined behavior due to missing synchronization while date formatting. Possible solutions: 1) add synchronization. bad idea due to performance 2) avoid of using java.text.SimpleDateFormat as a part of org.camunda.bpm.dmn.engine.impl.type.DateDataTypeTransformer member. 3) avoid of using java.text.SimpleDateFormat at all by using java 8 Date Time API (DateTimeFormatter) I was not sure about Camunda Team plans of java versions. I would prefer to use 3) but chose 2) If you guys can consider that it's OK to use java 8 Date Time API I would rewrite the PR for using it

            Nikola Koevski added a comment - - edited

            Hey dmitry@capbpm.com,

            Sorry for the delayed response, and thank you for the extended explanation.

            Unfortunately, we're stuck with SimpleDateFormat. The reason is that, even though we dropped Java 7 support, we still guarantee that java.util.Date instances can be converted to the internal DateValue class of our DMN Engine. Furthermore, we still support our legacy (JUEL-based) FEEL engine that accepts Java 7 Dates.

            From what I could see, the Java 8 DateTimeFormatter doesn't accept Java 7 Date instances, so we would have to overhaul a lot of the Date-handling logic inside the DMN Engine to make sure that everything works correctly. We would also have to drop some of the above-mentioned guarantees. We discussed this internally, and at the moment, this is not a priority.

            So, I'm going with the implementation you originally submitted, merging the PR and closing this ticket.

            Best,
            Nikola

            Nikola Koevski added a comment - - edited Hey dmitry@capbpm.com , Sorry for the delayed response, and thank you for the extended explanation. Unfortunately, we're stuck with SimpleDateFormat . The reason is that, even though we dropped Java 7 support, we still guarantee that java.util.Date instances can be converted to the internal DateValue class of our DMN Engine. Furthermore, we still support our legacy (JUEL-based) FEEL engine that accepts Java 7 Dates. From what I could see, the Java 8 DateTimeFormatter doesn't accept Java 7 Date instances, so we would have to overhaul a lot of the Date-handling logic inside the DMN Engine to make sure that everything works correctly. We would also have to drop some of the above-mentioned guarantees. We discussed this internally, and at the moment, this is not a priority. So, I'm going with the implementation you originally submitted, merging the PR and closing this ticket. Best, Nikola

            Hey @Nikola Koevski,

            no problem at all.
            thanks for you explanation

            Dmitry Molotchko added a comment - Hey @Nikola Koevski, no problem at all. thanks for you explanation

              nikola.koevski Nikola Koevski
              Dmitry Dmitry Molotchko
              Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: