-
Bug Report
-
Resolution: Fixed
-
L3 - Default
-
None
-
7.11.3
Given:
- A cluster exists with multiple nodes pointing to the same database
- The cluster was already up and running so that the History Cleanup jobs are stored in the jobs table
- History Cleanup job is configured on all nodes
- All nodes are down
- The previous run of History Cleanup job failed with an exception so that the job references the exception stacktrace (stored as bytearray)
When:
- The cluster is started so that all nodes are started at the very same time
Then:
- All nodes will reconfigure the History Cleanup jobs (i.e., release any locks, ...)
- Therefore the nodes will also delete the referenced stacktrace in the bytearray table (see [1])
- Two nodes (node1 and node2) are in progress of reconfiguring the History Cleanup jobs, so that the following happens:
- node1 and node2 executes the HistoryCleanupCmd [2]
- both nodes decide to reconfigure the History Cleanup jobs
- so that both nodes fetched already the existing History Cleanup jobs
- node1 already executed the code block in [1]
- node2 hasn't executed the code block in [1]
- node1 completes the execution of HistoryCleanupCmd and commits the transaction
- as a result, the stacktrace stored in the bytearray table gets deleted
- node2 executes the code block in [1] and fails with a NullPointerException because the respective bytearray does not exist anymore
02-Dec-2019 16:35:29.452 SEVERE [main] org.camunda.commons.logging.BaseLogger.logError ENGINE-16004 Exception while closing command context: null java.lang.NullPointerException at org.camunda.bpm.engine.impl.db.entitymanager.cache.DbEntityCache.getCachedEntity(DbEntityCache.java:145) at org.camunda.bpm.engine.impl.db.entitymanager.cache.DbEntityCache.setDeleted(DbEntityCache.java:362) at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.delete(DbEntityManager.java:513) at org.camunda.bpm.engine.impl.persistence.entity.EverLivingJobEntity.init(EverLivingJobEntity.java:79) at org.camunda.bpm.engine.impl.persistence.entity.JobManager.reschedule(JobManager.java:98) at org.camunda.bpm.engine.impl.cmd.HistoryCleanupCmd.reconfigureJobs(HistoryCleanupCmd.java:157) at org.camunda.bpm.engine.impl.cmd.HistoryCleanupCmd.execute(HistoryCleanupCmd.java:82) at org.camunda.bpm.engine.impl.cmd.HistoryCleanupCmd.execute(HistoryCleanupCmd.java:45) at org.camunda.bpm.engine.impl.interceptor.CommandExecutorImpl.execute(CommandExecutorImpl.java:28) at org.camunda.bpm.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:110) at org.camunda.bpm.engine.impl.interceptor.ProcessApplicationContextInterceptor.execute(ProcessApplicationContextInterceptor.java:70) at org.camunda.bpm.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:33) at org.camunda.bpm.engine.impl.HistoryServiceImpl.cleanUpHistoryAsync(HistoryServiceImpl.java:170) at org.camunda.bpm.engine.impl.HistoryServiceImpl.cleanUpHistoryAsync(HistoryServiceImpl.java:166) at org.camunda.bpm.engine.impl.BootstrapEngineCommand.createHistoryCleanupJob(BootstrapEngineCommand.java:62) at org.camunda.bpm.engine.impl.BootstrapEngineCommand.execute(BootstrapEngineCommand.java:42) at org.camunda.bpm.engine.impl.BootstrapEngineCommand.execute(BootstrapEngineCommand.java:32) at org.camunda.bpm.engine.impl.interceptor.CommandExecutorImpl.execute(CommandExecutorImpl.java:28) at org.camunda.bpm.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:110) at org.camunda.bpm.engine.impl.interceptor.ProcessApplicationContextInterceptor.execute(ProcessApplicationContextInterceptor.java:70) at org.camunda.bpm.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:33) at org.camunda.bpm.engine.impl.ProcessEngineImpl.executeSchemaOperations(ProcessEngineImpl.java:124) at org.camunda.bpm.engine.impl.ProcessEngineImpl.<init>(ProcessEngineImpl.java:93) at org.camunda.bpm.engine.impl.cfg.ProcessEngineConfigurationImpl.buildProcessEngine(ProcessEngineConfigurationImpl.java:798) at org.camunda.bpm.container.impl.jmx.services.JmxManagedProcessEngineController.start(JmxManagedProcessEngineController.java:38) at org.camunda.bpm.container.impl.jmx.MBeanServiceContainer.startService(MBeanServiceContainer.java:77) at org.camunda.bpm.container.impl.jmx.MBeanServiceContainer.startService(MBeanServiceContainer.java:63) at org.camunda.bpm.container.impl.deployment.StartProcessEngineStep.performOperationStep(StartProcessEngineStep.java:116) at org.camunda.bpm.container.impl.spi.DeploymentOperation.execute(DeploymentOperation.java:120) at org.camunda.bpm.container.impl.jmx.MBeanServiceContainer.executeDeploymentOperation(MBeanServiceContainer.java:160) at org.camunda.bpm.container.impl.spi.DeploymentOperation$DeploymentOperationBuilder.execute(DeploymentOperation.java:216) at org.camunda.bpm.container.impl.tomcat.TomcatBpmPlatformBootstrap.deployBpmPlatform(TomcatBpmPlatformBootstrap.java:83) at org.camunda.bpm.container.impl.tomcat.TomcatBpmPlatformBootstrap.lifecycleEvent(TomcatBpmPlatformBootstrap.java:61) at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:123) at org.apache.catalina.util.LifecycleBase.setStateInternal(LifecycleBase.java:423) at org.apache.catalina.util.LifecycleBase.setState(LifecycleBase.java:366) at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:925) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) at org.apache.catalina.startup.Catalina.start(Catalina.java:633) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:567) at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:350) at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:492)
[1]: https://github.com/camunda/camunda-bpm-platform/blob/68dfbd4ecab899ac0061d3ece1dfc0b38e54c698/engine/src/main/java/org/camunda/bpm/engine/impl/persistence/entity/EverLivingJobEntity.java#L77-L80
[2]: https://github.com/camunda/camunda-bpm-platform/blob/68dfbd4ecab899ac0061d3ece1dfc0b38e54c698/engine/src/main/java/org/camunda/bpm/engine/impl/cmd/HistoryCleanupCmd.java
This is the controller panel for Smart Panels app
NullPointerException when two nodes tries to reconfigure history clean up
-
-
Resolution: Fixed
-
L3 - Default
-
None
-
7.11.3
Given:
- A cluster exists with multiple nodes pointing to the same database
- The cluster was already up and running so that the History Cleanup jobs are stored in the jobs table
- History Cleanup job is configured on all nodes
- All nodes are down
- The previous run of History Cleanup job failed with an exception so that the job references the exception stacktrace (stored as bytearray)
When:
- The cluster is started so that all nodes are started at the very same time
Then:
- All nodes will reconfigure the History Cleanup jobs (i.e., release any locks, ...)
- Therefore the nodes will also delete the referenced stacktrace in the bytearray table (see [1])
- Two nodes (node1 and node2) are in progress of reconfiguring the History Cleanup jobs, so that the following happens:
- node1 and node2 executes the HistoryCleanupCmd [2]
- both nodes decide to reconfigure the History Cleanup jobs
- so that both nodes fetched already the existing History Cleanup jobs
- node1 already executed the code block in [1]
- node2 hasn't executed the code block in [1]
- node1 completes the execution of HistoryCleanupCmd and commits the transaction
- as a result, the stacktrace stored in the bytearray table gets deleted
- node2 executes the code block in [1] and fails with a NullPointerException because the respective bytearray does not exist anymore
02-Dec-2019 16:35:29.452 SEVERE [main] org.camunda.commons.logging.BaseLogger.logError ENGINE-16004 Exception while closing command context: null java.lang.NullPointerException at org.camunda.bpm.engine.impl.db.entitymanager.cache.DbEntityCache.getCachedEntity(DbEntityCache.java:145) at org.camunda.bpm.engine.impl.db.entitymanager.cache.DbEntityCache.setDeleted(DbEntityCache.java:362) at org.camunda.bpm.engine.impl.db.entitymanager.DbEntityManager.delete(DbEntityManager.java:513) at org.camunda.bpm.engine.impl.persistence.entity.EverLivingJobEntity.init(EverLivingJobEntity.java:79) at org.camunda.bpm.engine.impl.persistence.entity.JobManager.reschedule(JobManager.java:98) at org.camunda.bpm.engine.impl.cmd.HistoryCleanupCmd.reconfigureJobs(HistoryCleanupCmd.java:157) at org.camunda.bpm.engine.impl.cmd.HistoryCleanupCmd.execute(HistoryCleanupCmd.java:82) at org.camunda.bpm.engine.impl.cmd.HistoryCleanupCmd.execute(HistoryCleanupCmd.java:45) at org.camunda.bpm.engine.impl.interceptor.CommandExecutorImpl.execute(CommandExecutorImpl.java:28) at org.camunda.bpm.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:110) at org.camunda.bpm.engine.impl.interceptor.ProcessApplicationContextInterceptor.execute(ProcessApplicationContextInterceptor.java:70) at org.camunda.bpm.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:33) at org.camunda.bpm.engine.impl.HistoryServiceImpl.cleanUpHistoryAsync(HistoryServiceImpl.java:170) at org.camunda.bpm.engine.impl.HistoryServiceImpl.cleanUpHistoryAsync(HistoryServiceImpl.java:166) at org.camunda.bpm.engine.impl.BootstrapEngineCommand.createHistoryCleanupJob(BootstrapEngineCommand.java:62) at org.camunda.bpm.engine.impl.BootstrapEngineCommand.execute(BootstrapEngineCommand.java:42) at org.camunda.bpm.engine.impl.BootstrapEngineCommand.execute(BootstrapEngineCommand.java:32) at org.camunda.bpm.engine.impl.interceptor.CommandExecutorImpl.execute(CommandExecutorImpl.java:28) at org.camunda.bpm.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:110) at org.camunda.bpm.engine.impl.interceptor.ProcessApplicationContextInterceptor.execute(ProcessApplicationContextInterceptor.java:70) at org.camunda.bpm.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:33) at org.camunda.bpm.engine.impl.ProcessEngineImpl.executeSchemaOperations(ProcessEngineImpl.java:124) at org.camunda.bpm.engine.impl.ProcessEngineImpl.<init>(ProcessEngineImpl.java:93) at org.camunda.bpm.engine.impl.cfg.ProcessEngineConfigurationImpl.buildProcessEngine(ProcessEngineConfigurationImpl.java:798) at org.camunda.bpm.container.impl.jmx.services.JmxManagedProcessEngineController.start(JmxManagedProcessEngineController.java:38) at org.camunda.bpm.container.impl.jmx.MBeanServiceContainer.startService(MBeanServiceContainer.java:77) at org.camunda.bpm.container.impl.jmx.MBeanServiceContainer.startService(MBeanServiceContainer.java:63) at org.camunda.bpm.container.impl.deployment.StartProcessEngineStep.performOperationStep(StartProcessEngineStep.java:116) at org.camunda.bpm.container.impl.spi.DeploymentOperation.execute(DeploymentOperation.java:120) at org.camunda.bpm.container.impl.jmx.MBeanServiceContainer.executeDeploymentOperation(MBeanServiceContainer.java:160) at org.camunda.bpm.container.impl.spi.DeploymentOperation$DeploymentOperationBuilder.execute(DeploymentOperation.java:216) at org.camunda.bpm.container.impl.tomcat.TomcatBpmPlatformBootstrap.deployBpmPlatform(TomcatBpmPlatformBootstrap.java:83) at org.camunda.bpm.container.impl.tomcat.TomcatBpmPlatformBootstrap.lifecycleEvent(TomcatBpmPlatformBootstrap.java:61) at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:123) at org.apache.catalina.util.LifecycleBase.setStateInternal(LifecycleBase.java:423) at org.apache.catalina.util.LifecycleBase.setState(LifecycleBase.java:366) at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:925) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183) at org.apache.catalina.startup.Catalina.start(Catalina.java:633) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:567) at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:350) at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:492)
[1]: https://github.com/camunda/camunda-bpm-platform/blob/68dfbd4ecab899ac0061d3ece1dfc0b38e54c698/engine/src/main/java/org/camunda/bpm/engine/impl/persistence/entity/EverLivingJobEntity.java#L77-L80
[2]: https://github.com/camunda/camunda-bpm-platform/blob/68dfbd4ecab899ac0061d3ece1dfc0b38e54c698/engine/src/main/java/org/camunda/bpm/engine/impl/cmd/HistoryCleanupCmd.java
This is the controller panel for Smart Panels app
- is related to
-
CAM-12104 Backport CAM-11147 to 7.11
-
- Closed
-
-
Nikola Koevski
-
- Votes:
-
0 Vote for this issue
- Watchers:
-
3 Start watching this issue
- Created:
- Updated:
- Resolved: