Uploaded image for project: 'camunda BPM'
  1. camunda BPM
  2. CAM-12071 Official support for CockroachDB
  3. CAM-12159

[BUG] Long-running CRDB transactions fail when commit deadline is exceeded

XMLWordPrintable

    • Icon: Sub-task Sub-task
    • Resolution: Fixed
    • Icon: L3 - Default L3 - Default
    • 7.14.0, 7.14.0-alpha2
    • None
    • engine
    • None

      Given:
      A long-running Process Engine operation. E.g.:

      • GetHistoricVariableUpdatesForOptimizeTest#testFetchLargeNumberOfObjectVariables
      • SetExternalTasksRetriesTest#shouldSetExternalTaskRetriesWithLargeList
      • Waiting for the JobExecutor to execute all the avalilable jobs (in JobExecutorMetricsTest)

      When:
      The operation is executed and a DB commit is attempted.

      Then:
      The commit fails due to an exceeded deadline:

      Caused by: org.postgresql.util.PSQLException: ERROR: restart transaction: TransactionRetryWithProtoRefreshError: TransactionRetryError: retry txn (RETRY_COMMIT_DEADLINE_EXCEEDED - txn timestamp pushed too much; deadline exceeded by 1m28.241354308s (1594798089.172664493,2 > 1594798000.931310185,0)): meta={id=0e1b867a key=/Table/82/1/"152095"/0 pri=0.04620799 epo=0 ts=1594798089.172664493,2 min=1594797858.463020094,0 seq=107295} lock=true stat=PENDING rts=1594798089.172664493,2 wto=false max=1594797858.463020094,0
      	at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2455)
      	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2155)
      	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:288)
      	at org.postgresql.jdbc.PgConnection.executeTransactionCommand(PgConnection.java:776)
      	at org.postgresql.jdbc.PgConnection.commit(PgConnection.java:797)
      	at sun.reflect.GeneratedMethodAccessor41.invoke(Unknown Source)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.apache.ibatis.datasource.pooled.PooledConnection.invoke(PooledConnection.java:245)
      	at com.sun.proxy.$Proxy16.commit(Unknown Source)
      	at org.apache.ibatis.transaction.jdbc.JdbcTransaction.commit(JdbcTransaction.java:71)
      	at org.apache.ibatis.executor.BaseExecutor.commit(BaseExecutor.java:244)
      	at org.apache.ibatis.executor.CachingExecutor.commit(CachingExecutor.java:119)
      	at org.apache.ibatis.session.defaults.DefaultSqlSession.commit(DefaultSqlSession.java:223)
      	... 32 more
      

      Expected:
      The operation is successfully committed.

      Note:
      This happens due to the sliding window for which DB writes are accepted in CRDB, i.e. there is a timestamp below which no transactions will be accepted (see CockroachDB paper section 3.5 paragraph 2)

        This is the controller panel for Smart Panels app

              nikola.koevski Nikola Koevski
              nikola.koevski Nikola Koevski
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

                Created:
                Updated:
                Resolved: