Details
-
Bug Report
-
Resolution: Cannot Reproduce
-
L3 - Default
-
None
-
7.9.5
-
None
-
PRD
Description
Hi,
We have an app that is using camunda-external-task-client-java 1.0.1 to fetch External Tasks from our F5 load balanced Camunda (7.9.5-ee) instances hosted in WAS 9 servers (2 instances). Camunda is integrated with AD. When our app tries the fetch and lock, some of the 10 threads of the connection pool used by the client fail to fetch and lock with the following error, but seem to eventually be successful on subsequent tries:
org.camunda.bpm.client.impl.EngineClientException: TASK/CLIENT-02001 Request 'POST http://camunda:20000/engine-rest/external-task/fetchAndLock HTTP/1.1' returned error: status code '401' - message: Unauthorized at org.camunda.bpm.client.impl.EngineClientLogger.exceptionWhileReceivingResponse(EngineClientLogger.java:26) at org.camunda.bpm.client.impl.RequestExecutor.executeRequest(RequestExecutor.java:91) at org.camunda.bpm.client.impl.RequestExecutor.postRequest(RequestExecutor.java:68) at org.camunda.bpm.client.impl.EngineClient.fetchAndLock(EngineClient.java:68) at org.camunda.bpm.client.topic.impl.TopicSubscriptionManager.fetchAndLock(TopicSubscriptionManager.java:127) at org.camunda.bpm.client.topic.impl.TopicSubscriptionManager.acquire(TopicSubscriptionManager.java:95) at org.camunda.bpm.client.topic.impl.TopicSubscriptionManager.run(TopicSubscriptionManager.java:81) at java.lang.Thread.run(Thread.java:748) Caused by: org.apache.http.client.HttpResponseException: Unauthorized at org.apache.http.impl.client.AbstractResponseHandler.handleResponse(AbstractResponseHandler.java:70) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:223) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:165) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:140) at org.camunda.bpm.client.impl.RequestExecutor.executeRequest(RequestExecutor.java:82) ... 6 common frames omitted
This continually happens to some of the threads in the pool, and happens more frequently when there is higher Camunda activity during the day.
We have enabled debug logs and checked the HTTP requests from our app to Camunda and all the headers look fine, basic auth headers include the credentials.
We suspect it might be an issue between Camunda and AD, the Camunda debug logs show no errors, but we suspect it might be because the exception might be eaten silently from something like this: https://github.com/camunda/camunda-bpm-platform/blob/7.9.0-alpha5/engine-plugins/identity-ldap/src/main/java/org/camunda/bpm/identity/impl/ldap/LdapIdentityProviderSession.java#L310
Do you have any suggestions on how to narrow down our search or validate the connection between Camunda and AD? We’ve tried everything we can think of and are stuck, could use some deeper Camunda insight into this issue.
Thanks