External Calls - Callbacks and Polling

This article looks at two different ways you can call an external service within your workflow processes and wait for the result of the call before proceeding. For more complex cases one of the structural elements, like a sub process or call activity, may be a better choice.

Messages as Callbacks

In this example an external service, imagined as an End Point, is called and one of three possible paths will be followed. The End Point performs some task, then uses the workflow worker's sendMessage function to report back to the process instance, activating one of the paths.

Parallel Gateway

A parallel gateway creates multiple paths for the execution, one path calls the End Point, the second activates the event gateway and waits.

Why not just place the activities in sequence, like this?

With this configuration the attempt to send a message would fail with the error: 

Could not send message 'EXT_Complete' to process 97620: Process instance'97620' does not have executions listening for message 'EXT_Complete'. 

That happens because the message events and their listeners don't exist yet. Those listeners will only come into existence when the execution reaches the gateway, which only happens after the previous task returns. 

This design would work if the call to the external service behaved asynchronously (that is, it returns a response to the workflow so the gateway becomes active, then sends a message at a later time). Note that marking the activity in the modeller as asynchronous is not enough. The activity asynchronous flag commits the transaction before the task is started, then starts it at a point in the future - the task still needs to completed in the background activity for the execution to move on to the next element.

End Point Task

This task does the job of calling the external service, assessing the result, then sending a message back to the workflow.

The workflow worker's sendMessage function needs to know the business key of the instance to send the message to, so it gets sent to the End Point as a parameter.

The End Point task is also marked as asynchronous. If it wasn't the process would fail. This is for two reasons.

The act of starting a process instance is transactional. Although the business key is generated immediately, the instance itself doesn't truly exist (commit to the database) until all execution paths reach a "wait" state. This means that the any attempt by the End Point to message the instance will fail because the instance doesn't exist until the End Point task itself completes.

The second reason is because although parallel gateways create parallel executions, these executions are not truly concurrent. By marking the End Point task as asynchronous, all of the "parallel" executions are able to reach a wait state, and the message listeners generated, before the End Point is called.

There's more information on asynchronous behaviour in the Workflow Transactions - Foreground and Background Jobs article.

Message Events

The three intermediate message events each listen for a different message reference. These are sent by the End Point.

var callWorkflow = client.invoke("workflow", "sendMessage", {
    "processInstanceBusinessKey": params.businessKey,
    "messageName": "EXT_Complete"
});

The event gateway activates one of the outgoing three paths based upon the message received.

End Events

Each path in the model has a simple end event rather than a terminate event. The End Point path ends so that the process can continue along other paths. The event gateway automatically terminates the two redundant paths once one has been selected.

Polling

Rather than waiting in an indefinite state for a call to another service to return, you could create a polling mechanism that will give up on a call after several attempts have been made.

In this example the first attempt to call the service will be made as part of the transaction that starts the workflow. The user who submitted the form that started the process will receive confirmation, and a business key, either when the timer is reached, or if the call is successful on its first attempt.

Script Task

The first script task creates a "counter" that can be updated during the lifetime of the process.

execution.setVariable("attemptNo", 1);

End Point Task and Results

In this example an End Point that is called will return a property called "updateReceived" alongside a successful response from the job it does. The script task also increments the attempt number.

var tempAttemptNo = parseInt(execution.getVariable("attemptNo"), 10) +1;
execution.setVariable("attemptNo", tempAttemptNo);

Conditional Flows

The workflow has two conditional sequence flows. The first will be followed if the number of attempts is greater than three.

${attemptNo>3}

The second will be followed if the End Point response is successful.

${APICALLRESULT.updateReceived==true}