Payment Reconciliation is a background process that queries a payment provider to check the status of a payment. Once a response has been received from the provider a reconciliation end point can be called to provide further processing.
Why do we have Reconciliation?
All payments on the platform are handled by third party providers, which involves redirecting users away from your website to complete payment on the provider's payment portal. Once payment has been made (or cancelled or failed), the user is redirected back to your website and we are able to record the outcome.
In situations where the user doesn't return to your website, either because they gave up or their connection was interrupted, the payment reconciliation process allows us to query the payment provider to find out what happened.
Supported Providers
Currently only the Civica (Action, not eStore), WorldPay, Adelante Smart Pay (Multi-Item) and Capita (SCP Multi-Item and Paye.net) payment providers support payment reconciliation.
The Reconciliation Process
Payment reconciliation is enabled in the relevant payment field and handled by a workflow process and set of end points.
Workflow Process
The Payment Reconciliation workflow starts at the point a user is redirected to the payment provider. Once started the process waits for a notification that the user has returned to your website. If the user does not return from the provider within a set time (this varies per provider but varies from 30 minutes to an hour) the process begins to poll the provider directly after 1 minute, 5 minutes, 1 hour then 24 hours. Once 24 hours have passed the process will end with a failure event.
In both situations, once the payment provider has been successfully contacted and the status of the payment returned, a custom reconciliation end point can be called to handle the result.
Reconciliation End Point
The payment provider form fields let you set the name of an end point that should be called by the reconciliation workflow process when a response is returned from the payment portal. Your reconciliation end point needs to handle the response and perform any appropriate actions.
For example, in our Bookings product, the reconciliation end point passes the payment status to the booking workflow process, which will cancel the booking if payment wasn't successful.
Your end point should define a function with the following arguments:
| Argument | Description |
|---|---|
| pspQueryResult | This is a map containing successful and details properties with the same definition as the arguments of the response handler function (see Handling Responses) |
| formFields | A JSON object of which the properties are the form fields you selected in the payment field's Response Handler Form Fields property |
| userId | The name of the user that initiated the payment (this could be the anonymous user) |
| pspNotificationReceived | This boolean parameter is true when the user returned from the provider, and false if the user did not. Note that the payment may still be successful even when the user did not return |
History Records
The reconciliation process creates two history records.
Primary Histories
The main history written by the process is similar to the one below:
{
"labela": "Payment Reconciliation",
"labelb": "9407-0507-8389-5537",
"labelc": null,
"labeld": "f562f4ab-f247-4f4f-864d-f62de0e1b154",
"labele": "CASEMANAGEMENTREQUESTPAYMENTREADONLYV1EN",
"subject": {
"processDefinitionId": "paymentreconciliation:32:30290203",
"description": "Payment Reconciliation - This workflow supports payment fields to ensure that the reconciliation code of the client is executed even when the user does not return to iCM after a payment at the PSP",
"userId": "CMMANAGER1",
"processDefinitionKey": "paymentreconciliation"
},
"events": [{
"pos": [1, 5],
"event": {
"private": true,
"description": "Process Started",
"event": "STARTWORKFLOW",
"userRole": "user",
"userId": "CMMANAGER1"
}
}, {
"pos": [2, 5],
"event": {
"pspQueryParams": {
"webServiceUrl": "https://payment-webservice-url",
"generateCredentialsJsonParams": {
"subject": {
"identifier": 97,
"systemCode": "APN",
"type": "SecureBureauServiceSite"
},
"key": {
"id": 1
}
},
"apnReference": "f562f4ab-f247-4f4f-864d-f62de0e1b154",
"requestId": "1c17e09b-a73c-4a55-b055-7f3ddc8a2fa8"
},
"private": true,
"description": "Overview of data received",
"proxyUserId": "",
"formFields": {
"form_PAYREF": "1618-5860-6191-4044",
"taskBusinessKey": "1618-5860-6191-4044",
"paidByStaff": true
},
"event": "OVERVIEW",
"userRole": "user",
"userId": "CMMANAGER1",
"paymentFieldType": "CapitaPayeDotNet"
}
}, {
"pos": [3, 5],
"event": {
"private": true,
"description": "PSP notification was received",
"proxyUserId": "",
"event": "PSPNOTIFICATIONRECEIVED",
"userRole": "user",
"userId": "CMMANAGER1"
}
}, {
"pos": [4, 5],
"event": {
"errorOnQuery": "",
"pspSessionStatus": "COMPLETE",
"private": true,
"description": "PSP was queried for transaction result",
"proxyUserId": "",
"event": "PSPQUERIED",
"userRole": "user",
"userId": "CMMANAGER1"
}
}, {
"pos": [5, 5],
"event": {
"private": false,
"description": "Process Ended",
"proxyUserId": null,
"event": "PROCESSENDED",
"userRole": "workflowengine",
"userId": null
}
}]
}
Key elements to note are:
- labela - Always "Payment Reconciliation"
- labelb - The business key of the reconciliation process
- labelc - null
- labeld - The payment reference number
- labele - The form used to make payment and start the process
The following events may be present:
- OVERVIEW - the details present at the start of the process
- PSPNOTIFICATIONTIMEOUT - written when there's no response from the payment portal (likely because the user hasn't returned)
- PSPNOTIFICATIONRECEIVED - written when there is a response from the payment portal
- PSPQUERIED - written when the payment provider is queried and will include the current status of the payment (complete, in progress or error)
- PSPQUERYTOOMANYATTEMPS - written after 24 hours of trying to reconcile the payment
- NOUSERNORECONCILIATIONENDPOINT - written if the user did not return from the payment provider, and no reconciliation end point was defined, so no reconciliation was done
- RECONCILIATIONENDPOINTFAILED - written if an error was returned from your reconciliation end point
A scheduled task (automatically created by the product install process) calls the
- deletes any reconciliation histories that are "sealed". Sealed histories are histories that can no longer be written to, and represent completed (ie successful) reconciliation processes
- deletes any completed reconciliation process instances, as these have been successful
- deletes any reconciliation histories that are older than six months
Histories that are not sealed will remain in the platform (until six months old) as these represent in-flight payments, or payments that have not been reconciled and may need manual intervention.
Reporting Histories
The second history written by the reconciliation process follows our standard "reporting" history pattern. It contains similar information to the main history, but without any personal information (eg usernames and reference numbers).
{
"labela": "Payment Reconciliation",
"labelb": "1633-9540-9397-2529",
"labelc": "reporting",
"labeld": null,
"labele": null,
"created": 1678973175387,
"lastupdated": 1678973179110,
"sealed": false,
"subject": {
"description": "Payment Reconciliation Reporting History"
},
"events": [{
"pos": [1, 4],
"event": {
"private": true,
"description": "Start of the process",
"event": "STARTED"
},
"timestamp": 1678973175387
}, {
"pos": [2, 4],
"event": {
"private": true,
"description": "Timeout expired, no PSP notification was received",
"event": "PSPNOTIFICATIONTIMEOUT"
},
"timestamp": 1678973178887
}, {
"pos": [3, 4],
"event": {
"private": true,
"description": "PSP was queried for transaction result",
"event": "PSPQUERIED"
},
"timestamp": 1678973179077
}, {
"pos": [4, 4],
"event": {
"private": true,
"description": "The reconciliation endpoint call returned successfully.",
"event": "RECONCILIATIONENDPOINTFINISHED"
},
"timestamp": 1678973179110
}]
}
The possible end events are:
{
"event": "PSPQUERYTOOMANYATTEMPS",
"description": "The PSP Query failed the maximum number of times, giving up. An error will be thrown.",
"private": true
}
{
"event": "NOUSERNORECONCILIATIONENDPOINT",
"description": "The user did not return from the PSP, and no reconciliation endpoint was defined, so no reconciliation was done. An error will be thrown.",
"private": true
}
{
"event": "RECONCILIATIONFINISHED",
"description": "Returned from user successfully.",
"private": true
}
{
"event": "RECONCILIATIONENDPOINTFINISHED",
"description": "The reconciliation endpoint call returned successfully.",
"private": true
}
{
"event": "RECONCILIATIONENDPOINTFAILED",
"description": "The reconciliation endpoint call returned an error. An error will be thrown.",
"private": true
}
Setup
For detailed instructions, see the documentation for your payment provider field type (available on request, please contact us).
In summary, the following things need to be in place:
- Write a reconciliation end point that will be called by the process. The name of this end point should be added to your payment fields when they are used in any of your forms
- Each provider should have a configuration end point at
config.[environment].[providerName].getConfig which holds credentials needed to connect to the provider. There are examples inconfig.example.[providerName].getConfig - All website users (including the anonymous user) who need to make payment must be added to a user group called PAYMENT RECONCILIATION for the reconciliation workflow to start
- Check that a scheduled task exists to call the
paymentReconciliationCleanup end point each night
Share this page
