Degradation to Bacs Simulator
Incident Report for Form3 Staging
Postmortem

From approximately 02:30 - 15:00 BST our Bacs simulator encountered an issue where the simulator was producing higher load than usual resulting in a performance degradation in submission handling.

This resulted in processing latency on all submissions until mitigations were introduced at 15:00 BST where pending submissions were processed through to a final state.

A full fix has been introduced to prevent this issue recurring and we continue to monitor the simulator behaviour for further enhancements.

Posted Oct 08, 2020 - 16:39 UTC

Resolved
Our Bacs simulator experienced an issue with message processing latency from approximately 02:30 - 15:00 BST which has now been resolved. Queued payments have been processed and new payments are processing without issues.

This was due to an issue with the service handling logic when restarting, the service has stabilised and no further impact seen.

Our engineers are reviewing the handling logic for enhancement to prevent any further issues.
Posted Oct 07, 2020 - 15:30 UTC
Investigating
We have identified an issue impacting the Bacs simulator causing latency in payment processing.

Our engineers are currently investigating and further details will be posted as the investigation progresses
Posted Oct 07, 2020 - 12:06 UTC
This incident affected: Simulators (Bacs - Direct Participant).