Degradation to Bacs Simulator

Incident Report for Form3 Staging

Postmortem

From approximately 02:30 - 15:00 BST our Bacs simulator encountered an issue where the simulator was producing higher load than usual resulting in a performance degradation in submission handling.

This resulted in processing latency on all submissions until mitigations were introduced at 15:00 BST where pending submissions were processed through to a final state.

A full fix has been introduced to prevent this issue recurring and we continue to monitor the simulator behaviour for further enhancements.

Posted Oct 08, 2020 - 16:39 UTC

Resolved

Our Bacs simulator experienced an issue with message processing latency from approximately 02:30 - 15:00 BST which has now been resolved. Queued payments have been processed and new payments are processing without issues.

This was due to an issue with the service handling logic when restarting, the service has stabilised and no further impact seen.

Our engineers are reviewing the handling logic for enhancement to prevent any further issues.

Posted Oct 07, 2020 - 15:30 UTC

Investigating

We have identified an issue impacting the Bacs simulator causing latency in payment processing.

Our engineers are currently investigating and further details will be posted as the investigation progresses

Posted Oct 07, 2020 - 12:06 UTC

This incident affected: Archived Components (Bacs - Direct Participant).