Altapay - Increased rate of 3DS declines – Incident details

All systems operational

Increased rate of 3DS declines

Resolved
Operational
Started about 4 years agoLasted about 4 hours

Affected

Processing

Operational from 7:50 AM to 11:35 AM

Updates
  • Update
    UTC
    Update

    **Impact:** High - 4 One of 4 nodes partially dropped 3DSecure requests ‌ **Description & Scope of incident:** A change put in place to correct a DNS routing issue affected the connectivity between 1 of the 4 nodes and the connector service related to 3DSecure validation was returning HTTP 403 error. ‌ **Analysis of Incident:** The change put in place to correct the DNS routing issue was silently failing when the required ip configuration of apache2 was blocking it from the new source. In the morning the monitoring showed a high number of declined transactions around 3DSecure for multiple merchants. This was confirmed by Support and Merchants calling in. The changes implemented 27-05-2022 was reverted as part of the rollback procedure and the affected services were restarted.

  • Resolved
    UTC
    Resolved

    After extended monitoring we have registered no further issues with increased 3DS declines since the fix was implemented this morning.

  • Monitoring
    UTC
    Monitoring

    An issue with an increased rate of 3DS declines was identified and a fix has been implemented. Transactions affected by the issue can be identified on the payment gateway with the following error messages: 3DS Decline: null 3DS Decline: Could not format method callback url We are currently monitoring the situation to ensure the fix is working as intended, so these errors should no longer occur.