Altapay - Notice history

**Impact:** Low - 2 ‌ **Description & Scope of incident:** Some transactions cannot be found while searching. The search result returns empty search result leading one to think the transaction were not processed. Several merchants experienced the issue however not all were affected. While the search service was affected by this incident transactions were processed without any interruption. ‌ **Analysis of Incident:** After a deployment of the search service around 15:30 23/6, the historic search data was overwritten for merchants. This affected customers who had done payments in the timeframe of 15:30 – 16:00 on 23/6-2022 only leaving new payments searchable after 15:30 23/6. The issue was caused by incorrect user permissions, which also caused issues with the service to start. ‌ **Remedial Action:** Restore backup of the search service server. Changed endpoints to point to new server and then backfill missing data.

Swish reports they have resolved the issue.

Cloudflare reports all systems operational and this aligns with what we are seeing in our monitoring.

The ACS in question was NemID which had technical issues for a couple of hours. The issue is reported as resolved.

The issue has stopped and our partner reports the issue seems to have originated from a 3rd party access control server. We will continue to closely monitor that everything is operating smoothly.

We are continuing to work on a fix for this issue.

Our partner who handles the service which initializes 3DS requests confirms they are working on a solution on their end. We are awaiting further feedback from them and will provide an update accordingly as soon as possible.

The issue has been identified and we are working with Gothia on a solution. We are working with affected merchants individually hence we are closing this incident.

**Impact:** High - 4 One of 4 nodes partially dropped 3DSecure requests ‌ **Description & Scope of incident:** A change put in place to correct a DNS routing issue affected the connectivity between 1 of the 4 nodes and the connector service related to 3DSecure validation was returning HTTP 403 error. ‌ **Analysis of Incident:** The change put in place to correct the DNS routing issue was silently failing when the required ip configuration of apache2 was blocking it from the new source. In the morning the monitoring showed a high number of declined transactions around 3DSecure for multiple merchants. This was confirmed by Support and Merchants calling in. The changes implemented 27-05-2022 was reverted as part of the rollback procedure and the affected services were restarted.

After extended monitoring we have registered no further issues with increased 3DS declines since the fix was implemented this morning.

After further investigation an issue processing Mastercard between 09:49 - 09:56 has been identified and solved. We are awaiting further details from the acquirer who is investigating the possible cause with Mastercard.