Altapay - Search Service degraded performance – Incident details

**Impact:** Low - 2 ‌ **Description & Scope of incident:** Some transactions cannot be found while searching. The search result returns empty search result leading one to think the transaction were not processed. Several merchants experienced the issue however not all were affected. While the search service was affected by this incident transactions were processed without any interruption. ‌ **Analysis of Incident:** After a deployment of the search service around 15:30 23/6, the historic search data was overwritten for merchants. This affected customers who had done payments in the timeframe of 15:30 – 16:00 on 23/6-2022 only leaving new payments searchable after 15:30 23/6. The issue was caused by incorrect user permissions, which also caused issues with the service to start. ‌ **Remedial Action:** Restore backup of the search service server. Changed endpoints to point to new server and then backfill missing data.