Altapay - Connectivity issue – Incident details

Connectivity issue

Resolved
Major outage
Started over 2 years agoLasted about 1 hour

Affected

Processing

Operational from 2:11 PM to 2:11 PM, Major outage from 2:11 PM to 3:35 PM

Networking

Operational from 2:11 PM to 2:11 PM, Major outage from 2:11 PM to 3:35 PM

Updates
  • Update
    Update

    **Impact:** 5 - Very high ‌ **Description & Scope of incident** All outgoing traffic was not passing through after a network/route preventing transactions from being processed in the timeframe 14:40 - 15:20 ‌ **Analysis of Incident** A planned network change was implemented in the network infrastructure. This caused all outgoing traffic to the internet and 3rd party connections to fail while all incoming traffic was working as expected. Our monitoring system was not set up to trigger an alarm on this exact scenario which caused a delay in detecting the issue. ‌ **Remedial Action** As soon as the root cause was identified the network change was reverted. Mitigation plan: * Update the monitoring to include additional checks to catch similar issues. * Improve and review Internal change management process with focus on implementing measures which can avoid similar issues being introduced.

  • Resolved
    Resolved

    This incident has been resolved

  • Monitoring
    Monitoring

    A fix has been implemented and we are monitoring the situation

  • Identified
    Identified

    The issue has been identified and we are working on implementing a fix.

  • Investigating
    Investigating

    We are currently investigating a connectivity issue where payments are rejected