Northflank - Issues with Google Cloud platform - Affecting multiple services – Incident details

All systems operational

Issues with Google Cloud platform - Affecting multiple services

Resolved
Partial outage 30 %
Started about 1 year agoLasted about 3 hours

Affected

Northflank Platform

Major outage from 6:00 PM to 7:21 PM, Partial outage from 6:00 PM to 8:40 PM, Operational from 7:21 PM to 8:44 PM

Addons

Partial outage from 6:00 PM to 8:40 PM, Operational from 8:40 PM to 8:44 PM

Builds

Major outage from 6:00 PM to 7:21 PM, Operational from 7:21 PM to 8:44 PM

Logs and Metrics

Partial outage from 6:00 PM to 7:21 PM, Operational from 7:21 PM to 8:44 PM

Updates
  • Resolved
    UTC
    Resolved

    GCP BYOC cluster processing is now operational.

  • Update
    UTC
    Update
    We are observing improved operations in the US - Central region. Addon creation and backup issues are now resolved. Ongoing Issues: * GCP BYOC cluster processing
  • Monitoring
    UTC
    Monitoring

    Google Cloud has acknowledged the incident. For detailed information, please visit: https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1SsW

    Most services have been restored to normal operation. However, we are still experiencing the following issues:

    • Addon creation in the US - Central region

    • Addon backups in the US - Central region

    • GCP BYOC cluster processing

    We will continue monitoring the situation and provide updates as they become available.

  • Identified
    UTC
    Identified
    We have noticed issues with Google Cloud APIs. This seems to be widespread across the Google Cloud platform. We are currently determining the impact of the outage, and we will provide updates when we know more. Currently, logs, metrics, builds, BYOC and addon backups may be affected.