Northflank - Partial Control Plane Degredation – Incident details

All systems operational

Partial Control Plane Degredation

Resolved
Operational
Started about 17 hours agoLasted 6 hours 50 minutes

Affected

Northflank App

Operational from 6:44 AM to 9:53 AM, Major outage from 9:53 AM to 1:34 PM

Northflank Platform

Major outage from 6:44 AM to 1:34 PM

Logs and Metrics

Major outage from 6:44 AM to 1:34 PM

Updates
  • Resolved
    UTC
    Resolved

    This incident has been resolved.

  • Monitoring
    UTC
    Monitoring

    Services are operating normally, and we are monitoring the situation.

  • Investigating
    UTC
    Investigating

    The incident has expanded to our core control plane services, affecting login and control plane components. We are working to mitigate the impact. Existing user workloads deployed are unaffected.

  • Monitoring
    UTC
    Monitoring

    We have moved the affected parts of the logging and metric components to a different zone. We will continue to monitor the situation.

    A public Google status page is now available: https://status.cloud.google.com/incidents/Xs2XTHzAVC9YnafnPyzj

  • Identified
    UTC
    Identified

    Availability zone b in GCP europe-west2 is experiencing issues with VM to VM communication impacting parts of the platform. We are working on mitigating the impact. The majority of users and system components are unaffeced.