Northflank - Investigating node stability issues in the US - East region – Incident details

All systems operational

Investigating node stability issues in the US - East region

Resolved
Partial outage 15 %
Started 3 months agoLasted 1 hour 22 minutes

Affected

Northflank Platform

Partial outage from 12:17 PM to 1:39 PM

Services

Partial outage from 12:17 PM to 1:39 PM

Addons

Partial outage from 12:17 PM to 1:39 PM

Jobs

Partial outage from 12:17 PM to 1:39 PM

Updates
  • Resolved
    UTC
    Resolved

    We have recovered affected workloads in the region and there are no more on-going node failures. The root cause was an outage on the Google Cloud Platform in us-east4 which hosts Northflank's US - East region.

  • Monitoring
    UTC
    Monitoring

    We are seeing recovery across compute and storage. We are working to get the remaining workloads back online.

  • Investigating
    UTC
    Investigating

    We have noticed increased failure rates of nodes in the US - East region and are investigating the issue.