[2025-09-21] Network connectivity loss in the EU-EAST datacenter

On September 21, 2025, our EU-EAST regional zone experienced a major network outage in our colocation data center.

At 10:18 PM PST, a failure in core network hardware caused services in the EU-EAST region to become unavailable. Our automatic network failover mechanisms activated immediately and restored access to most services within 5 minutes.

However, some applications with custom firewall or routing rules did not fully recover during the failover. This resulted in partial unavailability for certain customers depending on their network provider.

Our colocation partner quickly dispatched engineers on site. They replaced the faulty equipment and fully restored network connectivity by 03:25 AM PST, Sep 22, 2025.

What we are doing next

  • Working with our colocation provider to review and strengthen high-availability configurations.

  • Auditing custom firewall and routing rules to ensure seamless failover across all services.

  • Adding additional monitoring to catch misconfigurations earlier.

We sincerely apologize for the disruption this caused. Ensuring reliability is our top priority, and we are taking steps to prevent similar incidents in the future.

1 Like