Affected
Operational from 8:29 PM to 2:11 PM, Degraded performance from 2:11 PM to 12:30 AM
Operational from 8:29 PM to 2:11 PM, Degraded performance from 2:11 PM to 12:30 AM
- ResolvedResolved
As of this time, all locations are stable and operating as expected. Our team will be deploying on-prem protection in Miami on a future date, which will be scheduled in advance.
- UpdateUpdate
We are currently experiencing issues affecting services routed through our Salt Lake City location. Our upstream provider utilizes mitigation capacity from Path Network. Path is presently encountering filtering and capacity limitations during volumetric events. This can cause packet loss, intermittent connection drops, increased latency, and short periods of instability for traffic passing through this region.
These issues are upstream of our network. Our provider has already begun the process of replacing Path with a higher capacity mitigation solution. This replacement aims to restore full filtering capability, increase throughput headroom, and stabilize all attack handling in Salt Lake City.
During this transition period, mitigation behavior may vary depending on the size and nature of incoming attacks. Our team is in active communication with the upstream provider to minimize impact while the new system is being integrated. As each stage of the upstream replacement is completed, customers should see noticeable improvements in stability and overall performance.
We will continue to monitor the situation closely until the new mitigation platform is fully active. Further updates will be posted as progress continues.
- IdentifiedIdentified
We are adding Salt Lake City to the scope of this incident, as there have been intermittent periods of packet loss, due to large DDoS attacks resulting in degradation of network quality of our DDoS mitigiation upstream. We are continuing to work on solutions for this.
- MonitoringMonitoring
We have rolled back changes made to our network that were affecting authentication to Minecraft servers and other services.
- IdentifiedIdentified
Our team has identified instability in our network, specifically with connection failures to services like Mojang's Authentication servers, discord APIs, etc. Our upstream is in the process of working on what is causing this. We will provide an update soon.
- UpdateUpdate
Our team has onboarded a new protection provider in Miami and is monitoring the results. There may be brief connectivity blips while the process occurs.
- MonitoringMonitoring
Services in Miami and Paris are currently online and operating normally. Earlier, we experienced a sudden connectivity loss across both POPs. Our team promptly switched to our backup transit provider, restoring service.
Additionally, our DDoS mitigation provider in Ashburn reported a brief period of elevated latency and packet loss, which has since stabilized.
We’re continuing to investigate the root cause and are in contact with our upstream NOCs for further information. We’ll provide updates as they become available.
- InvestigatingInvestigating
Our monitoring observed and alerted on packet loss and large traffic drops in multiple locations.
![[object Object]](/_next/image?url=https%3A%2F%2Finstatus.com%2Fuser-content%2Fv1725403191%2Fwgh4vd8qifckiudin9lh.png&w=3840&q=75)