Live Status\Incident History
December 22, 2024
Site Unreachable
Outage
Opened: Dec 22, 2024, 6:33 AM UTC
Duration: 8m 38s
Affected services:Mastodon Instance Frontend
- OpenedDec 22, 2024, 6:33 AM UTC
Issue opened on monitor failure.
- ResolvedDec 22, 2024, 6:42 AM UTC
Issue resolved on monitor recovery.
Site Unreachable
Outage
Opened: Dec 22, 2024, 4:16 AM UTC
Duration: 14m 20s
Affected services:Mastodon Instance Frontend
- OpenedDec 22, 2024, 4:16 AM UTC
Issue opened on monitor failure.
- ResolvedDec 22, 2024, 4:30 AM UTC
Issue resolved on monitor recovery.
Site Unreachable
Outage
Opened: Dec 22, 2024, 2:01 AM UTC
Duration: 9m 46s
Affected services:Static Content Server and Cache
- OpenedDec 22, 2024, 2:01 AM UTC
Issue opened on monitor failure.
- ResolvedDec 22, 2024, 2:11 AM UTC
Issue resolved on monitor recovery.
September 5, 2024
Site Unreachable
Outage
Opened: Sep 5, 2024, 1:41 AM UTC
Duration: 9m 44s
Affected services:Mastodon Instance Frontend
- OpenedSep 5, 2024, 1:41 AM UTC
Issue opened on monitor failure.
- ResolvedSep 5, 2024, 1:50 AM UTC
Issue resolved on monitor recovery.
August 17, 2024
Emergency Database Maintenance
Degraded Performance
Opened: Aug 17, 2024, 6:18 PM UTC
Duration: 4m 21s
Affected services:Database System
- MonitoringAug 17, 2024, 6:18 PM UTC
Database maintenance is being performed on the instance to restore write performance due to conditions preventing routine, scheduled maintenance operations from running successfully. Performance of the site may be severely degraded as a result.
- ResolvedAug 17, 2024, 6:22 PM UTC
Maintenance operation has completed, write performance restored.
May 3, 2024
Site Unreachable
Outage
Opened: May 3, 2024, 9:28 AM UTC
Duration: 5m 10s
Affected services:Static Content Server and Cache
- OpenedMay 3, 2024, 9:28 AM UTC
Issue opened on monitor failure.
- ResolvedMay 3, 2024, 9:33 AM UTC
Issue resolved on monitor recovery.
April 12, 2024
Healthcheck Assertion Failed
Outage
Opened: Apr 12, 2024, 11:05 PM UTC
Duration: 4m 34s
Affected services:Static Content Server and Cache
- OpenedApr 12, 2024, 11:05 PM UTC
Issue opened on monitor failure.
- IdentifiedApr 12, 2024, 11:07 PM UTC
False alarm due to monitoring configuration
- ResolvedApr 12, 2024, 11:10 PM UTC
Monitoring hook has been updated, no actual downtime on static content server.
March 14, 2024
Site Unreachable
Outage
Opened: Mar 14, 2024, 6:12 PM UTC
Duration: 9m 21s
Affected services:Static Content Server and Cache
- OpenedMar 14, 2024, 6:12 PM UTC
Issue opened on monitor failure.
- ResolvedMar 14, 2024, 6:21 PM UTC
Issue resolved on monitor recovery.
October 7, 2023
Site Unreachable
Outage
Opened: Oct 7, 2023, 6:44 PM UTC
Duration: 9m 36s
Affected services:Mastodon Instance Frontend
- OpenedOct 7, 2023, 6:44 PM UTC
Issue opened on monitor failure.
- ResolvedOct 7, 2023, 6:53 PM UTC
Issue resolved on monitor recovery.
Site Unreachable
Outage
Opened: Oct 7, 2023, 6:41 PM UTC
Duration: 14m 29s
Affected services:Static Content Server and Cache
- OpenedOct 7, 2023, 6:41 PM UTC
Issue opened on monitor failure.
- IdentifiedOct 7, 2023, 6:48 PM UTC
Storage migration in progress, performing validation checks before toggling over
- ResolvedOct 7, 2023, 6:56 PM UTC
Issue resolved on monitor recovery.
October 5, 2023
Site Unreachable
Outage
Opened: Oct 5, 2023, 7:42 AM UTC
Duration: 9m 28s
Affected services:Static Content Server and Cache
- OpenedOct 5, 2023, 7:42 AM UTC
Issue opened on monitor failure.
- ResolvedOct 5, 2023, 7:51 AM UTC
Issue resolved on monitor recovery.
September 24, 2023
Database Upgrade in Progress
Maintenance
Posted: Sep 24, 2023, 12:44 PM UTC
Affected services:Database System
- OpenedSep 24, 2023, 12:44 PM UTC
The system will be unavailable while the database is being upgraded
- IdentifiedSep 24, 2023, 12:45 PM UTC
Cluster upgrade in progress
- MonitoringSep 24, 2023, 12:51 PM UTC
Upgrade complete, validating state
- ResolvedSep 24, 2023, 1:30 PM UTC
Upgrade performs well, maintenance over
July 2, 2023
Heavy queue load and increased processing times
Degraded Performance
Opened: Jul 2, 2023, 6:11 PM UTC
Duration: 2m 12s
Affected services:Database SystemActivityPub Workers
- MonitoringJul 2, 2023, 6:09 PM UTC
We somehow touch a large chunk of the twitter migration, some tuning has been done to accomodate. We are currently monitoring the systems for bottlenecks.
- ResolvedJul 2, 2023, 6:13 PM UTC
Database has been cleaned up, a problematic fakerelay endpoint was putting undue pressure on the system. Situation normal.
June 1, 2023
Site Unreachable
Outage
Opened: Jun 1, 2023, 11:14 PM UTC
Duration: 14m 18s
Affected services:Mastodon Instance Frontend
- OpenedJun 1, 2023, 11:14 PM UTC
Issue opened on monitor failure.
- ResolvedJun 1, 2023, 11:28 PM UTC
Issue resolved on monitor recovery.
Site Unreachable
Outage
Opened: Jun 1, 2023, 11:14 PM UTC
Duration: 14m 48s
Affected services:Static Content Server and Cache
- OpenedJun 1, 2023, 11:14 PM UTC
Issue opened on monitor failure.
- ResolvedJun 1, 2023, 11:29 PM UTC
Issue resolved on monitor recovery.