Elevated Login and Authorization Errors

Incident Report for RebelGive

Postmortem

Amazon AWS had a major outage across multiple service categories. RebelGive uses some of these services to provide sign in, and other hosting solutions. As a result of AWS outages, some of RebelGive’s functionality was affected and people could not sign into GivingFlows or the Control Center for a few hours. We monitored the situation closely and took down GivingFlows very briefly to ensure donors did not run into unexpected behavior. Once AWS identified the root issue and began fixing, things started getting better and we re-enabled GivingFlows and were able to successfully verify functionality was restored.

Posted Nov 25, 2020 - 23:29 EST

Resolved

This incident has been resolved.
Posted Nov 25, 2020 - 23:25 EST

Update

Things look like they are working nicely now. Some users might encounter lingering issues, but from everything we can see, the logjams have cleared up and things are flowing along now. Continuing to monitor just in case as things with AWS get fully resolved and back to normal across all their systems.
Posted Nov 25, 2020 - 16:33 EST

Update

Things are starting to come back online. We're now just seeing degraded performance or intermittent issues signing in to RebelGive, etc. We're continuing to monitor AWS issues that have caused this interruption and will have a final update when we feel things are back close to normal.
Posted Nov 25, 2020 - 15:57 EST

Update

We are continuing to monitor for any further issues.
Posted Nov 25, 2020 - 13:34 EST

Update

We are continuing to monitor for any further issues.
Posted Nov 25, 2020 - 12:36 EST

Monitoring

We're experiencing an elevated level of login and authorization errors due to an outage in the Amazon cloud. For more details please visit: https://status.aws.amazon.com/.
Posted Nov 25, 2020 - 12:35 EST
This incident affected: API, GivingFlows, and Control Center.