SEEDS interface down

Incident Report for 4Schools

Postmortem

We discovered that a hosting provider updated an internal network address for a critical database server. Normally this is only expected during a maintenance window, but today the address changed right in the middle of the day. :-(

We have updated our application configurations to resume normal operations, and will be evaluating how these internal connections are managed so we don't trip over this kind of change next time.

Posted Jan 06, 2020 - 15:32 CST

Resolved

Services remain stable and the fix that we have put in place is expected to eliminate this issue moving forward. Thank you for your patience while we worked to identify and resolve the issue.

Posted Jan 06, 2020 - 15:20 CST

Monitoring

We have now identified and resolved the remaining SEEDS Google sign-in issues. We are now monitoring services to ensure that they remain stable. Thank you for your patience while we worked to identify and resolve the issues experienced.

Posted Jan 06, 2020 - 14:32 CST

Identified

We have identified the issue and made a change to correct the issue. At this time, users cannot login using Google in SEEDS, but can use the native SEEDS credentials and use SEEDS as expected.

We're continuing to look into the SEEDS Google sign-in issue and will provide updates as they become available.

Posted Jan 06, 2020 - 14:27 CST

Update

We're continuing to work on the issues being experienced and will provide more updates as we have information to share. Sorry for the difficulties. We're working hard to bring services back online.

Posted Jan 06, 2020 - 14:03 CST

Investigating

We are currently investigating this issue.

Posted Jan 06, 2020 - 13:03 CST

This incident affected: SEEDS4Schools (Application).