Hello all,
We are reaching out to you to provide some information about the root-cause analysis of the recent SSO issues, what we are doing to remedy them and improve service quality, and what we will be doing to communicate the issues better in the future.
The root cause was running into capacity constraints on our systems including our database servers. A number of steps have already been implemented like migrating historical data queries to secondary servers and upgrading systems to larger database servers. We are working on completing this upgrade this week and it was, in fact, that upgrade that caused today’s outage. We obviously backed off the upgrade as soon as we were aware of the impact the migration was causing and will be re-attempting this migration after hours during periods of lower system activity.
We are committed to ensuring that SSO services are functioning well and working diligently to ensure a high quality of service. With this improved quality of service, we hope to not run into these issues anymore (ideally extremely rarely with the goal of avoiding downtime altogether), but we commit to better communicate in the event that future incidences should arise.
Best regards,
Product Management Team