The Branch Engineering team was alerted on a major system outage which was impacting all our public facing components (Dashboard, Links, APIs). During this time our system was not accepting or handling any requests which severely impacted all our customers. Our team found a failing centralized authentication component that, despite having sufficient capacity and redundancy, caused this cascading service failure. As of about 9:15am PT all systems have been restored to regular service. We're conducting a further investigation and evaluation and will update this post accordingly.
At this moment, we've verifying all our critical, and then non-critical systems are 100% healthy in addition to investigating this failed component. If you have any questions or observe unexpected product behavior, please write in to Technical Account Manager team, or firstname.lastname@example.org
, or @BranchDevTeam on Twitter and we'll respond back as soon as possible. Thank you.