Knowledgebase Article Number

360043629894

Description

Published February 26, 2020, at approximately 4 p.m. EST. 
Last updated on February 26, 2020.

Resolution 

At Bastian Solutions, we continually strive to provide superb customer service at the highest standard, and not only reactive but proactive support. Transparency is a key component in helping achieve this objective and with that in mind, we are providing a full root cause analysis for the disruptions in service on February 20th 2020 described below.
 

Summary:

3:02 AM EST
The client notified Bastian that the Exacta touch was not working correctly getting an exception error

4:24 AM EST
Bastian 3rd Shift resource reached out to escalation resource and begin further investigation

5:00 AM EST
Escalation resources engaged DBA escalation resource and they came online and began investigating

5:57 AM EST 
After DBA resource moved AutoStore statistics DB to another node they were able to shrink both the ExactaDB and AutoStore statistics DB correcting the issue with Reports

6:10 AM EST
3rd shift resource touched base with site and after restarting services on the picking ports all were able to pick successfully.

Root Cause:

Overnight SQL jobs were not functioning properly after DB failover caused by SQL upgrade on Sunday 2/16. This caused the log drive for the SQL server to fill up and cause the issues experienced

Resolution:

ExactaDB and AutoStore Statistics DB shrunk

Next Steps and Preventive Actions:

Bastian DBA resources have put together a list of next steps.

• AutoStore Backup Purge - Schedule 30-minute downtime to correct and run job manually then schedule nightly
• Bastian Maintenance Plan backups- Need to remove backups older than 3 days and take a full backup
• Delta Purge Job - Need to validate it runs correctly and enable

Our commitment to you:

Bastian Solutions understands the impact of the disruption that occurred and affected operations for your organization. It is our primary objective in providing our clients with superb customer service and we assure you we're taking the required preventative measures to prevent reoccurrence.