|
At Bastian Solutions, we continually strive to provide superb customer service at the highest standard, and not only reactive but proactive support. Transparency is a key component in helping achieve this objective and with that in mind, we are providing a full root cause analysis for the disruptions in service on February 20th – February 23rd, 2020 described below.
Summary:
2:16 PM EST
K+N Contacted support notifying us that Touch Stations were waiting for Touch work
3:02 PM EST
Bastian 1st Shift resource engaged other Bastian DBAs that were active in the system investigating previous nights' DB issues. Technician believed it was related and resolved the ticket
3:38 PM EST - 4:37 PM
1st Shift technician called K+N and asked if things were flowing good and were notified that they were and that they would call back if they experienced any issues.
7:02 PM EST
K+N contacted support again stating the issue was occurring again
7:30 PM EST
2nd Shift contacted the site and they stated issue had stopped
8:13 PM EST
2nd Shift technician logged in to investigate and saw tasks in the AutoStore and grabbed logs from the touch stations.
Friday 2-21-2020
9:31 AM EST
1st Shift resource called site to validate if things had been running good and got the message that they had.
11:14 AM EST
The site called in letting the team know that the issue had been occurring all night and day since reported on 2/20.
11:44 AM EST
1st shift technician escalated to our T2 team
12:15 PM EST
Escalation and T1 notified and engaged Regional Support Manager
12:30 PM EST
Regional Support Manager Spoke with the site and got a clearer picture of the issue and began investigating issues with cubed orders not being batched.
1:00 PM EST
Regional Support Manager engaged development resource to assist with troubleshooting and investigation
1:30 PM EST
Regional Support Manager put orders on hold that were stuck in the pre-processor per Development resources suggestion. Needed site to validate and test.
2:30 PM EST
Continued investigation and troubleshooting with hourly update calls with K+N
3:00 PM EST
Engaged T3 escalation resource to assist in troubleshooting with Regional Support Manager and Development
3:30 PM EST
Continued investigation and troubleshooting with hourly update calls with K+N
4:30 PM EST
Continued investigation and troubleshooting with hourly update calls with K+N
5:00 PM EST
T3 Resource and Regional Support Manager engaged Project Team to assist with Troubleshooting and engaged more Development resources.
5:30 PM EST
Continued investigation and troubleshooting with hourly update calls with K+N
6:30 PM EST
Project team sent a detailed email showing the low amount of work sent to Exacta/Autostore throughout the day
8:17 PM EST
List of orders needed to be compared from what WM had and Exacta had. Found orders that had batch numbers, but were not batched into the AutoStore. Needed K+N to validate and let us know if we should re-batch.
Saturday 2-22-2020
9:16 AM EST
Email from Bastian Unified Support Manager requesting if information had been reviewed.
10:29 AM EST
K+N Responded stating that it is believed those orders are missing.
2:00 PM - 4:48 PM EST
Emails back and forth between the Bastian PM team and K+N trying to determine where missing orders are or if they exist. K+N Batch picked remaining orders and Project team investigating supplied examples.
5:00 PM - 8:00 PM EST
The project team provided logs and examples to Development resources and communicated to determine the root cause.
Sunday 2-23-2020
10:48 AM EST
The project team sent an email notifying K+N that we have a promote ready to resolve the issue and requested time to install.
1:00 PM EST
Touch Promote installed on stations 1-7 and site encouraged to notify support if they experience any issues after promote.
Root Cause:
A bug within Touch software code caused multiple bins to be sent away and tasks canceled when "Inventory Empty" notification in touch was clicked.
Resolution:
Development Team provided bug fix and file promote that was installed on Sunday 2-23-2020
Next Steps and Preventive Actions:
K+N Team monitor batches and existing work to validate that this does not occur again.
Our commitment to you:
Bastian Solutions understands the impact of the disruption that occurred and affected operations for your organization. It is our primary objective in providing our clients with superb customer service and we assure you we're taking the required preventative measures to prevent reoccurrence.
|
Comments
0 comments