At Bastian Solutions, we continually strive to provide superb customer service at the highest standard, and not only reactive but proactive support. Transparency is a key component in helping achieve this objective and with that in mind, we are providing a full root cause analysis for the disruptions in service on February 20th – February 23rd, 2020 described below.
Summary:
2:16 PM EST K+N Contacted support notifying us that Touch Stations were waiting for Touch work
3:02 PM EST Bastian 1st Shift resource engaged other Bastian DBAs that were active in the system investigating previous nights' DB issues. Technician believed it was related and resolved the ticket
3:38 PM EST - 4:37 PM 1st Shift technician called K+N and asked if things were flowing good and were notified that they were and that they would call back if they experienced any issues.
7:02 PM EST K+N contacted support again stating the issue was occurring again
7:30 PM EST 2nd Shift contacted the site and they stated issue had stopped
8:13 PM EST 2nd Shift technician logged in to investigate and saw tasks in the AutoStore and grabbed logs from the touch stations.
Friday 2-21-2020
9:31 AM EST 1st Shift resource called site to validate if things had been running good and got the message that they had.
11:14 AM EST The site called in letting the team know that the issue had been occurring all night and day since reported on 2/20.
11:44 AM EST 1st shift technician escalated to our T2 team
12:15 PM EST Escalation and T1 notified and engaged Regional Support Manager
12:30 PM EST Regional Support Manager Spoke with the site and got a clearer picture of the issue and began investigating issues with cubed orders not being batched.
1:00 PM EST Regional Support Manager engaged development resource to assist with troubleshooting and investigation
1:30 PM EST Regional Support Manager put orders on hold that were stuck in the pre-processor per Development resources suggestion. Needed site to validate and test.
2:30 PM EST Continued investigation and troubleshooting with hourly update calls with K+N
3:00 PM EST Engaged T3 escalation resource to assist in troubleshooting with Regional Support Manager and Development
3:30 PM EST Continued investigation and troubleshooting with hourly update calls with K+N
4:30 PM EST Continued investigation and troubleshooting with hourly update calls with K+N
5:00 PM EST T3 Resource and Regional Support Manager engaged Project Team to assist with Troubleshooting and engaged more Development resources.
5:30 PM EST Continued investigation and troubleshooting with hourly update calls with K+N
6:30 PM EST Project team sent a detailed email showing the low amount of work sent to Exacta/Autostore throughout the day
8:17 PM EST List of orders needed to be compared from what WM had and Exacta had. Found orders that had batch numbers, but were not batched into the AutoStore. Needed K+N to validate and let us know if we should re-batch.
Saturday 2-22-2020
9:16 AM EST Email from Bastian Unified Support Manager requesting if information had been reviewed.
10:29 AM EST K+N Responded stating that it is believed those orders are missing.
2:00 PM - 4:48 PM EST Emails back and forth between the Bastian PM team and K+N trying to determine where missing orders are or if they exist. K+N Batch picked remaining orders and Project team investigating supplied examples.
5:00 PM - 8:00 PM EST The project team provided logs and examples to Development resources and communicated to determine the root cause.
Sunday 2-23-2020
10:48 AM EST The project team sent an email notifying K+N that we have a promote ready to resolve the issue and requested time to install.
1:00 PM EST Touch Promote installed on stations 1-7 and site encouraged to notify support if they experience any issues after promote.
Root Cause:
A bug within Touch software code caused multiple bins to be sent away and tasks canceled when "Inventory Empty" notification in touch was clicked.
Resolution:
Development Team provided bug fix and file promote that was installed on Sunday 2-23-2020
Next Steps and Preventive Actions:
K+N Team monitor batches and existing work to validate that this does not occur again.
Our commitment to you:
Bastian Solutions understands the impact of the disruption that occurred and affected operations for your organization. It is our primary objective in providing our clients with superb customer service and we assure you we're taking the required preventative measures to prevent reoccurrence.
|
Comments
0 comments