Knowledgebase Article Number
Published Nov 13th, 2020, at approximately 2:15 p.m. EST.
At Bastian Solutions, we continually strive to provide superb customer service at the highest standard, and not only reactive but proactive support. Transparency is a key component in helping achieve this objective and with that in mind, we are providing a full root cause analysis for the disruptions in service on between October 21st and November 8th, 2020 described below.
On Oct 21st, 2020 a support call was received by Support Specialist I Brent Smith, stating that bins were circulating but no work was being presented in Exacta Touch. While working with tier 2 support resources, Brent determined that there were numerous tasks in Autostore with no corresponding work in Exacta, and that the process of loading and then discarding these tasks was blocking valid work from making it to the ports.
Support removed these 'ghost' tasks as quickly as they could be identified.
We also determined at this time that several orders with the 'CART-' name format were having trouble in preprocessor. These orders were placed on hold to allow other work to enter the system.
On the morning of Oct 22nd Bastian Solutions development team provided a hotfix which causes PreProcessor to confirm a task group exists in Autostore before re-attempting the batch process, in order to prevent the creation of 'ghost' tasks.
On October 23rd the issue of 'ghost' tasks was reported again. The Bastian Solutions development team confirmed the promote was working correctly and began reaching out to the Autostore team for additional assistance.
It was determined that Exacta was now failing to submit new work similar to the issue with the CART orders from Oct 21. Bastian installed another promote to correct an issue with assignment logic that caused this late on the 23rd.
No major issues were reported again until Oct 29th. The 'ghost' tasks problem reappeared and was attributed to communication issues with Autostore. Bastian Support cleared the ghost tasks and reached out to Autostore Support again for help.
On Oct 30th Bastian received Autostore updates from Autostore support and installed these.
No major issues were reported again until the evening of Nov 9th, when preprocessor appears to have failed to send tasks to Autostore again. This was resolved by restarting the PreProcessor service, but we will need to capture and investigate another example more thoroughly in order to see if this was related to the assignment problem.
Next Steps and Preventive Actions
Our commitment to you:
Bastian Solutions understands the impact of the disruption that occurred and affected operations for your organization. It is our primary objective in providing our clients with superb customer service and we assure you we're taking the required preventative measures to prevent re-occurrence.