Ok....now bear with me because I'm no tech guru. I'll just be explaining what I was told, whether I really understood it or not...
This morning's work got started an hour late due to some problems shutting things down, which was fine really because they extended the duration of the planned outage by two hours yesterday anyway to be safe.
So...all is going well, data is being backed up fine, until two (yes two) of the tape drives fail forcing us to have to rely on the old slow hardware. Things start to fall behind.
System is backed up, is flipped into production mode and crashes. Badly. REALLY badly. Here's where we run into serious problems in regards to our projected downtime.
Problem is escalated to the manufacturer, priority one. Funny thing, we figure out the solution before they do, make the changes, everything working fine.
Eventually, they call us and say "Hey...we know how to fix it" and we say "Yeah, you do what we just did a while ago" and they're all doing the Keanu Reeves "whoa" thing....
So....that's what happened.