The majority of the data recovery work we receive are just standard jobs. The customer doesn’t need to have it expedited or rushed, and they wait the 7 to 10 business days to get their data back. Sometimes though, we have a job that is so critical…it gets done the exact same day.
The Call
At 5:30PM on a Saturday in June of 2013 we get a call from a very big client that needed RAID data recovery. When I say this was a big client, I mean you would be hard-pressed to find a person who hasn’t heard of this company or utilized the things they manufacture. The gentleman on the phone sounded as if he had just taken a few deep breaths to hide his sheer panic. He described an issue that he had where the server that stored a years worth of designs had just failed. It was a RAID 5. It lost two hard drives simultaneously and brought the array down. No big deal usually, because this company has adequate backup programs in place…or so they thought.
When the tech went to restore the data from the backup, it was found that the data on the backup media was corrupted. Not only was it corrupted, but it hadn’t even been backing up the down server for the last 3 months. Ouch!
We were asked how long it would take to recover the data. It was a 6 drive array, so it would take some time to get all the drives imaged, and then rebuild the stripe, and then destripe and then pull the data. They had no time for all of that. They needed the data by Monday morning. Remember…we received this call at 5:30PM on a Saturday!
The Plan
The customer decided they were going to fly the drives in right then. They were coming from Missouri on a company jet, and would be at the Temple airport by 9:30pm. A rep was put in charge of the server, and when he landed they had a car waiting for him which brought him directly to our office. He arrived just after 9PM, and he dropped it off with us.
The plan revolved around doing the recovery as quickly as possible. If the drives had no physical problems then that would be a huge help. Most likely they didn’t…since it’s rare to have two separate hard drives suffer from a mechanical problem at the exact same time…unless it’s related to some electrical issue, but in that case it would have effected other drives as well.
We had decided to forgo the usual imaging step and work from the source drives. This saved a tremendous amount of time, because imaging a hard drive, even one that is in perfect condition, can take hours. Sometimes as many as 12 to 14 hours. We don’t ever work from source drives. We strongly advised our customer against this approach, but they insisted. We even had them sign a waiver stating they understood the risk involved. Granted, nothing we do can actually damage or write data to the drives, but we still prefer to work from cloned copies. With RAID arrays, you really have to play around with some of the parameters to get the stripe reassembled properly, and we don’t like doing anything that might accidentally alter the original data on the drives.
The Rebuild
We checked all of the hard drives, and just as I guessed, nothing was physically wrong with the two drives that failed. I had banked on it being a backplane issue…something in the RAID controller itself, and it was. So we moved forward with the recovery efforts. The longest part of the process in RAID recovery work is actually rebuilding the stripe. We are essentially piecing together a puzzle. In this case, the pieces are tiny bits of data that make up large files, and they are scattered amongst 6 different hard drives.
Once the stripe parameters are determined, you can then utilize another piece of software to emulate a server environment. If your calculations on the stripe size, drive rotation, header size, offsets, etc. are all correct, then you should be able to see the directory structure when you plug this information into the software with the drives connected.
By 4:30am…less than 12 hours after we received the call…we had the stripe rebuilt. Normally we would then destripe the data to a single image…but there was no time. We knew the directory they needed, and we started to pull the data from that directory first. We pulled everything directly to a new external hard drive. By 9:45AM the recovery was complete. Just over 16 hours after the call.
We called to have the rep come back to the office, I think we actually stunned him that it was finished that soon…I also think our 9am call may have woken him up…at least it sounded that way. By 10:30AM it was wheels up for the jet and by 3:30PM we got the call confirming all of the data was recovered and restored.
Disclaimer
This case is a rarity. While it represents our dedication and expertise when it comes to RAID data recovery…or any type of data retrieval for that matter…it’s not often that we have a client willing or able to fly their media here on a private jet within a few hours of calling us. We’ve only had that actually happen one other time, and that was for a major oil company. It’s also not often that we don’t create images of the source drives. Again that was at the behest of the customer. So while we were able to complete a monumental recovery job and the customer was back in business less than 24 hours after the initial call, it’s not typical of most recoveries. On average, expedited jobs are completed and shipped within 1 to 3 days. However, if you have your own jet, and you’d like to try and get same day service, we will be more than happy to oblige.
