Come out, come out wherever you are

Knowing that your data exists on a set of tapes, but being unable to access it, can be a frustrating experience especially when there is noting actually "wrong" with the tapes and the data is almost certainly intact. This was the situation one of our customers faced with their Tivoli TSM archive when the expiration date had been incorrectly set and the TSM application [quite correctly] decided that the tapes no longer contained any data that was required. The question to us was "could our tape data recovery service recover the data?". Scenario In the strict world of data storage, usually employed within mid-range and mainframe computing, the infrastructures that exist often utilise policies to manage the lifespan of tapes and data within the system. When the data expires it is regarded that the tapes and the data on them are no longer required, and any history of the tapes contents is removed from the on-line catalogues. The case below involved Tivoli Storage Manager and an oversight in a data storage policy that meant tapes that were intended to be kept for long term archival purposes expired and, with no means of re-cataloguing the tapes and integrating them back into the storage system, an alternative solution had to be found. The Case From a large vault of tapes, 40 x LTO Ultrium 1 tapes had been designated for long term archival purposes. The tapes were part of a Tivoli Storage Manager infrastructure. Unfortunately an oversight in the storage policy relating to the data stored on the tapes meant that as far as Tivoli was concerned, the data had expired and was no longer required. The catalogue information detailing the data was automatically removed from the online storage database and the data could no longer be restored. The tapes had been stored correctly and were in perfect working order, however due to the rigid structure of the storage environment and the inability to reintroduce tapes and data back into the system there was no way under normal operating circumstances to restore the data. When Altirium were initially approached about the possibility of recovering data files from TSM backup tapes we were, from our recovery engineers past experiences with TSM data, fully aware of the complexities of the backup data structure but we were confident that a recovery would be achievable. Having performed TSM data recovery work, and from the descriptions of the data and systems from which it originated, we knew that it was unlikely any of our existing recovery tools would be able to process this particular stream of data, but were confident that it was not be beyond our capabilities to create software to recover all of the files. The requirement for the data from the LTO cartridges had become urgent and so it was agreed that we could be sent a subset of tapes to perform a proof of process. From these it was practicable to ascertain that there were no data encoding issues that would prevent us dealing with all of the tapes, and we would be able to code software that traversed the Tivoli data and produced contents listing for each of the sample tapes. Coincidentally, and fortunately, two of the tapes we were sent were associated, that is one was a continuation from another. From these we were able to identify the manner in which data was buffered at the changeover from one tape to the next and so a complete tape data recovery was going to be the result. Altirium's data recovery experts were able to analyse the raw data stored on the tape and decode the various levels of buffering and data encapsulation that surrounded the file data. They were then able develop the software solution that would allow the original data files to be extracted. After proving the process to the client, Altirium were then instructed to perform the recovery procedure across all 40 of the LTO Ultrium tapes. The recovered data files were then backed up using an alternate backup software package that the client could then use to restore and integrate them back into their system. The Solution On receiving the small sample of tapes provided by the client, we were able to read the raw data stored on the tapes. The raw data is a mix of backup meta-data and user data. The data from TSM has a number of buffering layers. Due to the heterogeneous environment which TSM operates, data can come from many different type of backup agents, each agent with its own encapsulation. Altirium's data recovery experts have been dealing with and deciphering backup data for the purpose of data recovery over many years. Studying the data, they were able to effectively reverse engineer each level of encapsulation, allowing them to develop software that would eventually access and recover the original user files. Once the decoding layer of the software had been developed this could then be integrated into Altirium's suite of recovery and data processing used to recover the data files from all of the tapes, including the complete recovery of files that spanned adjacent tapes. All the recovered user files were backed up onto tape using alternative software that the client could use to restore the data back onto their system.

For more information on Come out, come out wherever you are talk to Altirium Ltd