Nothing is lost until you’ve looked for it
IMAGINE WITH US: The worst has happened. Everything you held dear is gone. You ignored the clicking of the hard drive, those error messages, that suspicious-looking file for too long. You clicked the thing you shouldn’t have clicked. You emptied the Recycle Bin out of habit after letting your kids near your PC. And now? Now your photos are gone, your documents are dust, those precious family videos cast to the wind. Windows has gasped its last breath. All is lost.
Except that isn’t necessarily the case. Data has a habit of hanging around. And if you’re very lucky, and very careful, you could — in this hypothetical scenario — get all, or the vast majority, of your data back. Note that we’re not going to be able to help you a huge amount in the case of massive hardware failure or physical damage, so if you’ve inadvertently sent 120V to that SSD, or unwisely dunked your laptop in the bath, you’re either going to be on the hook for specialist clean room data recovery — which could set you back thousands of dollars, if it works at all — or completely out of luck.
If you’re not just imagining things, and everything does appear to have gone, stop using your PC immediately, and read on. And if everything seems fine, read on anyway. You might learn a few things that’ll save your bacon one day, and we’ll help you get your hands on some tools that every home computer pro should have waiting in their back pocket for the worst of times.
And don’t worry — we’ve not discounted the possibility that Windows has simply collapsed under its own weight, rendering your precious data comfortably stored, but otherwise inaccessible. By accessing your drive from another operating system entirely, you’ll be able to get all that stuff back, too. And hey, if you fancy doing a bit of digital forensics, but your drives are currently in a good state, why not delete a few old files from a USB stick and follow along?
BEFORE WE CAN EVEN THINK about doing any kind of data recovery, it’s worth knowing how data is stored on the typical drive. Every situation is slightly different, of course, and we’re taking an incredibly basic look at NTFS formatting here; other filesystems work in different ways, although with the right software, you should be able to affect a recovery from just about anything.
The typical drive is broken into partitions. These aren’t physical breaks, they’re logical — a certain portion of the drive’s bits are allocated, in a contiguous manner, to split the drive apart into individually managed sections, known as volumes. Information about these volumes — their size, location on the drive, and the like — is stored in the drive’s Partition Table, which sits alongside the Master Boot Record (MBR) in the first sectors of the drive. The MBR is accessed when you boot your PC, so the BIOS knows where it’s heading.
Each volume is invisibly broken into more sections. The Master File Table (MFT) is the most important of these. It’s a database that contains information about every file and folder on your drive, pointers to them, and, in some cases, even entire files, if they’re smaller than about 512 bytes. Before the MFT is a boot sector, which holds a bit more specific information about the volume — most importantly, it points to where the MFT is located — and, if the partition is both active and designated as a primary partition, loads NTldr to kick off the boot process. The same sector is also duplicated at the very end of the volume, in case the first one goes bad, as is the MFT itself.
Sandwiched between these tiny sections is the filesystem data itself — the stuff the MFT points to. These are your files. When you start a fresh NTFS drive, they’re stored quite neatly; a few months or years into the drive’s life, they really are not. Files are placed into whatever space the MFT designates as free, and while the system makes a good stab at doing this sensibly, it often needs to break your files apart, and spread them over several areas of free space, with the references to each of the parts stored in the MFT. This is why mechanical drives tend to slow down; the read head needs to jump around the drive to hit each bit of the thing you’re trying to load. SSDs don’t have this problem, since they can jump between sectors almost instantaneously (relatively speaking, at least).
STAYING SAFE
So what does this all mean for you? Well, consider this: When Windows deletes a file, it doesn’t do anything to the filesystem data. It merely scrubs the reference from the MFT, and allocates the file’s space as “empty.” The data itself is still there, almost in its entirety. For now. If the MFT later chooses to store a new file in the same space? Some or all of that deleted file’s data will be gone.
This means the first critical step of any data recovery effort (and a point we’re going to keep making) is to stop what you’re doing immediately, and power down the drive. Don’t risk any of your data being inadvertently overwritten, because if you want it back, you want it intact. Usually, you’d shut down your machine using the regular Windows shutdown procedure, which is certainly the optimal choice if your problem is hardware failure, rather than a filesystem glitch. It means your drive’s heads (if it has them) are properly and safely disengaged, and any write operations completed properly, thus avoiding potentially corrupted files.
But there’s an issue with this: If you have Windows updates waiting, shutting down in the regular way is essentially giving Microsoft license to begin overwriting swathes of your drive’s precious bytes with patches that are, at least in the current circumstances, less than helpful. So if you’ve headed for the Start menu to shut down, and you’re getting the dreaded “and update” suffix, it’s time to do something we’d ordinarily never recommend: cut the juice. Don’t even hold down your power button for 10 seconds — cut the power at the wall, flick the switch on your PSU, or pull the battery if you can. The next time we use this drive, it’ll ideally be in read-only mode, which mitigates most of the risk to your data.
VITAL PREPARATION
What’s gone wrong? We don’t know for sure. You might, though. The structure of your drive means there are a few potential points of failure — the MBR, the partition table, the MFT, a problem with bad sectors in the file storage area — and that’s not to mention mechanical issues, a problem with your motherboard’s drive controllers, a malicious attack, or the ultimate destroyer of files: you. We’ll come to human error later on. For now, we need to help the people who haven’t done something silly.
If you have a donor machine you can hook the drive up to (as a secondary drive), that’s the ideal. If you don’t have that luxury, you need to cross your fingers that your file loss isn’t being caused by a malfunction on your current machine. Keep it powered down for now. Whatever the situation, have a fresh external drive ready to transfer any recovered files to, and be aware that you may need to do a little manual cleaning up after.
Now, head to a second machine. If you’re reading this and you haven’t actually suffered any data loss as yet, follow this advice anyway: As any good Boy Scout knows, one must always be prepared. On your second machine, download the x86 version of System Rescue CD from www.system-rescue-cd.org, and write it to a CD or USB stick. There are many rescue-focused distros of Linux, but this one — ugly textbased interface and all — is ours.
PARTITION RECOVERY
Time to get some stuff back. Plug in your large external drive, and boot your affected machine from your System Rescue CD media. You may have to head into your UEFI menu and disable Secure Boot before it works. Hit Return to boot with the default options, and you’re thrown straight to a command prompt. OK, not the most welcoming introduction, but we don’t need graphical finery for these tasks. You may see a suggestion to mount your drive at this point — don’t do it. This is a trap for young players; if your partition table is hosed, you’re going to want to work on the raw disk first of all. So type “testdisk” and you can begin taking a look at your connected hardware. Don’t worry about creating a log file at this point.
You’re first given a list of all the connected storage devices — with luck, you’ll see the affected drive. If you don’t, you can safely diagnose your problem as physical: Meaning there’s a cable loose, a power failure, or your drive controller has gone up in smoke. Presuming you do see the drive, take a note of its designation (sda, sdb, and so on) as we’ll need this a little later. Ensure it’s selected, and hit Return, selecting the “Intel” partition type unless you’re working on a particularly odd drive. Now choose “Analyze.” One of two things is going to happen now: If your drive is properly structured, it finds the partition table, and lists the partitions — again, take note of these, and those of the external drive you’ve plugged in for your recovered files. If the partition table of the rogue drive is missing or damaged (TestDisk looks for a particular byte at a particular point on the drive to discover this), you can select “Quick Search” to have it scan your drive, cylinder by cylinder, for the telltale signs of partition divisions. With any luck, it’ll find something, and you’ll be given the option to restore the partition table. If not, you can choose to do a deeper scan, which is a last-ditch effort; it takes a long time, and stresses a failing drive, but if you have no other option, it’s worth a try.
CONNECTING DRIVES
With a healthy partition table, your drive should be in good enough shape to work with — you can now start hunting for your files. TestDisk usefully comes with a companion program, Photorec, the name of which is somewhat out of date — it doesn’t just recover photos, but a host of known file types from a selection of over 480. But before we can use it, we need to mount your secondary drive to give it a location to send the files to. Mounting, essentially, is the process of linking a partition of your drive to a Linux file — absolutely everything in Linux is a file — and telling Linux exactly how it should be accessing that drive.
Presuming your backup drive is NTFS formatted, we can use a program called ntfs-3g to do the mounting. First, create a mount point — the folder location that’ll represent your drive — by typing mkdir /mnt/ntfs at the prompt. Then (replacing “sdb” with the partition designation that you made a note of earlier) type ntfs-3g /dev/ sdb1 /mnt/ntfs to solidify the link between the two locations, and bear in mind that you will need the backup drive to be in good working order, too — if ntfs-3g detects bad sectors or NTFS errors, it forces it into read-only mode. Now fire up Photorec by typing photorec at the prompt.
FILE RECOVERY
This is the real meat of file recovery. Photorec uses technical knowledge of partition anatomy and the exact bytes that make up certain file types to skim through your drives and automatically pull off corrupted or lost files. And while its interface is incredibly ugly, it’s the most effective tool going. Once you’ve started it up, pick your affected drive (Photorec only works in readonly mode, so it’s safe), choose the partition that you noted earlier, and select “Search.” Photorec asks for a location to save its restored files to — navigate through the Linux file tree to find the mount point we made earlier, and pick a folder on your drive if you have one prepared. Hit C when you’re done. Photorec now sifts through your drive looking for broken, deleted, or incomplete files, and does its best to recover them fully — grab a cold one, because this is going to take a while. All being well, you should have the majority, if not all, of your stuff back at this point.
There are simpler ways to get the job done. You’re welcome to stick within Windows and try running the portable version of Piriform’s Recuva from a USB stick (www.piriform.com/recuva/builds), which offers a pretty interface and the option to select precisely what you’ve lost, rather than just batch-copying everything, but it’s more limited to inadvertently deleted files, and isn’t particularly helpful if you have an ongoing malware issue. There are also overkill methods — for instance, you could, at the other end of the spectrum, use partimage (included with System Rescue CD) to pull off a full bit-for-bit copy of your affected drive, preserving it for later forensic analysis. Check the guide at www.partimage.org/Main_Page to see how it’s done — you can compress the image as it goes, fitting it on to a smaller drive than the source, and later restore it if you suspect hardware failure.
System Rescue CD has one more niche use, too. If Windows itself has given up the ghost, and is blocking your access to your otherwise perfectly stored files, you can mount both your original and backup drives, and copy them off directly. Alternatively, and this may be a slightly more comfortable option, you could boot up another Linux live distro — the likes of Ubuntu — and manually back up using a graphical interface.
PREVENT DATA LOSS
Recovering files is our aim here, but all of this can be avoided with a sensible computing regime. If you care about your data, build in a backup routine. The prevailing wisdom states that you should follow the rule of three (three copies of your data, stored on at least two kinds of media, at least one of which is kept offsite), and it makes sense to get as close to this as you can. Start duplicating your critical files to a cloud backup service such as CrashPlan, and using external media to keep secondary copies of the most important items. Copying them to another partition on the same drive — or another physical drive in the same machine — is not adequate. Move your backup away from your main machine — to a different floor of your house, perhaps, or even off-site. If you’re hit by a flood or a fire, your insurance might buy you a new PC, but it won’t cover your files.
An automated process is best — again, something that CrashPlan and its ilk offer — because it pulls vital files when you’re not using your machine, ensuring they’re duplicated when disaster strikes. If you don’t want to pay for a subscription, it’s plausible to back up small pockets of files on free services such as Google Drive or Microsoft’s OneDrive, but unless you’re organized with your local and mirrored folders, you’re one step away from a forgetful accident. If you want to take it further, and customize your own backup solution, check out Linux app rsync (https://linux.die.net/man/1/rsync), which has a Windows wrapper in the form of DeltaCopy (www.aboutmyip.com/AboutMyXApp/DeltaCopy.jsp) — you won’t find a more versatile app when it comes to cloning data.
STOP RECOVERY FOR GOOD
We’ve discussed the ways in which your drive stores files, and how they don’t really get removed when you delete them or format your drive. What if there are files on there that you really want to stay deleted, and be excluded from any recovery efforts? We can use a little of our knowledge of file storage to work out how to obliterate old data — the best way to make sure files don’t rise from the dead is to ensure that “free” space on your drive is completely overwritten. You could painstakingly write files to your drive until it’s full, but that’s madness. Better to use a secure deletion tool.
You have a few choices here, but we tend to favor Eraser (https://eraser.heidi.ie), which goes overboard in its efforts, overwriting your files several times with specific random data patterns in order to remove them — and, on magnetic drives, the microscopic traces they leave behind — completely. We recommend you keep it hanging around — you can securely delete files just by dropping them on the app, or even have it grind through any unused space on your partitions, applying its algorithms to that space to scrub it clean of any ghostly data.
If you really want to get rid of all the data on a drive — if you’re passing your hardware on or recycling it, say — System Rescue CD includes the tool Wipe (http://wipe.sourceforge.net), which does an intense, thorough, repeated overwriting of every byte of data on a drive. Or if the hardware doesn’t matter, and you’re absolutely desperate to get rid of it, do as one of once did: Take a hammer to your drive, and throw it in a lake. But think of the fish before you do it.
WHAT ABOUT RAID?
RAID drives, as you might imagine, can be tricky beasts to recover data from, and certain configurations introduce issues of their own. Run a striped set in RAID 0 formation, for example, and you’re asking for trouble — yes, you’ll have some of the fastest storage going, but if one of your drives goes down, you’re stuck. There are various options that claim to be able to recover files from RAIDs 0, 5, and 10 — ReclaiMe (www.freeraidrecovery.com) is the most prominent — but don’t hold your breath for any decent results. On the flipside of the coin, RAID 1 and similar mirrored configurations positively aid in data recovery; if one of your drives goes down, you may need to invest a little time and money in remirroring with a new drive, but catastrophic failure is only going to come from your own hand or a malware attack. If this is the case, you have the same chance to recover files as you would with a single drive.
The other issue RAID introduces is the controller itself. If whatever’s managing your RAID configuration — your NAS, your main computer, or specialist RAID hardware — goes bad, it could throw your drives into a position where recovering data is entirely impossible. It’s not unknown for specialists to be able to engineer specific tools for specific instances of RAID failure, but this is a highly costly procedure, with no guarantee of success.
We don’t have an answer for the irony of RAID being both safer and more dangerous than storing on a single drive, other than to suggest, again, that you always keep stringent backups away from your regular hardware.