So I have a nearly full 4 TB hard drive in my server that I want to make an offline backup of. However, the only spare hard drives I have are a few 500 GB and 1 TB ones, so the entire contents will not fit all at once, but I do have enough total space for it. I also only have one USB hard drive dock so I can only plug in one hard drive at a time, and in any case I don’t want to do any sort of RAID 0 or striping because the hard drives are old and I don’t want a single one of them failing to make the entire backup unrecoverable.
I could just play digital Tetris and just manually copy over individual directories to each smaller drive until they fill up while mentally keeping track of which directories still need to be copied when I change drives, but I’m hoping for a more automatic and less error prone way. Ideally, I’d want something that can automatically begin copying the entire contents of a given drive or directory to a drive that isn’t big enough to fit everything, automatically round down to the last file that will fit in its entirety (I don’t want to split files between drives), and then wait for me to unplug the first drive and plug in another drive and specify a new mount point before continuing to copy the remaining files, using as many drives as necessary to copy everything.
Does anyone know of something that can accomplish all of this on a Linux system?
What I do is on the originator drive, I create new subdirectories and start categorizing items by content; like I’ll put all the ebooks into one directory, and all the television into another. It just makes it easier for me to find things later if I can just head to the drive with all the television on it.
If there’s a particular directory with a lot of content, I might create further divisions - maybe shows that are finished vs those who are still getting new episodes, or sitcoms vs drama, that kind of thing.
Then I make a list of how big each master directory is, and I start copying them over to the most appropriate-sized drive. I usually find that I can fit in one large directory, and a couple of smaller ones, and then the last drive gets all the leftovers. I also tape a post-it note to each drive saying something like “2022-23 television” or “science fiction audiobooks” or whatever.
I also create a new directory on the originating drive called something like ++COPIED and, once I’ve copied content to a new drive, I move the original directory to ++COPIED: I’ll still have access if I need it, but I don’t have to keep track of it any longer. Once everything is successfully copied over, I can just delete that one directory.
It’s a manual process, yes, but it does make it easier for me to find stuff when I want to look at it again later.