January 6, 2004

It All Happened So Fast.

Filed under: digital — mhoye @ 5:12 pm

So, if you’ve been looking around wondering where the rest of my site went, the answer is that it is gone. An attempt to remove a subdirectory of my web site was typed into a the wrong terminal window, and the Linux command line is a harsh mistress. I managed to control-c fast enough to save some stuff but not much.

rm -rf * is not something you want to treat lightly, no sir. It’s been five years since I made that mistake, and I hope it’ll be another five ’til I do it again. I’ll rebuild it as time allows, but we’ll see how that goes.

I tried some pretty fancy stuff to get that stuff back, including scouring the google cache, the wayback machine at and some more esoteric stuff like checking /proc for open file handles in the hopes that some of it could be resurrected, but it was all to no avail. A Movable Type rebuild and some google-searching got me back my weblog and PHP sidebar links, but the rest of this stuff is going to have to be rebuilt from scratch.

Well, fuck.


  1. Well, it’s a good thing you had good backups. (Note: Nobody has ever uttered those words non-ironically.)

    One of the neat things about Panix is that they run their storage systems on these NetApp boxes that have automatic hourly and nightly snapshots, so if you delete something (or even overwrite it with bad data), you can get it back how it was 1-5 hours or 1-2 days ago. It’s saved my ass more than once…

    Comment by Mike Kozlowski — January 6, 2004 @ 6:08 pm

  2. Oh, *fuck*.

    I’m very sorry to hear.

    Comment by Jeff — January 6, 2004 @ 6:29 pm

  3. Ouch! I made that mistake once, which was one time too many. Unix is a fickle bitch.

    Comment by Lara — January 6, 2004 @ 6:37 pm

  4. Well, it was pretty much inexcusable of me not to have my shit backed up.

    The phrase that popped into my head at first was “those whom the gods would destroy, they first make proud”, but I’m hardly worthy of a line of that gravity. Those who the gods would infuriate, they first make blase and casual, I suppose.

    Comment by Mike Hoye — January 6, 2004 @ 7:34 pm

  5. I’d love to get snapfs going for this machine, but it’s not going to happen before 2004Q3, at the earliest.

    I’ll try to set up something hackish this weekend that’ll at least let you go back one day. I might need more disk, though.

    Comment by shaver — January 6, 2004 @ 8:35 pm

  6. Don’t spend a lot of time on it. I’m gluing together gzip, tar and sftp with some bash-flavored glue right now, and that should keep me out of trouble for the time being.

    Comment by Mike Hoye — January 7, 2004 @ 12:29 am

  7. I almost hate to suggest it, but do you have enough spare disk space to dd the disk image to some other location and run some recovery routines on it? I’m not going to suggest that manually walking inodes is easy or anything, but it might be workable for some of the smaller important stuff.

    Really sorry to hear about that. Wish I could help.

    Comment by Gnomon — January 7, 2004 @ 2:45 am

  8. I’m not, lucky us, in charge here, but I think that it would be easier and more space-efficient to just tell cron to tarball a user’s non-movable-type files off to some corner of drive at four thirty in the morning.

    Comment by Mike Hoye — January 7, 2004 @ 2:52 am

  9. I looked at doing just that, Gnomon, when I heard Mike’s anguished screams, but I don’t have spare disk on that machine without breaking the RAID set, and that seemed like a way to go from “moderate frustration” to “utter catastrophe” in the blink of an eye.

    Even then, I tried to get in with strings on the raw device, and that didn’t turn up appropriate-looking but the movable-type files. Maybe with better search criteria, or debugfs, but I don’t have great expectations.

    mc’s undelete didn’t help either, likely because the inodes were quickly recycled by our mail daemon or the M-T rebuild.

    Comment by shaver — January 7, 2004 @ 11:27 am

  10. (BTW, I recommend “rsync -e ssh” over any hand-glued tarring and bzipping and sftping, but that’s just me.)

    Comment by shaver — January 7, 2004 @ 11:49 am

  11. (BTW, I recommend “rsync -e ssh” over any hand-glued tarring and bzipping and sftping, but that’s just me.)


    Comment by Coop — January 7, 2004 @ 1:59 pm

  12. rsync is the mantis.

    Comment by Sean Neakums — January 9, 2004 @ 5:50 am

RSS feed for comments on this post.

Sorry, the comment form is closed at this time.

Powered by WordPress