blarg?

More Different Than Thinking

Command And Control

10:52 <@mhoye> Weird. Anyone here seen a situation where ls won't show you a file, even ls -A, but you can copy it out of the directory?
10:52 <@mhoye> A directory full of other, similarly named files that you can see just fine?
10:52 <@mhoye> I am suspecting filesystem corruption here.
10:53 <@shaver> if you grep for the filename in the ls -A output, do you get a hit?
10:53 <@mhoye> Nope.
10:53 <@mhoye> But if I cp filename ../ it appears in the lower dir.
10:54 <@Myke> what FS is that?
10:54 <@mhoye> hfs+
10:55 <@mhoye> with ACLs enabled, but there's nothing in there that would indicate a problem.
10:55 <@Myke> I have run into that
10:55 <@shaver> not me
10:55 <@shaver> I would shit myself
10:56 <@mhoye> I considered that as a plan, but I'd like to explore other options first.

Steve Jobs, the joke goes, doesn’t use a calendar; he just yells at people until it’s the right day.

So, you know how Macs are all about hiding the implementation details from users? Let me tell you about this funny thing I learned about OSX’s HFS+ file system today: the “kIsInvisible” flag. You can flip it on and off if you have the developer tools installed – SetFile is the name of the command-line app you want. Once that bit’s flipped, as far as I can tell the only way to find out that file X is in that directory Y is to already know it’s there.

To my knowledge there are at least four ways that a Mac can hide the contents of its filesystem from a user. They are:

  • The standard unixy dot-filename convention. Largely harmless, an “ls -A” in a terminal will show you everything.
  • Munging permissions. Again, the Finder won’t show you stuff you don’t own, but “ls -A” in a terminal will. If you need to move a Mac from one Active Directory domain to another, this is where all your user’s files went.
  • This kIsInvisible flag, which is apparently a legacy holdover from the pre-plus HFS days but still works like a charm, hiding the file from users (even root) and any scripts that process files in that directory using an ls of the contents.
  • The “/.hidden” hack, in which a file called “.hidden” in the root of your boot drive can contain a list of files that OSX will never, ever show you.

Those last two in particular are 100% awesome, and I’m sure the motives for implementing them looked excellent to somebody. And there are, of course, no security implications to any of this at all! None! But it’s nice to be reminded every now and again that for all its rounded-off brushed metal finish, OS X fundamentally doesn’t trust its users; there are some surprisingly sharp edges under the hood here.

But you’re never supposed to open it, though, right? So that makes it your fault.

3 Comments | Skip to comment form

  1. neil

    That seems odd. I use setfile to set -a V all the time, and it *always* appears in the list when I run ls -la.

    Is there a sequence you’re using where you can reproduce this? Now I’m curious…

  2. Coop

    Enjoy your voided warranty! :)

  3. mhoye

    Sorry, Neil. We discovered this when we were having a problem on our SAN, and we’re pretty enthusiastic about not having it happen again.