blarg?

All Networks Are Political

Moar Hydrant

mhoye The terrible thing about being this awesome is that it’s hard for people to fully appreciate it when I make it look so easy.

I spent part of last week getting my office out from under the catastrophic failure of our three ostensibly-redundant AC units in the server room. They’re water-cooled units that were all fed, very much to my surprise, from a single cold-water pipe that some baboon in the basement of our building broke. When I got to work Wednesday morning the server room had spiked to 43C, showing no sign of going anywhere but up, and about a quarter of my precious machines had already shut themselves down in self-defense.

We recovered from that mess with a little bit of luck, some nontrivial effort and a ton of advance planning done months ago; we did it a lot faster than anyone thought we could, and I’m proud of it. Still, it revealed a few gaps in our disaster-management and -recovery processes that need to be addressed. We were even lucky enough that the only immediate equipment failure was a single drive in an unused raid set, which was a tiny stroke of luck, and none of the machines that shut down to protect themselves were near the core of our dependency tree, which was huge. But the cold hard fact of the matter is that an orangutan turning the wrong valve seven floors away got quite close to paralyzing my company, and the orangutan found that valve before we did.

We’re going to solve the hell out of that problem, make no mistake; we are going to solve it so hard it’ll be walking funny for months. And ending the week with the personal thanks of my CEO felt pretty good. But this little vignette brings me, circuitously, to the thing I really want to talk about.

Every couple of months, somebody writes an article about how terrible their IT departments are for not letting them install whatever they want, how IT is just there to hold them back and not let them have any fun, and mostly that’s because IT is lazy and doesn’t feel like catching up, much less keeping up, with these shiny modern times. Even my friend David Eaves recently referred to a “distant IT overlord” in an otherwise well-aimed post recently, much to my chagrin.

First off, just to be clear let me tell you this: I would love to be a Distant IT Overlord. Love it! But as it stands users keep coming to my office with their needs and questions, and despite my best efforts (starting all my conversations with “no”, answering questions with “cut the baby in half” or “shoot the hostage”, you know, the usual) I still seem to end up actually trying to actually help them get their jobs done.

Embarrassing, I know! I keep pushing for those little changes in the decor that could make all the difference, like a moat around the office or having some random intern’s severed head hoisted on a pike by the server room door, but we never seem have the budget. Maybe next year.

These articles are invariably, and tellingly, written by people with no IT background at all. Which is OK, that’s most of us, but they’re also written by people with no idea what the implications of their demands might be, and no real interest in finding out. Some of that, as Eaves notes, is cultural and deeply ingrained, but I think he only gives the barest nod towards the reasons those attitudes exist and (surprisingly, from a veteran negotiation and policy wonk) to the idea that there’s an actual culture there, and hence an understanding to be had and if we’re patient and a bit lucky some common ground to be found.

I like to think that I understand a part of it, so I’d like to lay some of it out for you here. This isn’t to garner sympathy; I’m not going to claim that my job is harder than yours, and I know there’s a lot of people in IT who just have no business interacting with other humans. I’m even a little unhappy that this discussion always ends up with the Us V. Them tone they always seem to. But to some extent that is inevitable; in any company, the IT department is ultimately responsible for the well-functioning and continuity of the company itself, not the well-being or ease-of-use of the individual user or even the individual project, and that stance can be inherently problematic.

Which is not to say that everyone in IT is going to ascend bodily into heaven, sure, but we spend our days wrangling complicated, frequently opaque systems subject to an obscene variety of unsavory threats. The consequences of a misstep can be fierce and doing your job just right often means nobody ever notices, a fact that can warp your perspective a bit.

But it’s not that we don’t love you, users. Honestly, we do. We hate a lot of these things you complain about as much or more than you do, but the laws of physics and economics look very different in the server room than from your home office.

The canonical example is, why can’t I have more storage space? Why do I have to limit myself to five hundred megabytes on the network drives, why do I need to clean my mailbox out so often when I only have a few hundred megabytes of mail in it, how come you’re always asking us to delete our old mail when I can go down the block and buy a terabyte of drive space for a hundred bucks?

The short answer is that our equipment is really expensive. Our drive space, for example, costs somewhere between fifty to one hundred times what yours does, and it’s lot more complicated than just “buy a drive and plug it in”. And that fifty-to-one-hundred not an exaggeration; that one terabyte is four SAS drives, which cost about 15x what you think they should on their own. And they need drive bays on the NAS to live in, that may or may not be there. Will we be able to just spin that drive up or would we need to rebuild the array? Can we dynamically resize those partitions? Maybe not. And do you have several spare terabytes of extra space on the tape backup system, for redundant offsite copies? Do we even have enough slots in the tape robot to put more of those tapes into it? How many times can we add drives before we need to reconsider the amount of power coming into the server room? Buy enough of those, and do we need another UPS to power them all when the building power browns out? What do those cost?

That’s the process, and it sucks, but it’s also the only way any of this can be made to work reliably at all. Our gear doesn’t cost that much for no reason; it costs that much because having our whole company sitting on their thumbs with no email costs so much more. What does this change force us to also change, and what does that cost? Maybe a lot, and if we don’t have the budget for all of it we can’t do the first bit and we end up saying no. So we say we can’t give you an extra gigabyte of space on the mail server because it’s too expensive and you go back to your office thinking those guys in IT are a bunch of goddamn lazy cheapskates. And I understand, ’cause that’s sure what it looks like.

But that extra terabyte of drive space might not solve the real problem; the mail you want to save. Did you know that Exchange data stores can only be a certain size before performance starts to seriously degrade? Yeah, it surprised me too. You can set up multiple stores to keep those numbers down but it’s recommended you do that across different storage devices, so we need new servers for that, and then we need new OS licenses and CALs and if it seems like I’m saying it never ends that’s because it’s pretty much like that. These weird, seemingly arbitrary restrictions are all over the place (Microsoft, I’m looking at you) and working with and around them is definitely nontrivial.

And we’re on a budget, and the economy is in the toilet. And you want, but your department never seems willing to foot the bill, curiously. And it doesn’t help when we do things like check timestamps or file sizes just to see what our actual usage looks like, and find out that the reason you need more drive space is that you won’t delete your mail from 2002 (that you haven’t touched since 2004; yes, I can check that) or get rid of that pirated .ISO you’re saving or the porn or MP3s that shouldn’t be there anyway. Because (and this happens all the goddamn time) first of all, if you’d just done a little housecleaning you’d have plenty of room, and second now that I’ve seen that I need to talk to HR. And if it comes to that I guarantee that the story that gets around will be that you made a simple request to IT for a bit more drive space and the Distant IT Overlords had you escorted to the curb. That’s not what’s happened, sure, but in the absence of respect fear works just fine.

And, and and. And all of that is most basic example I could think of, and I’m still glossing over the uglier technical bits.

The most amazing thing about all this is that it’s not only possible for me in IT and you in userland to hold hands and find our way through this thicket together, but it’s possible to do it remarkably well; user expectations can be managed and met, networks can be disciplined and machines secured, management can be made content and all of that done while the organization grows and IT cooperates with users in nurturing it along. But it can’t happen case-by-case, and it can’t happen without resources that may be more scarce than you realize.

So here’s the thing: if you want more disk space or a new browser or anything from IT at all that isn’t a correctly-functioning version of the status quo, don’t ask me for it. Not because I don’t care, and don’t want people to have nice things, but because IT can’t do what you want for individuals users, for free. Go to your managers, and their managers, and my managers. If you have management onside then your problems are my problems, but more importantly I will have things like budgets and project milestones and all these things we need to make changes to a complex environment without boning the entire thing.

Sure, it will take time. But if your argument is compelling (modulo the resources it requires) you will either get what you want, or have a clear understanding of why it’s not going to happen. But you need to step back from the hardware far enough to understand that any complicated system is ultimately political; it’s about negotiating, weighing resources against priorities and carefully crafting plans towards possible futures.

You also need to step back from the hardware because it’s not your hardware, it’s mine, and you don’t know how it works and I’m possessive and don’t touch it. Which is to say that yes, in addition to being political it’s also personal, and you’re going to have to sell your idea.

Which is all to say, help me help you. I know, the Other Kind of IT Guy is out there, and he’s a bit of a dick, but he’s relatively rare and a management issue too. But if you sneer and think this simple and that I should just let you do whatever you want, that tells me enough about what you think of me and my job that you’ll have a hard time convincing me I should help you out.

And that guy we had to escort out of the building? Yeah, don’t be that guy. I’ve got my Distant IT Overlord boots right here.

40 Comments | Skip to comment form

  1. David Humphrey

    Great post. Long ago I worked as a junior IT guy at a major Bay Street financial shop, and I’ve seen almost all of what you describe above. I know all about the divide between what’s happening on the floor, and what is happening in the server room. In our shop, these two spaces where physically separated by a floor, which made it worse.

    I think that one of the things non-techies have no clue about is the difference you point out above, namely, that what an IT shop uses and what you use at home are not even close to the same thing. The web has made us all think that everything is virtual. In reality, there is more physical stuff than ever running all those virtual worlds. Our IT boss used to have a couple of days per year where he’d do an open house in the server room (guess who had to recable to make it look pretty in there?). I found that most people didn’t come, and none of the people who needed to come. I don’t know how you get over this. I found it strange, because I’d be forced to attend all kinds of mind-numbing financial meetings in their world so I could understand their issues. It just never worked the opposite way.

  2. Mike Bruce

    I think you give the IT world in general too much credit. Bad IT is not the exception. If it were, the world wouldn’t be filled with companies mandating IE6.

    I know good IT people, and I’m sure you’re one of them, but the profession as a whole is tainted by dumbness and hostility to users.

    I can’t say I find much to object to in the Slate article, either. Yes, there are lots of aspects of IT (shared disk space, Exchange weirdness, etc.) that are necessarily constrained by budget and the limitations of the physical world. Being able to use a decent variety of software isn’t one of them.

  3. mhoye

    “Being able to use a decent variety of software isn’t one of them.”

    Oh, but it is. Particularly if you need to test and deploy that software, to a large number of machines whose users don’t need it to do their jobs. It’s not just about what you can do, it’s about what else you’re not doing while you’re doing that.

  4. Jamie

    It goes even further than that. For all that Michael the Bruce is laughing at IE6, he has no idea how much back end groupware doesn’t work with anything newer. God help you if you’re on a Mac in that environment and aren’t technical enough to deal with a VM to get to it. Updating that software takes months, is expensive, and probably STILL doesn’t fucking work with anything but IE6. Oh, and if the upgrade goes south, you better have a backup from the minute before you started, because the newer version changes the whole gods be damned database format and there is NO going back.

  5. mhoye

    …which is, ultimately, my entire point. There’s a damn good reason they’re not upgrading, if you’re willing to listen to it.

  6. Mike

    Oh, sweet Jeebus. I ran into Farhad’s article when it was quoted approvingly by someone that’s written similar screeds (Cory Doctorow, for those keeping score at home).

    Because there’s never a problem with someone downloading software, and then finding out the hard way that the “open source” software actually has some clauses in that license that apply when used in a commercial setting.
    There’s never a worry about users taking a USB key with virus-infected files on it and then plugging it into a machine that’s not running antivirus software (or even an OS from within the last 3 years, never mind patches) because the vendor refused to support it if antivirus was installed and running (or patches applied), and causing an outbreak of whatever the virus du jour is.

    The software that users choose will always work with the existing infrastructure — there’s never a need to worry about a vendor not supporting a web browser or an OS. And if there is a conflict, then making the changes to support it is not only possible in all cases, but trivial and transparent to the users.

    Unfortunately I don’t seem to live in Farhad and Cory’s world where everyone just needs to be enabled to do their own thing so that we can reach that mystical phase 3, PROFIT!

  7. mhoye

    It’s worth mentioning here that Cory Doctorow, despite his pretensions at technical jargon, is an ignorant simpleton.

  8. Mike Kozlowski

    All else aside, enjoy famed tech writer and dumbshit David Pogue talking about what IT people do, if you want to enjoy raw cluelessness.

  9. Mike Bruce

    It goes even further than that. For all that Michael the Bruce is laughing at IE6, he has no idea how much back end groupware doesn’t work with anything newer.

    Don’t care. No, really, I don’t care. Software is for users. Figure something out. Install multiple browsers. Whatever. And if you have software that works with IE6 but not IE7, you have software written by ignorant morons and you should seriously consider what other monumental dumbness is lurking in there.

    Oh, but it is. Particularly if you need to test and deploy that software, to a large number of machines whose users don’t need it to do their jobs.

    So don’t do that. Whatever, right? If Bob in accounting wants to install Firefox, let him deal with it.

    There are a bunch of problems in IT, but I will highlight this one: Trying to solve human management problems with technology.

    Rather than trying to prevent people from installing regular programs and using whatever websites they want, which is doomed to failure and is very frustrating for everyone, punish people that do dumb shit. The solution to dudes looking at porn at work isn’t to design a complicated system to block porn, it’s to fire dudes who look at porn! If they cost a bunch of time and effort by doing something unreasonable to gunk up their computer, formally reprimand them for it, or whatever consequence is locally appropriate.

    It’s untenable and counterproductive to try to treat a large networked operation as a perfectly controllable environment.

  10. mhoye

    I know you don’t care, but you know that’s insane, right? If Bob in Accounting fucks up his computer and we let him deal with it, odds are good that people or bills aren’t going to get paid that month.

    Home software is for home users. Company software is for the company. Don’t like it, make the case, but “don’t care” won’t do it.

  11. Mike Bruce

    So let Bob explain to his boss that he’s an idiot, and let his boss fire him for being an idiot.

    Presumably if Bob poured his coffee into his computer, that would also cause trouble. Or if he brought a small child into the office and let them play with his computer. Or if he regularly took LSD before doing the numbers.

    Computers can’t fix stupid.

  12. Jamie

    You may not care, but we don’t get to pick what accounting software the Finance Dept. uses. They do. We don’t pick the software that HR wants, they do. In fact, we get very little choice in what software gets used by various depts. They decide what best meets their needs, and we have to make it work. Enterprise IT is a whole different world from a small software shop or an ISP environment. Add in the requirement that we’re dealing with software and data that are covered by ITAR and we can’t just let anyone do anything they want. It’s cost my current employer on the order of fifty million dollars in fines to the Fed in the past.

    As far as letting Bob in accounting fuck up his machine, that’s a non-starter as his fuck up might very well bring the whole network down.

    NASA was pretty much the land of users being allowed to do whatever the fuck they wanted, and the network was unusable in general. We had dedicated links in a single facility because the network was so fucked up we couldn’t move traffic coming in from our satellites to the Origins, in the same room let me note here, for processing because there was so much shit flooding the network. I highly suspect NASA is no longer a free for all, but it’s been ten years and I don’t have to care anymore.

  13. mhoye

    “So let Bob explain to his boss that he’s an idiot, and let his boss fire him for being an idiot.”

    And let company bills not get paid? And maybe contracts get lost? And let company employees who are living too close to the wire not make rent that month, or child support, or their insurance payments, or God knows what else?

    Yeah, awesome plan; not to put too fine a point on this, but it’s amazing that you can ignore network effects when we’re talking about a device plugged into a network. If that’s your idea of risk management, you are an idiot.

  14. Anne

    I can give you a real world example of an IE6 problem that there is no getting around. Would I like to have all of my people on at least IE7? Yes, I would. However, Nissan, in their infinite wisdom, has a portal that doesn’t work on anything but IE6. ANYTHING. Not to mention the specific service pack and all the other dependencies.

    If my users cannot use that portal, vast amounts of money are lost. It cannot be waved away. This is the stuff we are forced to think about in the real world of IT.

  15. Mike Bruce

    And let company bills not get paid? And maybe contracts get lost? And let company employees who are living too close to the wire not make rent that month, or child support, or their insurance payments, or God knows what else?

    Trying to prevent this by securing a PC is insane. If one dude installing bad software on one computer can cause these kinds of problems, you are already fucked.

    As far as letting Bob in accounting fuck up his machine, that’s a non-starter as his fuck up might very well bring the whole network down.

    Build a better network.

    Add in the requirement that we’re dealing with software and data that are covered by ITAR

    Regulatory stuff is, of course, irrelevant to a discussion of general IT principles.

  16. Jamie

    We are building a better network. It involves not letting random people install random bits of non business software on their workstations, since we don’t really give a shit how much you love Bonzi Buddy.

    And if you think regulatory stuff is irrelevant, I’d like to introduce you to HIPAA and Sarb-Ox. Both of which are DIRECTLY applicable to a corporation’s electronic systems. ITAR is also directly relevant, whether you’d like to admit it or not, for those of us in that industry.

  17. Mike Bruce

    If my users cannot use that portal, vast amounts of money are lost. It cannot be waved away. This is the stuff we are forced to think about in the real world of IT.

    Non-IE browsers can coexist with IE6 just fine, though.

    If you’re going to need to support it indefinitely, some kind of strategy for either having it coexist with newer IEs or being run from a terminal services server, or something along those lines, is not an unreasonable thing, either. Does it even run on newer versions of Windows?

  18. Mike Bruce

    ITAR is also directly relevant, whether you’d like to admit it or not, for those of us in that industry.

    I’ve never worked in your industry, and probably never will. So it’s certainly irrelevant to me and the millions of other people who don’t work in your industry.

    We are building a better network. It involves not letting random people install random bits of non business software on their workstations, since we don’t really give a shit how much you love Bonzi Buddy.

    That’s not building a better network; your network still sucks if one bad node can bring it down. It’s also off-point; people often want to do things or install things that are directly relevant to the work they’re doing.

  19. Jamie

    In a perfect world of zero latency infinite bandwidth networks with operating systems that have no flaws, you’d be correct. If a user needs something specific to their job, all they have to do is ask. You’d be amazed how well that works so long as what you’re asking for is actually relevant to getting your job done. Purchasing won’t sign off on it unless we can show them that though. They’re funny like that.

  20. mhoye

    “Build a better network.”

    Above, where I write “…people with no idea what the implications of their demands might be, and no real interest in finding out”, that is now officially you.

  21. Mike Bruce

    In a perfect world of zero latency infinite bandwidth networks with operating systems that have no flaws, you’d be correct.

    You don’t need to assume anything crazy. There are all kinds of strategies for partitioning networks, identifying and disabling bad ports, etc.

    Yeah, it takes time and money to do some of those things, but it takes time and money to keep everything locked down. And you get more benefits from building a better network than from encasing every individual computer in metaphorical bubble wrap.

  22. Jamie

    It’s not just the end stations. You might want to read through the CERT listings for Juniper and Cisco. Those critical bits of network infrastructure ARE being regularly targeted. It’s easier and cheaper to lock down the end stations, which are the normal and easy vector for malicious software to take advantage of. And which may or may not do a damn thing to the workstation it’s actually running on but quietly use it as starting point. This is not academic, it’s being done.

  23. Mike Bruce

    So you’re saying it’s easier to lock down a very large number of systems that users have direct access to and run a variety of software on, than it is to secure a small number of systems that no non-administrator has direct access to?

    Even if it’s true, it doesn’t seem all that persuasive; a fragile, insecure network is still fragile and insecure even if you can prevent 99% of attacks from getting into it.

  24. Jamie

    If we don’t lock down those systems, we get SQL Slammer all over again. No thanks. Our network is no more or less fragile than the one carrying the bits between Hoye’s blog here and our respective hosts. You can’t always switch out the version of JuneOS or IOS on your routers, switches, firewalls, access concentrators, etc, because where one bug got fixed, something else got broken that we need working. There’s a reason why even critical fixes to just about any piece of critical gear, be it server, switch, router, proxy, or firewall, gets extensively tested before deployment. The other option is worse.

  25. Mike Kozlowski

    Bruce@17: Non-IE browsers can co-exist, but you can’t have multiple versions of IE on the computer at the same time. Which means that you can’t let people upgrade to IE7/8 either accidentally or on-purpose. Which means you need to turn off Windows Update on people’s computer. Which means you need to have a process for pushing patches to their computers, because leaving them completely unpatched isn’t possible. Which means you need to know that your patch pushes aren’t going to break things. Which means you need to have controlled configurations.

    As soon as you pull one piece of the puzzle out, the rest kind of slips into place.

  26. Mike Bruce

    Our network is no more or less fragile than the one carrying the bits between Hoye’s blog here and our respective hosts

    And the internet seems to hum along just fine.

    Attacks are going to happen, systems are going to be compromised. You need to be able to deal with that. I’m certain, in fact, that you’re competent and talented and can deal with that, which is what I’ve obliquely been trying to say. A lone idiot almost certainly can’t destroy your network, or Hoye’s network. It can cause you a pile of work and inconvenience, sure.

    So obviously what you want to do is take steps to minimize that work and inconvenience by reducing the incidence of problems, the impact of any individual problem, and the cost of cleaning up. What I’m saying to you is that you can approach each of those things in a lot of different ways, and the strategy of trying to regulate the activity of employees through technology is a relatively poor one.

    There are external factors at play in that, obviously. I don’t think (and I hope I haven’t given this impression) that IT can unilaterally change the way things work. As long as the IT department is going to be blamed when things break, even if the problem was really some moron downloading some horrible piece of malware, you gotta do what you gotta do.

  27. Jamie

    Kozlick: You don’t have to turn off Windows Update, but you DO have to run a WSUS server and use GPOs to force the clients to use it and only it, and then set the IE7/8 installs and not allowed. This is fun because the DoD authenticated systems require TLS 1.0, but IE6 breaks it when going through a proxy (IE7/8 work fine), but our internal systems don’t work properly with IE7/8. It gets even more fun now that mother RTN is deploying a locked down Vista Enterprise image, which of course only has IE7 on it. Left hand, meet right hand. Now fucking talk to each other.

  28. Mike Bruce

    you can’t have multiple versions of IE on the computer at the same time

    This is true enough for most purposes, though not entirely true. You can have multiple versions, but it’s a weird hack and some things don’t work right.

    It’s clear to everyone that running with XP and IE6 forever isn’t tenable, though, right?

  29. mhoye

    “What I’m saying to you is that you can approach each of those things in a lot of different ways, and the strategy of trying to regulate the activity of employees through technology is a relatively poor one.”

    If I had a dollar for every time I’d heard somebody say “programming is hard, and I’m good at it, so everyone else’s job must be easy” I would already have retired. “I work with complex systems all the time, and I’ve been successful, so everything else in the world is just a complex system, right? And if the world was just like the mental model of the world that I’ve built up in my head, it would work great.”

    Sadly, as always, this says way more about the person saying it than it does about the rest of the world.

    Again: I don’t think you get to both admit that you’ve never done this sort of work and never will, while simultaneously opining on exactly what the best way to do things is. And pushing back hard when people try to outline the problems. You, right now, are precisely who I mean when I am talking about people who both don’t know and agressively don’t want to learn.

    It’s a different world over here, with different constraints and limitations. Things can take a lot more time, cost a lot more money, and not everyone gets everything they want.

  30. Mike Kozlowski

    Bruce@28: Sure, it’s clear that’s not tenable forever. But it’s ALSO clear that it needs to be IT that tells everyone when it is tenable and manages the transition process.

    Hoye@29: Man, casting Bruce as Paul Graham is a low blow. But anyway, while I have tons of sympathy for the “there are reasons we do things this way” defense, you have to admit that many, if not most, central IT departments are hellish nightmares to deal with, right? That people absolutely despise them for reasons that are not irrational or ill-founded?

    So one of three things is true: 1) Your place is great and has no such conflicts, and those other places are just full of fuckups who do stupid things for bad reasons. 2) There are fundamental problems in the traditional conception of IT and its role in business needs some radical rethinking. 3) We live in the best of all possible worlds, and such friction as exists between IT and real people is just inevitable.

    I suspect there’s an element of 1, a soupcon of 3, and more than just a bit of 2 in play. You’re focusing mostly on the 3, and Bruce is focusing mostly on the 2, and while I can’t quite agree with either of you, neither do I find either of you insane and wrong.

    (PS I have been a network administrator, so am immune to your Grahamist accusations.)

  31. Mike Bruce

    I don’t think you get to both admit that you’ve never done this sort of work and never will

    When did I say that? I was an administrator (in large and small environments) for years. I’d probably be a lot less hard on the whole IT paradigm if I hadn’t been an administrator.

  32. mhoye

    When did I say that?

    I assumed that when you said “I’ve never worked in your industry, and probably never will”, that’s what you meant.

    But you were an admin for years, and your answer is “make a better network”? Wild.

  33. Mike Bruce

    By “your industry” I mean Jamie’s industry, not IT.

    But you were an admin for years, and your answer is “make a better network”? Wild.

    Yeah? Specifics are awfully specific, you know? Find weaknesses and fix them. That’s the easy, fun part of IT when you have a mandate to go and do it.

  34. kris

    supergluing USB ports (or admins software disabling them) is a far simpler task than attempting to monitor and disable every packet passing through your firewall to make sure bob in accounting didn’t bring in that game from home to play at lunch that secretly emails the contents of his documents directory to Russia. Yes, the firewall is a single point of configuration for you, but that means you’re up to your elbows every day figuring out if that traffic on port 8080 is supposed to be there or not. And since you have abrogated responsibility for knowing what’s on client machines you cannot know what is legitimate traffic.

    ask a game designer who’s worked on an all-in-one-box like a playstation vs one who’s writing games for windows with all their infinite variety of hardware.

    Having a stable, reliable, known environment makes everything easier – and as soon as you say “bob, do what you like with the software on your machine.” you’re giving up on a sane reliable environment and moving into a world where every machine is hand tailored and requires special instructions.

    and that’s assuming that Bob’s savvy enough to recognize that his computer even *could* be vulnerable, or how to be less so.

    Thing is, a lot of requests come through that IT can quickly say “no” to because they understand the underpinnings far better than random users.

    My network monkey can wander around the office upgrading every instance of IE6 in the place ( assuming : approval, available time, etc) in an afternoon, but it might take bob 45 minutes to find the download location, install the new IE beta and then start panicking when his browser’s unrecognizable and sites all look different. Then you have to fish him out of the deep end and do the bloody thing over again anyway. Hopefully before something unrecoverable happens.

    “don’t care” and “build a better network” are both things that I, as an IT manager, would be terribly unhappy to hear from anyone on my team. IT is a service industry. People who need our services, internally or externally are our consumers, and you just gave them the finger and told them to jump off a bridge.

  35. Mike Kozlowski

    “Having a stable, reliable, known environment makes everything easier”

    … for IT. Having an environment where you can install the tools you want to use to do your job in the best way possible makes things easier for people. There is a real, fundamental tension here.

    (And hopefully you don’t really need anyone walking around to upgrade IE.)

  36. kris

    … for IT. Having an environment where you can install the tools you want to use to do your job in the best way possible makes things easier for people. There is a real, fundamental tension here.

    — in my opinion this doesn’t take into account that a lot of business users don’t know what they need beyond being able to open word docs. Installing “tools you want to use to use to do your best at your job” only works if
    a) you know what tools you want
    b) those tools will play nice with everything else on the board already.
    c) you know enough to reliably gauge a & b

    c is very often missing.

    “I’ll just upgrade this copy of office” and suddenly you’re sending out xlsx files instead of the xls that everyone in the company uses, and no one can open your files.

    and of course that failure is IT’s fault for saying “nope, I won’t upgrade just you so you can use some new feature of office 2007″

    The reason that we need IT in the first place is because the general user does not or can not understand the technology that they require to do their job. How many people in IT have had to set up a blackberry for their boss? Same concept. Then you balloon outwards and see that the CEO prefers palm so he’s got a pre and the CMO likes the slick iPhone look and you’re talking about supporting three platforms. Multiply that by everyone in the company from James the guy who’s about to retire and never caught on to the whole “cd-rom is not a cupholder” concept to Jenny down the hall who clicks on every “Paypal wants to give you 150 thousand dollars!” and you’re playing catch up forever.

    If I went to my doctor and said that I wasn’t in any pain but I’d like to try some vicodin ’cause i think it’d help my recurring headaches, I’d hope he’d say no… I wouldn’t try to substitute my knowledge for that of a doctor and just start popping pills. Saying that bob in accounting should take care of his own PC when said machine is essentially a black hole of mystery to him is pretty much the same as my doctor tossing me a bag full of narcotics and telling me to go to town. And it’d be the ER doc who had to pump my stomach that would really suffer. In a closed environment like IT in a company there’s not usually an ER Doc analog on call, so the guy who takes the requests is also going to be the one cleaning up after mistakes, can you blame them for being cautious and wanting to have clear optics into whatever problem comes up?

    If someone has a tool that they need to do their job, they should be able to make a reasonable case for it and policy should be changed to follow suit – ad-hockery just means I’ll have to wade through a generous portion of malware, spyware, crufty machines with old drivers and screwy system settings before I can even think about diagnosing an issue.

    and that’s just supporting userland, not even counting things like infrastructure that non IT people don’t understand.

    an example :
    a regional manager decided that he wanted to rearrange the office and he didn’t want to wait for IT to look at it. It was (after all) just moving some desks around. Then the network cables weren’t long enough, so the manager ordered some and put em in place, so easy!
    And this is how I ended up with an office in 2009 that’s cabled with CAT-3 running between gig-e switches.

    And you better believe that network runs like greased lightning, right? Yep. good old bob should totally take on more IT work – his work day in and day out as an accountant working with excel totally prepares him to maintain his own computer and work space. I’ll be sitting over there with my head in my hands wondering how the hell the CEO managed to brick his laptop trying to install “Catz & Dogz” for his kid to play with.

  37. Mike Bruce

    a lot of business users don’t know what they need

    A lot of people have brown hair.

  38. Mike

    Non-IE browsers can coexist with IE6 just fine, though.

    Sure they can. Of course, we still end up hearing complaints, because the complaint then becomes “Why can’t you guys let me use $browser with that website/portal/app.”

    Having an environment where you can install the tools you want to use to do your job in the best way possible makes things easier for people.

    To quote Hoye, “If I had a dollar for every time….” That arguement is how we ended up having a major app down for two days, and tied up two other admins as they worked with vendor support to figure out that the problem was a botched, uncommunicated change that a user had made in the production environment. Of course, in the spherical, frictionless universe that we live in this was not ignored and unremarked-upon.

    That’s in addition to the user that spec-ed out their own laptop and just needed us to place the order, to just needing us to just installing the user’s preferred Linux distro (that none of the apps we have in-house are built to support) to just put a standard Linux build on this hardware that doesn’t acutally support Linux (per the vendor’s website).

  39. Jim Millen

    Others have made the technical case far better than I could – what I don’t think gets emphasised anywhere near enough are the cultural and financial limits on IT in the workplace.

    Sure, we could build a network that self-monitors, cuts off virus infected end-points, handles any amount of load from YouTube videos etc. Technologically, it’s entirely possible – but for any number of users, it’s going to be a multi-million dollar project.

    Good luck convincing a CEO that’s it’s a useful way to spend megabucks, just so Bob in accounts can use whatever software he wants.

    Nearly all IT people I’ve ever worked with would like nothing better than to have the latest and greatest services & systems available for their users. It’s the paying for it that’s the problem – especially as most top executives see IT as a necessary evil, the cost of which must be cut to the bone.

  40. kris

    “a lot of business users don’t know what they need”

    “A lot of people have brown hair.”

    a lot of people eat at mcdonalds. a lot of people live in europe. a lot of people commit suicide every year.

    Oh, sorry, I thought we were just rambling off useless bits of trivia.