Dev vs Test/Ops

Dare's post 'Replacing Operations with Developers' reminded me of many discussions I had at Microsoft, especially when products were slipping and features were getting cut. A friend of mine once opinioned that developers are optimists and testers are pessimists. Developers (esp at Microsoft) tend to be smart, ambitious, and believe that they can solve any problem with clever coding. This has lead to some horrible situations, when developers believe that other parts of the team are holding them back.

Part of the problem stems from the nature of the task. Developers often a working with development plans that involve dedicating days or weeks to individual problems. Testers and Operations tend to work in terms of hours. If most of your day is spent task-switching and the rest is spent fire-fighting, you have a problem, and I have seen this happen repeatedly in Test and Ops teams, due to over-commitment or bad management. (I've seen the same happen to dev teams, but it is slightly less common.)

Another problem that sometimes arises is simple hubris. Developers like to think they can do/build anything, and so often look down on the Testers and Operations staff. This serves no-one and often exacerbates the problem.

But that isn't to say that Developers should not be actively involved in test and lab operations. Developers must understand what Testers and Operations have to deal with before they can honestly evaluate what these critical staff are really contributing. Test driven development does not mean no test staff. It means that the test staff can do really test integrations, back-compat, etc... all the complicated tests that test that all the pieces fit together like they are supposed to. One of my favorite things I saw happening at MS before I left was a move to Model-Driven-Testing. This required a lot more of the Testers, but resulted in some of the best bug reports. Even better, it got the testers writing non-trivial code and doing something which really felt rewarding. API testing is monotonous. Data-driven-tests are often worse. A good test team is one that is excited about what they are doing and finding bugs that make a Developer scratch their head. This only happens when the code that the Testers are bashing at is already solid.

All of this also depends heavily on what are your requirements. At Microsoft, despite what people may say, the targeted quality bar was very high. Any bug that gets through may mean a painful back-compat hack in the next revision. If you aren't working on a project that has such stringent requirements, then maybe TDD can mean the end of an explicit test-team. Honestly though, anything that takes more than 5 dev-years to implement (yea, I pulled that number out of my arse) probably needs a dedicated test team.

All this applies just as well to Ops. A friend of mine works for network ops for a tech firm that works with banks. He can barely write script, but I wouldn't want to replace him with a developer. He spends his days trouble-shooting network integration issues with customers. How many developers do you know who know anything about IBM mainframe networking? He spends days working with customers to troubleshoot the worst issues. No amount of tools or automation will make his job go away.

I guess one way to think about it is that Developers leverage working in a world of limited variables. To do their job, they need to constrain the inputs. Testers job is to be creative and introduce some new variables, they can expect a customer to experience. Operations live in a world of near infinite variables where half the job is just figuring out what the variables really are. The tricks that make developers so effective at writing code, do not necessarily transfer to these other disciplines.