Automation is not helping ship reliable Software quicker

Microsoft announced that their long overdue Vista will be delayed even further.

This annoyed a number of Microsoft employees, some who don’t work with Windows, some who don’t even work with Microsoft anymore (ex). But seriously, the effects are far-reaching. I am sure there was considerable thought put inot the decision.
Anyhow, one voice piped in that automation of tasks, including Quality Assurance wasn’t helping: (from Mini-MSFT)

In the last 18 months this org:

1) Cut the number of testers (several times) from approx 50 to now much less than a dozen. Of
course, many top performers also left MS entirely because of middle mgmt in this org.

2) Hired more PMs
3) Cut the scope of testing (anyone done any real code coverage testing lately?)

4) Cut the number of promotions in the test orgs – nothing like a little ‘de-incentivization’ to increase ‘bad attrition’

5) Dictate that everything can and should be automated.
(Ignore that eyeballs catch more in less time…) way to go Darren. Of
course, you
were probably lied to by your underlings, so it’s not entirely your
fault. Uhh, yes it is – you made the call.

6) Hire only a small handful of devs to write automation
code. Oh, and don’t forget to swamp them with added process and have
embittered leads review their code…

7) Hire more PMs

8) Outsource all testing to non-accountable and barely trained CSG firms
overseas (Ever try to translate/clarify a bug written not by a tester, but by their lead based on notes? )

9)
Limit the number of heads
the abovementioned overseas firms can use. > Fewer testers, less
experienced, with little training, a much (ahem) ‘slower’ approach to
testing. Results: Client appcompat % hovering at <40% (GASP –
INTERNAL INFO… better moderate this one out!!!!)

Here’s
an anomaly for PM’s to ‘splain away. If automation is such a great
tool, why is it not finding more bugs than a small handful of testers
in a lab on the other side of the planet?

Very Interesting observation. The largest software projects are
traditionally proving grounds for automation techniques. I wish the
authors of Windows Vista come out with a sequel to The Mythical Man-Month.

Updated: San Jose’s Mercury News also cites the same book and the Vista delay but in an altogether different context.

The challenge of big software projects was probably best
described by Frederick P. Brooks Jr. in his classic 1975 book, “The
Mythical Man-Month.” Brooks, a professor of computer science at the
University of North Carolina at Chapel Hill, stated then what he called
Brooks’ Law: “Adding manpower to a late software project makes it
later.” The need for latecomers to get up to speed and communicate
with their
predecessors, Brooks says, takes up more of the team’s time than what the added workers contribute.It’s even theoretically possible to reach a kind of software
gridlock, where the team is so big that all its time goes to
communicating among
each other and making revisions — with the project never reaching
completion.