This is the big question : how much ?
Cheap people only calculate cost with money. But if you want well-tested software you’ve got to see the bigger picture.
Here is how I see it. But since it’s quite a new matter for me, I’d appreciate to debate about it and have your experience in comment.
- Your company’s goal is to make money
- To make money you must produce good useful software (to sell or to use internally)
- To produce stable tested and useful software you need a good team (who cares about tests and quality more than having a 1000 features)
- And to have a good team you need more than money
Let’s compare 3 kind of tests I hope you already practice in your company : unit tests, manual integration tests and manual regression tests.
[table id=1 /]
Alright, that’s my vision, and you can guess in favor of what I preach. But when talking to fellow devs and seeing different projects, I can tell this table is pretty accurate.
This week I spent 2 days automating every test I could think of in the service I’m developing.
I though my application was simple and ok before putting unit tests everywhere, even on the most trivial classes.
But I discovered bugs. Sometimes simple copy/paste bugs. Sometimes bigger ones.
So I started to automatize those 208 manual tests. It became clear that I saved more than 2 days.
Also I saved money by increasing my understanding of the product and my competences in mocking and JUnit.
The project leader was ok to spend 2 days improving code by refactoring and unit testing after I asked him. And I’m sure he took the right decision. Of course we’ll never know exactly but I think we saved debugging days.
The Cost Of Debugging
Sometimes debugging could seem faster than implementing unit tests. And in absolute time it could be true. But be honest : if you have many procedures and many people in the bug hunting sessions, your costs are going to crash the ceiling.
Necessary but costly heavy testing sessions go like this :
- Prepare an environment for the testers to tests
- Ensure that testers test against the right version of specifications
- When a tester find a bug (sometimes that could be avoided simply by 3 lines of JUnit) he fills a report
- The project manager gets the bug report, test again and attribute it to a developer if needed
- The developer stops everything he is doing to try to reproduce, correct and test again
- If the developer is a good one he implements immediately a unit test to avoid finding this bug manually ever again
- Back to the project manager for validation
- Back to the tester for validation
- Go on, hit some numbers in Excel and try to find out the cost of letting your people think instead of just passing some manual tests.
- Come back here and tell us where you’ll invest more in your project. We’d be delighted to debate about it here.
- Ask yourself honestly : What’s the cost of NOT doing unit tests ?
For the developers out there
What’s your opinion/experience on this ? Do you feel managers don’t allocate enough budget on unit tests ? Are they taking to many risks by only paying for manual tests ?