Unit testing has been around for quite some time now for .NET, thanks to tools such as NUnit and TestDriven.NET. The idea of unit testing, however, still has not convinced the majority of developers to change their ways. With the release of VS2005 Team Suite (VSTS), unit testing are now on the agenda of most IT managers, and thus even the most ignorant programmers may soon have to deal with unit testing.
I have participated at several VSTS unit testing introductions at different customers the last year, and the initial response is the same everywhere: “this is nice in theory and in a small demo, but it will be too much extra work to implement unit tests that covers our code”. Add in the “how do we test databases with inserts, updates, delete?” challenge, and you know that the attendees are only human, skeptical to change and skeptical to ideas from the management.
The reason for this typical response, I believe, is due to the typical way unit testing is demoed: the math library. Write a test to add two numbers and make it pass. The assert phase of the test is always Assert.AreEqual, checking to see that the result is the sum of the two numbers. I bet almost every introduction to unit testing that you have read or seen, are in fact result testing.
People are being brainwashed into believing that unit testing is testing for exact results and nothing but testing results. Result testing is of course a very common type of unit testing, but you will soon find out that writing tests for anything but a math library is not trivial when testing for exactness of returned values/data/content is your perception of unit testing.
My point is that writing unit tests are much simpler when asserting that the returned data falls within an expected range, is not null, did not fail or failed as expected, etc. Think of unit testing more like doing calculations in your head when shopping, where a rough amount is adequate, rather than trying to get the correct answer with two decimals.
Another point on the ‘exact result testing’ mindset: when introducing unit testing, someone will always point out that is will require a lot of work to write tests that asserts every possible outcome of a real-life method. This is correct. But again, what is sufficient to start with? The best is the enemy of the good. Start with a few tests that assert the typical outcome of the method. Always add a new test when a bug is encountered, failing at first, fix the bug and make the test pass. Always add a new test when adding to a class or changing the way it works. Given enough time, this strategy will ensure that you get good enough unit testing coverage of your code.
One last typical dialog to illustrate the above point:
Developer: “The logic behind creating a new customer/account is quite complicated, and the combination of possible input to the business logic will be impossible to test, so why bother with writing extra code for unit testing?”
Me: “Ok. How do you test the biz-logic today? You do test it, do you not?”
Developer: “Of course! I have a test form with plenty of input fields, a couple of buttons, and a datagrid to show the results.”
Me: “How do you ensure that your testing covers all aspects of your biz-logic?”
Developer: “I enter some typical values, push the buttons and checks that the grid gets filled…”
Me: “So, you just check that the returned result seems to be as expected, or do you closely examine each and every value in the grid?”
Developer: “No… That is not necessary… As long as the result is not weird, I assume that the biz-logic is correct…”
Me: “Then, why do you insist on the unit tests to be ‘exact result’ tests, and that they should cover all combinations of input? Are you persistent enough to always do all your ‘push button’ tests each time you change your code? Wouldn’t an automated test regime that is at least as cautious about ‘output falls within range’ as your manual testing, be an improvement?”
At this point in the discussion, most developers admits that they are bored stiff by the test forms, and that writing unit tests instead of test forms seems to be far better.
The thing that influences most developers to start employing unit testing is that it makes refactoring their application much safer and allows them to change the architecture and design of their software with more confidence. Add in that automated unit testing also makes regression testing an application a breeze, no more multi-button test forms to fill out in a frenzy just before deadline.
Read this MSDN Mag article to learn more about VSTS unit testing (note the math library examples).
The next step on the path to becoming a true test believer is to embrace Test Driven Development (TDD). Check out Scott Bellware’s blog to learn more about TDD.
No comments:
Post a Comment