Unit Testing: A Very Short Parable

One day a key employee, a very respected senior programmer, was called into his boss’s office. For one-half hour he sat and endured a lecture by his boss about his inability to get work done on time and within budget. It made him wonder what the boss’s problem was, because the programs always worked . . . eventually. Wasn’t the objective to create working programs?

Message: Managers always complain about something.

He then suggested to the programmer that if a little effort were put into developing a unit test plan, the quality of the work would definitely improve. He also was quite confident that the work would be completed much faster, maybe even on-time as originally planned. The manager was very confident that his assertions were correct, and furthermore, writing and executing the unit test plan were already identified as tasks in the project plan.

Message: They’re still talking about unit test plans,
but I’m a programmer not a tester.

Even if it wasn’t the first time this topic was broached, the programmer was bewildered by the conversation. Project plans were typically shown to him at the onset of major application work, but he pretty much never encountered them again. Additionally, schedules or budgets for maintenance efforts didn’t exist. These were the unplanned-for "fixes" and "modifications" you were expected to work on in your "spare time." Even when project plans were created, they never contained budgetary information. Project managers who created budgets were probably the ones most concerned with the expectation that he stay within them. He seriously wondered why the budget should be his problem?

In the past, programmers in the shop were judged by whether or not the program worked, not by how much it cost, or how long it took. Budgetary concerns would only add another obstacle to getting his work done on time. Thinking about it, he realized that he had never before been criticized for late delivery of programs. (He thought about all of the other programmers in the department. Did they all have this problem?) In fact, the testers always appeared to be happy when he delivered his work. Though on the last project he did hear them say something like "At last." But the testers always have unusual expectations anyway.

Message: If the program doesn’t work, early delivery makes no sense. And it is not my fault that they are always making changes to the spec.

So the programmer didn’t worry about the lecture; he noted that managers seemed to forget about unit test plans almost as quickly as they made the recommendation to write them. It was not totally clear why they wanted the unit test plan anyway. On the surface it made so little sense. You write a program, then you write a test plan to demonstrate that the program works. Doesn’t management understand that you can execute four or five tests in the time it takes you to write one in a test plan? Why do they want to waste time with so much testing? Doesn’t a quality development approach mean that we should do it right the first time and not have to test later to see if it was done correctly?

Conclusion: Discussions about unit test plans and unit testing do more to slow down a project than any late changes made to the spec.

A few days later . . .

Over a long holiday weekend, the programmer reconsidered his attitude towards the lecture he received a few days earlier and concluded that it was possible that there was a better way. It couldn’t hurt to examine the issue. Especially if it was not a simple issue but a potentially serious problem.

He started by reassessing his reluctance to write a unit test plan by building a table of reasons for and against the practice. He built the table quickly without giving thought to its order– just recording whatever came into his mind. After five entries for both the Pro and Con columns, it seemed appropriate to stop and appraise the content.

After reviewing the table, he noticed that at least one very significant item was omitted from the table. Ostensibly, the purpose of the unit test plan is to demonstrate that the program works properly and is mostly bug free. How could something this obvious not be the first item on the pro side of the table? Obviously, some of the more important opportunities presented by a test plan were being overlooked.

Message: The biggest problem programmers face is miscommunication. The bugs aren’t what slows down the project, it is building the wrong product and not understanding what the client wants.

Arguments against writing the unit test plan could not all be easily dismissed. He examined each reason to see if it was truly legitimate or if there was an acceptable compromise. It takes too long seemed a reasonable complaint. Project time is limited and everyone is typically under deadline pressure. But maybe there was a fair test to see if writing the unit test plan actually lengthened the time to delivery. Concluding that he didn’t know the answer, the compromise was to try programming the next couple of programs after writing a unit test plan. If the manager was estimating the how long the program should take in the same way as in the past, then finishing ahead of time would demonstrate a value in writing the unit test plan before coding.

The specs are never complete, seemed an insurmountable problem. However, he realized that while specs may not be complete at the onset, they are malleable and frequently change multiple times before a program is delivered. (If his recollection was accurate, the specs always changed at least once before the program was delivered.) He then reasoned that the biggest problem with spec changes is that they occur so late in the process. If a necessary change became apparent earlier in the development life cycle, it would be far more manageable. Here again, a unit test plan might be valuable. It forces many of the "to be determined" issues to be confronted and dealt with before coding. The test plan would also be instrumental in identifying loose ends and missing functionality.

There doesn’t seem to be adequate time to write the plan is identical to it takes too long. The experiment would resolve this issue.

Nobody else is writing them, could potentially put you at a disadvantage if your work takes longer than everyone else in the shop. But then, no one seems to be delivering work on schedule, and project managers are forever making adjustments to the plan. It may be that everyone is behind schedule because they don’t take the time to plan properly before beginning their work. Maybe the written unit test plan is the solution.

I don’t know what to include in a unit test plan is a serious concern. A short investigation confirmed that there were no sample unit test plans in the shop, so the next step was to ask the system testers for assistance. After the initial shock of a programmer coming to them with unit testing questions, they agreed to help write a unit test plan.

Then the pro side . . .

This part of the analysis would be more difficult. Life would be easier without the need to write unit test plans. Maybe not easier, but certainly it was a known path.

Completing his evaluation of the CON side, he turned to the PRO side. His number one reason for developing a unit test plan was to confirm an understanding of the spec before investing time in coding. Actual testing would be a secondary consequence of having the written plan. The defects in the program were never of great concern because they were usually corrected with simple changes to the code. Real problems that required design changes or additional functionality seemed to always be the result of miscommunication. Apparently there was some potential to clarify what needed to be done before the massive time investment in coding.

A reviewable document actually fit quite well with the idea of confirming understanding. The document could be the basis of a review allowing others, but primarily system testers, to get a feeling for the programmer’s understanding. It would also provide insights to organization and design.

The third item on the pro side went beyond unit testing and involved the system tester as well. Communicates to system testers what has already been tested. Reduces redundant testing. It was now becoming apparent that system testers couldn’t really begin to test the system’s functionality until they were confident that the system components actually worked properly. This was a rather devastating thought because it had such far reaching consequences. System testers repeated most of his tests because they had no idea what he was testing, or for that matter not testing. Was it possible that functional problems were found in production versions of the system because the system testers just never got to those tests due to scheduling constraints? Even worse, was the possibility that system testers knew that they were not going to get results from the programmers and actually never planned to test functions, resigning themselves to testing only at the component level?

Someone else could help me test if I’m pressed to meet a deadline. There was an interesting idea. In his years of experience as a programmer, he couldn’t recall anyone ever assisting programmers with testing. Now he wondered why? Maybe it was the absence of written unit test plans that made it difficult to help a programmer who was having difficulty making a completion date.

The last element in the table, helps to organize my thoughts before starting to code, made even more sense to him at this stage of analysis. He asked himself how often he had to rework a section of the program because of a forgotten feature. Sometimes the rework occurred because of a good idea that arose in the middle of coding. Whether he forgot something or got a better idea, he still had to go back and rework the program.

Message: The PROS significantly outweigh the CONS.

Now what . . .

Building the table was eye opening and instrumental in changing the programmer’s perspective. Originally, his belief was that the business objective was to write working programs. And while it was certainly true that the program had to work, the business objectives were probably far more complex. He thought that next week he might assess the business objectives.

Meanwhile, he had educated himself in the area of testing. Basic objections to writing a unit test plan paled in comparison to the potential benefits of writing a test plan before coding began. The next logical step would be to outline the lessons learned and prepare to make a transition from the old way of programming to the new way.

• A unit test plan can not be written without specifications. If the specs are inadequate, they will be improved (rewritten, reviewed, etc.) before coding.
• Before coding, every effort will be made to confirm an understanding of the spec. This can be done through reviews of the spec and by writing a unit test plan.
• The unit test plan can be written by the programmer or it can be written by system testers for the programmer.
• Reporting the results of unit testing to the system testers will enable them to eliminate redundancy in testing. This approach should enable the system testers to concentrate on testing functionality.
• Doing the job correctly the first time will undoubtedly save time and money.
• The plan will probably be used at least twice. Any transition involves learning the new way and that can be time consuming. It might not be possible to see benefits until the second or third attempt.

Epilogue...

It took three attempts until the programmer was satisfied with the approach and the quality of the unit test plans. His programs were being completed on-time and rarely did the system testers bring functional issues to him for fixing. On the other hand there were a few customer related problems. They were being forced to take an active role in defining what they wanted and were unable to just wait to see what was produced.

Another bright spot was the number of other programmers that decided to buy into the success of the approach.