STEP 4: PLAN, CONTROL AND MONITOR TESTING
Without question the biggest user of project resources, whether it be manpower, computer time, or management judgment, is the test phase.
It is the phase of greatest risk in terms of dollars and schedule.
It occurs at the latest point in the schedule when backup alternatives are least available, if at all.
The previous three recommendations to design the program before beginning analysis and coding, to document it completely, and to build a pilot model are all aimed at uncovering and solving problems before entering the test phase.
However, even after doing these things there is still a test phase and there are still important things to be done.
Figure 8 lists some additional aspects to testing.
In planning for testing, I would suggest the following for consideration.
- Many parts of the test process are best handled by test specialists who did not necessarily contribute to the original design. If it is argued that only the designer can perform a thorough test because only he understands the area he built, this is a sure sign of a failure to document properly. With good documentation it is feasible to use specialists in software product assurance who will, in my judgment, do a better job of testing than the designer.
- Most errors are of an obvious nature that can be easily spotted by visual inspection. Every bit of an analysis and every bit of code should be subjected to a simple visual scan by a second party who did not do the original analysis or code but who would spot things like dropped minus signs, missing factors of two, jumps to wrong addresses, etc., which are in the nature of proofreading the analysis and code. Do not use the computer to detect this kind of thing - it is too expensive.
- Test every logic path in the computer program at least once with some kind of numerical check. If I were a customer, I would not accept delivery until this procedure was completed and certified. This step will uncover the majority of coding errors. While this test procedure sounds simple, for a large, complex computer program it is relatively difficult to plow through every logic path with controlled values of input. In fact there are those who will argue that it is very nearly impossible. In spite of this I would persist in my recommendation that every logic path be subjected to at least one authentic check.
- After the simple errors (which are in the majority, and which obscure the big mistakes) are removed, then it is time to turn over the software to the test area for checkout purposes. At the proper time during the course of development and in the hands of the proper person the computer itself is the best device for checkout. Key management decisions are: when is the time and who is the person to do final checkout?
STEP 5: INVOLVE THE CUSTOMER
It is important to involve the customer in a formal way so that he has committed himself at earlier points before final delivery.
To give the contractor free rein between requirement definition and operation is inviting trouble.
Figure 9 indicates three points following requirements definition where the insight, judgment, and commitment of the customer can bolster the development effort.
Figure 10 summarizes the five steps that I feel necessary to transform a risky development process into one that will provide the desired product.
I would emphasize that each item costs some additional sum of money.
If the relatively simpler process without the five complexities described here would work successfully, then of course the additional money is not well spent.
In my experience, however, the simpler method has never worked on large software development efforts and the costs to recover far exceeded those required to finance the five-step process listed.