Practical Software Engineering
SQA is something which should be applied throughout the development
Quality is often measured in thousands of lines of code or mean time to
make a change.
Roger Pressman quotes Philip Crosby:
The problem of quality management is not what people don't know about it. The
problem is what they think they do know....In this regard, quality has much in
common with sex. Everybody is for it. (Under certain conditions, of course.)
Everybody feels they understand it. (Even though they wouldn't want to explain
it.) Everybody thinks execution is only as matter of following natural
inclinations. (After all, we do get along somehow.) And, of course, most
people feel that problems in these areas are caused by other people. (If only
they would take the time to do things right.)
can refer to deterministic events (e.g. motor burns out) when it cannot be
predicted when they will occur; or random events.
- issues that related to the design of the product which will operate well
for a substantial length of time.
- a metric which is the probability of operational success of
The probability space -- the space of all possible occurrences must first be
defined, e.g. in a probability model for program error it is all possible paths
in a program. Then the rules for selection are specified, e.g. for each path,
combinations of initial conditions and input values. A software failure occurs
when an execution sequence containing an error is processed.
is the application of probability theory to the modeling of failures and the
prediction of success probability.
A definition commonly accepted is:
Software reliability is the probability that the program performs
successfully, according to specifications, for a given time period.
Specifications -- precise statements of:
Errors are found from a system failure,
and may be: hardware, software, operator, or unresolved.
- the host machine
- the operating system and support software
- the operating environment
- the definition of success
- details of hardware interfaces with the machine
- details of ranges and rates of I/O data
- the operational procedures.
Time may be divided into:
Different variables from time may need to be
- operating time,
- calendar time during operation,
- calendar time during development,
- man-hours of coding,
- development, testing,
- computer test times.
repairable if it can be debugged and the errors corrected. This may not be
possible without inconveniencing the user, e.g. air-traffic control system.
- load -- on a timesharing system
- input cycles -- if data only arives occasionally.
Software availability is the probability that the program is performing
successfully, according to specifications, at a given point in time.
Availability is defined as:
These measurements are used to:
- the ratio of systems up at some instant to the size of the population
studied (no. of systems).
- the ratio of observed uptime to the sum of the uptime and downtime:
A = Tup / (Tup + Tdown) (of a single system)
If the system is still in the design and development phase then a
third definition is used:
- quantify A and compare with other systems or goals.
- track A over time to see if it increases as errors are found.
- plan for repair personnel, facilities (e.g. test time) and alternative
Various hypotheses exist about program errors, and seem to be true, but no
controlled tests have been run to prove or disprove them:
- the ratio of the mean time to failure (uptimes) and the sum of the mean
time to failure and the mean time to repair (downtime):
A = MTTF /(MTTF + MTTR)
- Bugs per line constant. There are less errors per line in a high level
language. Many types of errors in machine code do not exist in HOL.
- Memory shortage encourages bugs. Mainly due to programming "tricks" used
to squeeze code.
- Heavy load causes errors to occur. Very difficult to document and test
- Tuning reduces error occurrences rate. This involves removing errors for
a class of input data. If new inputs are needed, new errors could occur, and
the system (hardware and software) must be retuned.
Many researchers have
put forward models of reliability based on measures of the hardware, the
software, and the operator; and used them for prediction, comparative analysis,
and development control. Error reliability and availability models provide a
quantitative measure of the goodness of the software. There are still many
This is still in its development phase.
- The normalized number of errors is constant. Normalization is the total
number of errors divided by the number of machine language instruction.
- The normalized error-removal rate is constant. These two hypotheses apply
over similar programs.
- Bug characteristics remain unchanged as debugging proceeds. Those found
in the first few weeks are representative of the total bug population.
- Independent debugging results in similar programs. When two independent
debuggers work on a large program, the evolution of the program is such that
the differences between their versions is negligible.
Boehm, Brown and Lipow identify key issues, and say measures should show
where a program is deficient. Managers must decide on the relative importance
They define a hierarchical software characteristic tree,
the arrow indicates logical implication. The lowest level characteristics are
combined into medium level characteristics. The lowest level are recommended
as quantitative metrics. They define each one. Then they evaluated each by
their correlation with program quality, potential benefits in terms of insights
and decision inputs for the developer and user, quantifiability, feasibility of
automating evaluation. The list is more useful as a check to programmers
rather than a guide to program construction.
- on-time delivery
- efficient use of resources such as:
- processing units
- peripheral devices
- maintainable code issues such as:
Gilb also devised a set of software metrics:
and many more. He defines each in detail. The
reality of applying these measures is disheartening. Many are difficult to
obtain, and no expected range is given. They are not all independent.
- reliability -- the probability that a given program operates for a
certain time without a logical error
1 - (inputs causing execution failures
/ total inputs)
- maintainability -- the probability that a failed system will be
restored to operable condition within a specified time.
- repairability -- as maintainability but all resources assumed to be
- flexibility -- with many sub-categories
However, this is still a developing field and he has pioneered some software
Halstead used 'methods and principles of classical experimental
science'. He counted:
He defined vocabulary h as h1+ h2 and implementation length N as N1 +
N2. From these he devised equations for: length, volume, potential volume,
boundary volume, program level, intelligence, programming effort...
- number of unique operators (IF, DO, = , PRINT) h1;
- number of unique operands (variables or constants) h2;
- total usage of the operators N1;
- total usage of the operands N2;
- number of times each operator occurred f1,j (j=1..h1);
- number of times each operand occurred f2,j (j=1..h2).
N1 = h1 log2 h1 + h2 log2 h2.
His length equation was tested on 14 algorithms and found to be very close to
actual length. Other experimental evidence is also convincing. However, it
ignores the issues of variable names, comments, choice of algorithms or data
structures. It also ignores the general issues of portability, flexibility,
Zak lists five productivity attributes:
From a survey of managers and technicians:
- ability to achieve schedules
- freedom from bugs
In an experiment, five programming
teams were given a different objective each:
- quality of external documentation
- programming language
- availability of tools
- programmer experience in data processing
- programmer experience in the functional area
- effect of project communication
- independent modules for individual assignment
- well-defined programming practices
When productivity was evaluated each team ranked first
in its primary objective. This shows that programmers respond to a goal.
This is the main programming costs in most installations, and is affected by
data structures, logical structure, documentation, diagnostic tools, and by
personnel attributes such as specialization, experience, training,
- minimum internal memory
- output clarity
- program clarity
- minimum source statements
- minimum hours
Methods for improving maintainability are:
Bugs are sometimes seeded to
establish a maintainability measure. For example, a program has 100 seeded
bugs. During debugging 550 bugs are found, 50 of which were seeded. It can
then be estimated that 500 real bugs remain.
- automated audits of comments
- test path analysis programs
- use of pseudocode documentation
- dual maintenance of source code
- structured program logic flow.
Software Maintenance has very high cost. Gansler (1976) quotes Air Force
avionics software at $75/instruction to develop, and $4000/instruction to
Maintenance includes the cost of rewriting, testing, debugging and integrating
Documentation is one of the items which is said to lead to high
maintenance costs. It is not just the program listing with comments. A
program librarian must be responsible for the system documentation, but
programmers are responsible for the technical writing.
Other aids may be text editors, and Source Code Control System (SCCS) tool for
producing records. Some companies insist that programmers dictate any test or
changes onto a tape every day.
For hardware, this covers inspection and test of materials, maintenance of
standards for workmanship, calibration of equipment, acceptance testing.
|Rank Problem area
- User demands for enhancements, extensions
- Quality of system documentation
- Competing demands on maintenance personnel time
- Quality of original programs
- Meeting scheduled commitments
- Lack of user understanding of system
- Availability of maintenance program personnel
- Adequacy of system design specifications
- Turnover of maintenance personnel
- Unrealistic user expectations
- Processing time of system
- Forecasting personnel requirements
- Skills of maintenance personnel
- Changes to hardware and software
- Budgetary pressures
- Adherence to programming standards in maintenance
- Data integrity
- Motivation of maintenance personnel
- Application failures
- Maintenance programming productivity
- Hardware and software reliability
- Storage requirements
- Management support of system
- Lack of user interest in system
|(Lients et al. (1976), Table V.)
For software, there is no prototyping (except as phase one of a two-phase
design), no incoming parts to be inspected, no standards for measuring software
Rules to follow in software contracting:
- Get legal advice from the beginning.
- Negotiate with a senior person.
- Negotiate with only one person.
- Document all verbal agreements.
- Make sure the contract specifies everything you will get: the prices, the
terms, the conditions.
- Do not announce the final decision until the contract is signed.
- Remember that no matter what the contract says, success with software
depends first of all on a good business relationship between buyer and
Source: Heninger (1979, p. 3). Three additional sections are suggested for
this outline: 1(a), "Software Characteristics"; 2(a), "Software Interfaces";
and 6(a), "Defensive Programming Techniques."
- 1. Introduction
- Organization principles; astracts for other sections, notation guide
- 2. Computer characteristics
- If the computer is perdetermined a general description with particular
attention to its idiosyncrasies; otherwise a summary of its required
- 3. Hardware interfaces
- Concise description of information recieved or transmitted by the computer.
- 4. Software functions
- What the software must do to meet its requirements, in various situations
and in reponse to various events.
- 5. Timing contraints
- How often and how fast each function must be performed: This section
is separate for section 4 since "what" and "when" can change independently.
- 6. Accuracy constraints
- How close ouput values must be ideal to values to be acceptable.
- 7. Response to undesirable events
- What the software must do if sensors go down, the pilot keys in invalid data, etc.
- 8. Subsets
- What the proram should do if it cannot do everything.
- 9. Fundamental assumptions
- The characteristics of the program that will stay the same, not matter
what changes are made.
- 10. Changes
- The types of changes that have been made or expected.
- 11. Glossary
- Most documentation is fraught with acronyms and technical terms.
- 12. Sources
- Annotated list of documentation and personnel, indicating the types of
questions each can answer.
It is desirable to add sections:
- 2A. Software Characteristics
- which includes design philosophy, language,
algorithms, data structures.
- 3A. Software Interfaces
- which should discuss decisions on: existing
operating systems, compilers, interpreters, assemblers, existing software
development tools, existing code modules, subroutines or data bases.
- 7A. Defensive Programming Techniques
- which includes expected range of
input variables, key intermediate variables, output variables, range checking,
parallel computation and checking, rollback, error-recovery techniques.
Formal Technical Review Meetings
Formal Technical Reviews (FTR) are conducted during design and coding phases.
Calculations of cost savings by a defect amplification model:
|Errors Found ||Number ||Cost Unit ||Total|
|During design ||22 ||1.5 ||33|
|Before test ||36 ||6.5 ||234|
|During test ||15 ||15 ||315|
|After release ||3 ||67 ||201|
| || || ||783|
|No Reviews Conducted|
|Before test ||22 ||6.5 ||143|
|During test ||82 ||15 ||1238|
|After release ||12 ||67 ||804|
| || || ||2177|
|(Source: Roger Pressman. Software Engineering: A Practitioner's Approach. 1997. Table 8.1, pp. 191)|
- Review the product, not the producer.
- Set an agenda and maintain it.
- Limit debate and rebuttal.
- Enunciate problem areas, but don't attempt to solve every problem noted.
- Take written notes.
- Limit the number of participants and insist upon advance preparation. (3-4)
- Develop a cheaklist for each work product that is likely to be reviewed.
- Allocate resources and time schedule for FTRs.
- Conduct meaningful training for all reviewers.
- Review your early reviews.
Practical Software Engineering, Department of Computer Science