Practical Software Engineering

The Software Engineering Institute Standards

The Software Engineering Institute

The Software Engineering Institute at Carnegie Mellon University has set out a set of standards in the form of Software Process Maturity.

This model has five levels of process assessment, and outlines the techniques for achieving these levels.

Background

The Wealth of Nations, Adam Smith 1776. Principle that specialized workers could make far more pins in a day than workers each producing whole pins. He split the process into wire drawing, wire straightening, cutting, pointing, grinding at top to attach head, ....

This principle has been used as a model ever since, not only for manufacturing, but also applied to management.

Today, three things have changed:

customers are in charge -- they have more knowledge, more choice
competition intensifies -- suppliers find new ways of doing business
situations change all the time -- not just those we expect

Jobs as a set of tasks is no longer appropriate, a process model is needed.

Who is in charge of process? Everyone and no-one.

Is there a VP for process?

Total Quality Management (TQM)

based on the work of Dr W. Edwards Deming who is reputed to have transformed Japanese products from shoddy to first in the world.

He developed 14 points for management to transform organizations.

Create constancy of purpose toward improvement of product and service, with the aim to become competitive and to stay in business and to provide jobs.
Adopt the new philosophy. We are in a new economic age. Western management must awaken to the challenge, must learn their responsibilities, and take on leadership for change.
Cease dependence on inspection to achieve quality. Eliminate the need for inspection on a mass basis by building quality into the product in the first place.
End the practice of awarding business on the basis of price tag. Instead, minimize total cost. Move toward a single supplier for any one item, with a long-term relationship of loyalty and trust.
Improve constantly and forever the system of production and service, to improve quality and productivity and thus constantly decrease costs.
Institute training on the job.
Institute leadership. The aim of leadership should be to help people and machines and gadgets to do a better job. Leadership of management is in need of overhaul, as well as leadership of production workers.
Drive out fear, so that everyone may work effectively for the company.
Break down barriers between departments. People in research, design, sales, and production must work as a team to foresee problems in production and in use that may be encountered with the product or service.
Eliminate slogans, exhortations, and targets for the work force that ask for zero defects and new levels of productivity.
Eliminate work standards (quotas) on the factory floor. Substitute leadership. Eliminate management by objective. Eliminate management by numbers and numerical goals. Substitute leadership.
Remove barriers that rob the hourly worker of his right to pride of workmanship. The responsibility of supervisors must be changed from stressing sheer numbers to quality. Remove barriers that rob people in management and engineering of their right to pride of workmanship. This means abolishment of the annual merit rating and management by objective.
Encourage education and self-improvement for everyone.
Take action to accomplish the transformation.

System of Profound Knowledge

The purpose of the System of Profound Knowledge is to transform Western leaders so that they will:

improve and innovate the system of interdependent stakeholders of an organization over the long term to allow all people to experience joy in their work and pride in the outcome. Stakeholders include employees, customers, suppliers, stockholders, the community, and competitors.
optimize the system of interdependent stakeholders of an organization over the long term so that everybody wins.
improve and innovate the condition of society. Society includes local, regional, national, and international systems; for example, the entire educational system (public and private schools; primary and secondary schools and universities), the environment, public health, and the economic and social well-being of communities and countries.

A paradigm is a way of looking at the world. That way determines how an individual processes information and behaves.

Dr. Deming's System of Profound Knowledge is an example of a new paradigm of management. Four paradigm shifts are needed for success.

Paradigm 1

Manage to create a win-win environment, not a win-lose environment. The focus of a win-win environment is optimization of all stakeholders in the organization's system of interdependent stakeholders.

Paradigm 2

Manage to create intrinsic motivation, not extrinsic motivation. Intrinsic motivation is the individual's desire to do something for the sheer joy of it. Extrinsic motivation is the individual's desire to do something to gain rewards or to avoid punishments.

Paradigm 3

Manage with a long-term process and results orientation, not with a short-term results-only orientation. A long-term process and results orientation promotes improvement and innovation of organizational processes. Highly capable processes facilitate prediction of the future and, consequently, a higher likelihood of achieving the organizational mission.

Paradigm 4

Manage to promote cooperation, not competition. In a cooperative environment, everybody wins. The customer wins products and services he can brag about and rejects commonly accepted levels of defects, rework, shoddy workmanship, and poor service. The firm wins returns for investors and secure jobs for employees. Suppliers win long-term customers for their products. The community wins an excellent corporate citizen.

In a competitive environment, most people lose. The costs resulting from competition are unknown and unknowable, but they are huge. Competition causes individuals, or departments, to optimize their own efforts at the expense of other stakeholders. This form of optimization seriously erodes the performance of the system of interdependent stakeholders.

Total Quality Management

Point 1. Top management has to make a commitment to transformation.

Point 2. Management's education and self-improvement are both exciting and frightening.

Point 3 Daily management includes the development, standardization, control, improvement and innovation of methods used by employees in their daily routine. (PDSA cycle)

Plan--employees develop a plan to improve or innovate a "best practice" method.
Do--employees test the plan using planned experiments on the "best practice" method on a trial basis.
Study--employees examine the results of the planned experiments.
Act--employees take corrective actions which lead to a new or modified plan or a revised "best practice" method and formalize it through training.

Point 4. Cross-functional management involves weaving both horizontal and vertical threads across as well as within divisions and departments.

Point 5. Policy management is performed by using the PDSA cycle to improve and innovate the methods responsible for the difference between corporate performance and customer needs and wants.

A SWOT analysis (Strengths, Weaknesses, Opportunities & Threats) can be performed to initiate this change. This is done by:

understanding the history, development and growth of the organization
identifying the organization's internal strengths and weaknesses. These may be
- values and beliefs of employees,
- philosophy and skills of management,
- knowledge and training of employees,
- hiring, training, and supervisory processes,
- methods for improvement and innovation of processes, and
- methods for exceeding customer requirements.
identifying the organization's external opportunities and threats. These may be
- degree of cooperation within the industry,
- degree of trust and loyalty between customers and industry,
- closeness of substitute products,
- the state of quality management technology in the industry.

The SWOT analysis looks at all the data in order to understand the crises in the organization which is then synthesized by top management and communicated to independent stakeholders and used to encourage commitment to transformation.

How does this apply to Software Development?

Watts Humphrey:

The history of software development is one of increasing scale.

Initially a few individuals could hand craft small programs;
the work soon grew beyond them.

Teams of one or two dozen professionals were then used,
but success was mixed.

While many organizations have solved these small-system problems, the scale of our work continues to grow.
Today large projects require the coordinated work of many teams.

Complexity is outpacing our ability to solve problems intuitively as they appear. What is required is a more structured approach to software process management.

In addition to working with very large projects, it has been found that the same methods are effective for smaller groups.

In simplified form, they are helpful for individual programmers.

If our methods do not serve the individual professionals, they will not endure.

People and the Software Process

Talented people are the most important element in any software organization. So the crucial initial step is to get the best people available. The better and more experienced they are, the better the chance of producing first-class results.

Once you have the best people you can get, however, what next?
If everyone wrote in different programming languages, used special conventions, or didn't coordinate their design and code changes with their peers, the results would be chaos.

Successful software organizations have learned that even the best professionals need a structured and disciplined environment in which to do cooperative work.

The Myth of the Super Programmers

There is a common view that a few first-class artists can do far better work than the typical software team.
The implication is that they will know intuitively how to do first-class work, so no orderly process framework will be needed.

If this were true, one would expect that those organizations who have the best people would not suffer from the common problems of software quality and productivity.
Experience, however, shows that this is not the case.

A few leading software organizations have consistently hired the top graduates from the best computer science schools. They are thus staffed with the best available people, yet their programming groups have many of the same problems that plague everyone else.

It seems that the super programmer approach requires better people than are available, even from our leading universities.
Attracting the best people is vital, but it is also essential to support them with an effectively managed software process.

Process and Technology

The Myth of the Silver Bullet

Another myth is the widespread belief that some technologically advanced tool or method will provide a magic answer to the software crisis.

This is not only wrong, but it is dangerous.

Technology is vital for long-term improvement, but unthinking reliance on an undefined "silver bullet" will divert attention from the need for better process management.

When asked to name their key problems, few software professionals even mention technology. Their major concerns are:

open-ended requirements,
uncontrolled change,
arbitrary schedules,
insufficient test time,
inadequate training, and
unmanaged system standards.

They may mention

limited machine time,
poor terminal availability, or
low-quality tools,

but these are more management than high-technology problems.

The effective use of software technology is limited by several factors:

an ill-defined process,
inconsistent implementation, and
poor process management.

These problems must be addressed first.

The characteristics of an effective software process

It must be predictable.

Cost estimates and schedule commitments must be met with reasonable consistency, and the resulting products should generally meet users' functional and quality expectations.

The software process is the set of tools, methods, and practices we use to produce a software product.

The objectives of software process management are to produce products according to plan while simultaneously improving the organization's capability to produce better products.

The basic principles are those of statistical process control, which have been used successfully in many fields.

A process is said to be stable or under statistical control if its future performance is predictable within established statistical limits.

When a process is under statistical control, repeating the work in roughly the same way will produce roughly the same result. The basic principle behind statistical control is measurement.

The numbers (measures) must properly represent the process being controlled, and they must be sufficiently well defined and verified to provide a reliable basis for action.

Also, the mere act of measuring human processes changes them.

Since people's fears and motivations are involved, the results must be viewed in a different light from data on natural phenomena.

Measurements are both expensive and disruptive; overzealous measuring can degrade the process we are trying to improve.

Software Process Development

The entire software task must be treated as a process that can be controlled, measured, and improved.

A process is a set of tasks that, when properly performed, produces the desired result.

A fully effective software process must consider the relationships of all the required tasks, the tools and methods used, and the skill, training, and motivation of the people involved.

To improve their software capabilities, organizations must take six steps:

Understand the current status of their development process.
Develop a vision of the desired process.
Establish a list of required process improvement actions
in order of priority.
Produce a plan to accomplish the required actions.
Commit the resources to execute the plan.
Start over at step 1.

The framework addresses the six improvement steps by characterizing the software process into one of five maturity levels.

Software professionals and their managers can identify areas where improvement actions will be most fruitful by establishing their organization's position in this structure.

The Five Levels of Software Process Maturity

(1) Initial

Characteristics: Chaotic--unpredictable cost, schedule, and quality performance.

Needed Actions: Planning (size and cost estimates and schedules), performance tracking, change control, commitment management, Quality Assurance.

(2) Repeatable

Characteristics: Intuitive--cost and quality highly variable, reasonable control of schedules, informal and ad hoc process methods and procedures.

Needed Actions: Develop process standards and definitions, assign process resources, establish methods (requirements, design, inspection, and test).

(3) Defined

Characteristics: Qualitative--reliable costs and schedules, improving but unpredictable quality performance.

Needed Actions: Establish process measurements and quantitative quality goals, plans, measurements, and tracking.

(4) Managed

Characteristics: Quantitative--reasonable statistical control over product quality.

Needed Actions: Quantitative productivity plans and tracking, instrumented process environment, economically justified technology investments.

(5) Optimizing

Characteristics: Quantitative basis for continued capital investment in process automation and improvement.

Needed Actions: Continued emphasis on process measurement and process methods for error prevention.

Software Process Improvement -- Summary

This maturity framework was developed at the Software Engineering Institute (SEI) of Carnegie Mellon University.

It is based on the extensive software experience of the SEI staff, and it has been found to reasonably represent the status and key problem areas of many software organizations.

As the practice of software evolves, however, and as our knowledge improves, we can expect this framework to evolve as well. It is presented here to provide a broader perspective on the improvement needs of software organizations.

Software process improvement actions are divided into 15 categories, which are further grouped into 4 major topics. In summary, these topics are:

Organization
Project management
Process management
Technology

1. Organization
This deals with management leadership of software organizations. This leadership is typically exercised through policies, resource allocation, management oversight, communications, and training. Policies state the basic principles of organizational behavior. The organization structure establishes basic responsibilities and allocates resources. Management oversight concerns management awareness of organizational performance. Communication deals with the means to ensure available knowledge to support timely action. Training ensures that the software professionals are aware of and capable of using the pertinent standards, procedures, methods, and tools .

2. Project management
Project management deals with the normal activities of planning, tracking, project control, and subcontracting. Planning includes the preparation of plans and the operation of the planning system. The tracking and review systems ensure that appropriate activities are tracked against the plan and that deviations are reported to management. Project control provides for control and protection of the critical elements of the software project and its process. Subcontracting concerns the means used to ensure that subcontracted resources perform in accordance with established policies, procedures, and standards.

3. Process management
With improving organizational maturity, a process infrastructure is established to uniformly support and guide the projects' work. This involves process definition, process execution, data gathering and analysis, and process control. The process definition provides a standardized framework for task implementation, evaluation, and improvement. Process execution defines the methods and techniques used to produce quality products. Analysis deals with the measurements made of software products and processes and the uses made of this data. Process control concerns the establishment of mechanisms to assure the performance of the defined process and process monitoring and adjustment where improvements are needed.

4. Technology
This topic deals with technology insertion and environments. Technology insertion covers the means to identify and install needed technology, while environments include the tools and facilities that support the management and execution of the defined process.

Process Management at Level 1

The critical steps are:

Install a management review system. This involves senior management and ensures that plans are produced, approved, and tracked in an orderly way.
Insist on a comprehensive development plan. This must include code size estimates, resource estimates and a schedule.
Set up a Software Configuration Management (SCM) function. This is crucial to maintaining control and must be in place and operational before completion of detailed design. These functions should then be expanded to include requirements and design.
Ensure that a Software Quality Assurance (SQA) organization is established and sufficiently staffed to review a reasonable sample of the work products. Until there is evidence that the work is done according to plan, this is essentially a 100% review. With successful experience the sampling percentage can be reduced.
Establish rate charts for tracking the plan. Typical milestones are: requirements completed and approved, the operational concept reviewed and approved, high-level design completed and reviewed, percent of modules with detailed design completed, percent of modules through code and unit test. Similar rate charts can be established for each phase of test.

Until Level 1 organizations demonstrate their ability to perform to a plan, the traditional management systems should be retained, whatever they are. It is essential to ensure that any new management system is working before abandoning the old one.

Basic Prerequisites for Changing an Organization's Software Process

To make major process improvements software organizations need:

an overall plan,
some dedicated process people,
clear goals, and
management's commitment to these goals.

A Framework for Software Process Change

The six requirements for software process change are:

1. Sell top management
Significant change requires new priorities, additional resources, and consistent support. Senior managers will not provide such backing until they are convinced that the improvement program makes sense.

2. Get technical support
This is best obtained through the technical opinion leaders (those whose opinions are widely respected). When they perceive that a proposal addresses their key concerns, they will generally convince the others. However, when the technical community is directed to implement something they don't believe in, it is much more likely to fail.

3. Involve all management levels
While the senior managers provide the resources and the technical professionals do the work, the middle managers make the daily decisions on what is done. When they don't support the plan, their priorities will not be adjusted, and progress will be painfully slow or nonexistent.

4. Establish an aggressive strategy and a conservative plan
While senior management will be attracted by an aggressive strategy, the middle managers will insist on a plan that they know how to implement. It is thus essential to be both aggressive and realistic. The strategy must be visible, but the plan must provide frequent achievable steps toward the strategic goals.

5. Stay aware of the current situation
It is essential to stay in touch with current problems. Issues change, and elegant solutions to last year's problems may no longer be pertinent. While important changes take time, the plan must keep pace with current needs.

6. Keep progress visible
People easily become discouraged if they don't see frequent evidence of progress. Advertise success, periodically reward the key contributors, and maintain enthusiasm and excitement.

Software and the Software Development Process

Software is increasingly important in the day-to-day operation of human society:

there is 7% growth per year in programming personnel, predicting
3 million people in USA and
10 million worldwide in the year 2000.

The Electronic Industries Association study of US Defense Department software costs predicted a 12% annual personnel growth rate from 1985-95.

A 1991 study shows that the annual growth in the Canadian software industry is projected to be 20%.

In 1989 and 1990, the Software Engineering Institute (SEI) carried out assessments of software processes, involving 167 projects in the USA and
196 projects in Japan.

The results of these assessments show that:

86% of the US organizations (98% in Japan) are at level 1 (initial) ;
13% (1% in Japan) are at level 2 (repeatable);
1% (0.5% in Japan) are at level 3 (defined);
0% (0% in Japan) are at level 4 (managed);
0% (0.5% in Japan) are at level 5 (optimizing).

These assessments clearly point out the appalling state of most software development processes.

Process Improvement

In 1987 and 1990, the SEI conducted process assessments at the Software Engineering Division of Hughes Aircraft, which led to a concrete action plan (e.g. formation of a software engineering process group, implementation of quantitative process management, enhancing the training program, and standardizing an effective review process).

Carrying out this action plan resulted in:

the division's progress from level 2 (repeatable) to level 3 (defined) ;
an estimated annual saving of about $2 million;
successful control of the effects of changing software requirements;
improvement in the quality of work life.

Another example of the SEI approach to process improvement is that of Raytheon, which started its software engineering initiative in 1988.

Over a period of 3-4 years:

the average annual savings in cost of quality was $5.8 million,
productivity gain was 29%,
investment in process improvement was $1 million.

Practical Software Engineering, Department of Computer Science

Rob Kremer