Frequently Asked Questions

[Process]

[CMM-CMMI]

[SCAMPI]

[ISO15504]

Process Improvement

Does the concept of process encompass the method, the procedures and the steps?

The term process is, in fact, more global.
The software process of a company or agency refers to the way of doing things, the tools used, the methodology, the procedures, the standards and rules to respect, the organization of tasks, people assigned to activities, the roles assigned to the different participants, etc.  A process represents the transformation of input elements to make an output product, in a given objective, and using an agreed upon method more or less elaborated and more or less documented.

Does process improvement apply only to large organizations?

Not at all.
Improving the practices by using a reference model such as the CMMI or SW-CMM, applies to any organization or agency where systems solutions, totally or partially based on software, play an important role on the operations, independently of the number of employees.
Obviously, the model must be interpreted in relation to the size of the companies or agencies, but it is applicable in the overall context. Any entity that develops or maintains software, independently of its size, will benefit by improving its software process.

What are the benefits that a company or agency can expect from a process improvement initiative?

Many companies and agencies have documented and published the benefits obtained when investing in the improvement of their software process.
A study by the Software Engineering Institute, published in October 2003, shows convincing statistics on the positive repercussions of  process improvement. Below are a few excerpts from section 4.1 (pages 7 to 10) of the report " Demonstrating the Impact and Benefits of CMMI : An Update and Preliminary Results", Dennis R. Goldenson & Daine L. Gibson, CMU/SEI-2003-SR-009):

Impact on Cost

Impact on Delivery

Impact on Quality

Impact on Client Satisfaction

Return on Investment (“ROI”)

Here are some quotes from documents presenting data on process improvement benefits.

"We observed improved cost and schedule performance with increasing process maturity. Specifically, the least mature organizations were likely to have difficulty adhering to cost and schedule baselines. In contrast, the more mature organizations were likely to have on-baseline cost and schedule performance." (Patricia K. Lawlis, Robert M. Flowe et James B. Thordahl, "A Correlational Study of the CMM and Software Development Performance," Crosstalk: The Journal of Defense Software Engineering, Vol. 8, No. 9, September 1995)

"Data from 13 organizations were collected and analyzed to obtain information on the results of CMM-based software process improvement efforts. We report the cost and business value of improvement efforts, as well as the yearly improvement in productivity, early defect direction, time to market, and post-release defect reports. " (James Herbsleb, Anita Carleton, et al., "Benefits of CMM-Based Software Process Improvement: Initial Results", Software Engineering Institute, CMU/SEI-94-TR-13, August 1994.)

"It is clear from the data  (…) that SPI can have a significant bottom line cost savings to a software development organization (as much as a 67% reduction in development and rework costs). " (Thomas McGibbon, "A business Case for Software Process Improvement" , A Data & Analysis Center for Software (DACS) State-of-the-Art Report, September 1996 )

"Initial uses of CoSQ show that it is a very large percentage of development costs, 60 percent and higher for organizations which have not yet undertaken process improvement programs. CoSQ use can, however, demonstrate significant cost savings—such as RES' fourfold reduction in rework—for software organizations willing to undertake a process improvement initiatives." (Dan Houston et J.Bert Keats, "Cost of Software Quality : A Means of Promoting Software Process Improvement", Quality Engineering, Vol. 10, No.3, March 1998)

"Be realistic: If an organization deploys 2 or 3 improvements per year the average return is between 5 and 10%. However after 2 or 3 years of continuous improvement the ROI becomes impressive. The development team, after four years of effort, became 30% more productive" (Peter Lloyd, "Return on Investment from Continuous Software Process Improvements in an International Company", 5th International Conference on Software Quality, 23-26 October 1995, Austin TX)

"We found that defect density improved(…) with each software quality initiative but at a decreasing rate (…) An implication of this finding is that much of the effect of quality improvement may be realized from the initial quality improvement efforts." (Sandra A. Slaughter, Donald E. Harter et Mayuram S. Krishnan, "Evaluating the Cost of Software Quality", Communications of the ACM, August 1998, vol. 41, no. 8)

In general, as we can see from these excerpts, these organizations have recovered their initial investment very rapidly (in less than 18 months). The benefits can be measured in terms of less rework done, in more errors detected and fixed early in the life cycle (thus diminishing the cost of fixing), in accrued productivity, in more respect for and management of the requirements. These benefits can also be measured in an enhanced capability to deliver new or existing products within shorter schedules; furthermore, improving the credibility in front of the customers, etc.  No companies or agencies have ever declared losses when investing in the improvement of their software process.  By extension, we can reasonably assume that, while awaiting empiric data, it would be the same for systems process improvement.

How can the progress of software process improvement be measured?

A mechanism to measure progress should be in place. The SW-CMM model, for example, is subdivided in maturity levels and the CMMI is divided into organizational maturity levels “staged” representation or process capability levels representation “continuous”.  Found inside each level are several components  to be achieved and its accomplishment is detected by satisfying (or not) a number of clear and measurable goals.  It is easy to classify a company or agency within the maturity levels scale by doing an assessment (e.g. SCAMPI), and then to track the progress achieved from one evaluation to the next one.

What is a MATURE process? What are the characteristics of this type of process?

A mature process is a defined, managed, controlled, and measured process used in a coherent way within a company or agency. This provides the ability to repeat an activity with a predictable degree of success, because it is based on a process which permits these previsions thanks to its numerous characteristics.

Is the success of a company or agency influenced by the maturity of its software process?

There is a strong correlation between the maturity of the process and the quality of the product created by the process. If the quality of these products have a strong incidence in the success of a company or agency, as it is usually the case, then is easy to see the effect that a mature process will have in the success of a company or agency.  Examples of answers to the question of benefits below illustrate this.

How long it would take for a software process to become mature? What is the effort involved?

This depends on the company or agency: its starting point, its commitment, its determination, its ability to assign the required resources for improvement etc.
With the SW-CMM model ("staged"), There are empirical measurement figures showing that it would take approximately 5 years, to an entity of 50 software engineers or more, to bring its software process from a VERY immature level  (level 1 weak, using the SW-CMM as a baseline) to a level ensuring a good repeatability (mastering an SW-CMM level 2) ; and from 2 to 3 years to progress to each one of the superior steps (the SW-CMM model has 5 in total).
In general, there is consensus that a mastered level 3 represents a good level of maturity, where the risks are well managed. Nevertheless, many companies or agencies start with certain assets in place and should expect to reach  level two in 2 or 3 years, maybe 1 year if they have already invested in the improvement of their software process, independently without necessarily conducting a formal assessment.

The CMMI, the domain of applicability is vaster than for the SW-CMM: it touches all systems activities and not only software activities. Consequently, we can expect that the progression will take longer, the scope being larger. Even with the CMMI if we limit ourselves strictly to SOFTWARE, the same statistics mentioned above apply.
We also know that with the "continuous" version of the CMMI, we can focus uniquely on one or several process areas.  However, the conclusive results could be seen in the organization as a whole only by including a number of appropriate process areas (rather than only one).

The mere fact that an organization or agency commits to process improvement places it in a class apart, better able to meet challenges in becoming more efficient and competitive! During the progression, and not only at the achievement of a certain level, the tangible and the intangible benefits will be obtained.

What are the key factors to successfully achieve a significant and permanent improvement?

One of the most important success factors is the commitment of the management of a company or agency. It is fundamental for such an activity, as it is the software process improvement, that senior management provides a tangible support in both, in the short and long term. Then, there are other factors such as the coordination of this initiative with the strategic objectives of the company or agency as a whole, the assigning of skilled resources, a rational management of the software process improvement (as for any other large product, it must be tracked and controlled), and a good management of the benefits, the usage of a coherent referential is an important factor to know where the company or agency stands now and where they are going (measuring progress), and also the participation, at several levels of EVERYBODY, under the coordination of an Engineering Process Group (EPG).

What are the risks and limits associated to the software process improvement?

 Because it is an activity that requires a long term vision and commitment, it is vulnerable if the commitment (in terms of management or assigned resources) disappears. When this happens, it is very difficult to regain afterwards the necessary "momentum" needed for a successful operation.

In several companies, the information technology group is part of a hierarchy that more or less reports directly to top management.  Thus, the process improvement activity for systems development and maintenance, wholly or partially dependent on software, could be going on in the context of an enterprise that may not always have to the same general philosophy as to the rest of its activities.
In such a situation the information technology group, in spite of improving its process and, in consequence, the quality of its products, would be limited in its capacity to influence the whole organization if a similar initiative is not initiated by the rest of the units working with them.  On the other hand, this could be a good opportunity for the other groups to realize the need for improving their process, and to imitate the information technology group's determination for resolutely tackling problems.

Is software process improvement related to ISO-IEC 9000 certification?

ISO certifications, if smartly used as an improvement catalyzer, is totally compatible with a continuous software process improvement initiative.
Nonetheless, we should be aware of the dangers of misinterpreting the benefits of the certification (step translated in the delivery of a certificate). Some certified entities neglect to maintain the improvement effort in the long term, just focusing on a burst of energy during the period preceding the audit, and not maintaining the same effort afterwards (even with repeat evaluations imposed by the standard).
In this sense, we can not say that the ISO certification guarantees CONTINUOUS improvement. However, if intelligently used, an ISO-type certification (in particular with version 2000) could eventually produce a movement for continuous improvement.

What particular role do the CMU ("Carnegie Mellon University") and the SEI (Software Engineering Institute) play in relation to software process improvement?

The SEI is a prestigious institute linked to Carnegie Mellon University in Pittsburgh. CMU is recognized worldwide for the quality of its teaching, particularly in technology information.
Fully financed by the US Department of Defence, which establishes the SEI in the 80's, the SEI conducts research and technology transfer projects in the systems development, largely dependent on software, domain.
Its achievements have rapidly grown outside the boundaries of the defence sector and have reached the industry has a whole, even outside the American borders. Its work has often been adopted as a "de facto" standard around the world.
Among its programs of activities we find the process improvement program, that produced the SW-CMM and the CMMI, and the SCE, CBA IPI, and SCAMPI methods. The SEI ensures the evolution, according to the needs of its financial supporter, influenced by the industry as a whole.

How can we verify the quality of a qualified consultant's expertise who offers the Software Process Improvement services?

In addition to the usual criteria used for selecting any consultant in strategy issues that will be providing services to the Information Technology group, it is also advisable to examine if the skills of the consultants offering such services are accredited by authorities with a recognized reputation.
The Carnegie Mellon University, through the SEI ("Software Engineering Institute") is a prestigious and reliable reference. That is why Alcyonix has established close links (materialized by partnership agreements) with the SEI. Also, its experts have several SEI certifications.