Frequently Asked questions

[Process]

[CMM-CMMI]

[SCAMPI]

[ISO15504]

The SCAMPI Appraisal Method

What is a SCAMPI?

SCAMPI, "Standard CMMI Appraisal Method for Process Improvement, is a method which uses the CMMI as a baseline to identify the strengths and weaknesses of software and/or systems processes of an organization or agency. The method relies on a discipline and strict rules which assure the exhaustiveness and objectivity of an evaluation.
It is characterized largely by consensus (the evaluation is conducted by a team) and by the implication of the entity in its own evaluation. The SEI developed this method and administers a Lead evaluator accreditation program.
By extension, a process evaluation accomplished using the SCAMPI method is often referred to as a "SCAMPI". Several SCAMPIs have been conducted worldwide since the publication of the SCAMPI method, on which the SEI  periodically publishes statistics.

Does the SCAMPI method only apply to large companies?

Although conceived to evaluate large projects, the SCAMPI is not only for large companies or agencies. Several defined variations within the method permit a SCAMPI to be adapted to the profile of each organization or agency.
However, there are important limits  in the ability to apply the SCAMPI method to small entities (less than 20 people in the evaluated division). In this case, other evaluation methods which are better suited to small entities should be used. The CMMI could nevertheless be used as a baseline.

If the software process of our company or agency is evaluated using the SCAMPI method, is it possible to compare the results with those obtained by the rest of the industry?

The rigorous approach of the SCAMPI method combined with the systematic use of a coherent reference (the CMMI), allows us to compare the maturity of our own software process with the rest of the software industry, using the statistics published by the SEI. The SCAMPI method permits you to compare the maturity of your organization or its process capability on a scale defined by the CMMI and to precisely determine the degree of achievement of the objectives.
We can compare this data with the  SEI's statistics, and then easily see the relative position of the company in relation to other companies and agencies in the industry that have used the SCAMPI method to assess their own processes.

Does a SCAMPI appraisal provide a certificate of maturity, as official recognition of the status?

An assessment conducted with the SCAMPI method finishes with a presentation of the findings (optionally by delivery of a report) to the manager who requested the assessment, in presence of all the participants.  The results belong to the sponsor, who will decide its further diffusion.
No certificate is emitted to the company or agency assessed. In order to support its claims of being classified in a specific maturity level, the company has to show, as needed, the results of the SCAMPI.
However, the SEI guarantees the quality of its lead assessors, through its accreditation process. Although the accredited lead assessors send the results of all the evaluations conducted to them, the SEI does not provide any nominative information in this regard. The data is only used for analysis purposes and for the generation of periodic statistics.

The SEI's position reflects its purpose of putting emphasis not only in just achieving certain results at one point, but rather on maintaining, in the long run, a CONTINUOUS improvement movement.
It is not the SEI's objective to become a certifier of companies and agencies. The SEI concentrates on the definition, the evolution, and the support of models (SW-CMM and CMMI) and methods (CBA IPI and SCAMPI), also on the accreditation and the quality control of the evaluators. The SEI leaves to the companies and agencies the responsibility of publishing their own maturity level (justified or not) while providing reliable ways to verify those claims when requested.

What concrete steps should be taken to verify that the software process of an organization or a company has reached the level claimed to have been reached according to the CMMI scale?

The first thing is to verify if the assessment was conducted using a formal method, like the SCAMPI. When a formal method has not been used, is quite difficult to verify if the claimed level was achieved (it doesn't mean necessarily that the level has not been attained).
If the type of relationship with the company or agency allows (e.g. if you are becoming an important client), you can request access to the report produced by the formal assessment. We would first verify if the person who conducted the appraisal is an accredited lead appraiser; the SEI WEB site publishes the directory of accredited appraisers and, if there is still doubt, you can verify his/her status directly with the SEI.
Remember that the SEI will not reveal any nominative information on the companies or the agencies, and only provides information relative to the lead appraisers. A non-accredited lead assessor is not necessarily incompetent, but does not have objective means to validate his or her competence in the assessment field.
Once this verification has been satisfied, you should verify the date of the assessment and, knowing that, you can then ask what concrete actions were deployed, in order to address the weaknesses identified.  A SCAMPI provides an excellent portrait of your processes but this portrait reflects this reality at a precise moment.
Finally, we would identify which precise entity, in the context of the whole company or agency, has been assessed. All these steps will help you to understand the context of the software process that has been characterized as being at the level "n".

What is the difference between a SCAMPI assessment and an audit?

There may or may not be a difference: it depends on the vocation assessed by the SCAMPI.
In many cases a SCAMPI appraisal is carried out by the entity itself, conducted with an internal team supported by external expertise, and the objective is to review its own software process in order to improve it.
An audit, on the other hand, is conducted by an independent expert who appraises the status of a project (or several projects) and provides a report to the company or agency that has hired his/her consulting services; a service supplier can carry out the audit.
The same reference (e.g. the CMM) can be used by an audit or by an assessment (like a CBA IPI) and the method can be similar.  In the case of an internal evaluation, marked differences between a SCAMPI and an audit would be seen.

Nevertheless, A SCAMPI can be requested as an audit (external team). In this case, no notable difference would be seen in the approach applied by other types of audits.

Why are the accreditations provided by the SEI to CMMI instructors and SCAMPI lead appraisers important?

The accreditation process is designed and controlled by the SEI in order to maintain a high level of skills within the community of SCAMPI appraisers and CMMI instructors. The stages to be completed to get the accreditation are not easy to pass, even for an experienced software engineer (with more than 10 years of experience), and may take several months of intense effort from the part of the applicants.
Richard Basque, founder of Alcyonix, is both a SW-CMM and a CMMI instructor as well as an accredited lead evaluator for the SCAMPI, the CBA IPI and the SCE methods. Denis St-Pierre is a certified CMMI instructor and a SCAMPI lead appraiser.

How many SCAMPI appraisers accredited by the SEI are there? What requirements must they meet?

There are only a few hundred SCAMPI lead appraisers recognized by the SEI throughout the world.
The SEI has established very clear and stringent rules  to accredit lead appraisers. The SEI defines the acceptance rules to be met by the candidates and specifies the rigorous path to be followed in order to obtain the official accreditation.
This way the SEI can exercise quality control on the selection of the candidates, and ensure that potential customers will get the services of competent assessors.
Le SEI maintains a directory of accredited leads on its WEB site.

Does documentation on the SCAMPI method exist?

All SEI documents relating to the SCAMPI method, are classified as PRIVATE documents and are reserved only for lead evaluators or to SCAMPI team members led by these lead evaluators (they have to order a kit from the SEI, which verifies if the purchaser is properly accredited).

Who should I contact if I am interested in conducting a SCAMPI in my company or agency?

You can contact one of the accredited lead assessors who offer their services to external companies, other than their current employers (many accredited lead assessors working for big enterprises only provide internal services).
Their names are available on the SEI list (or by contacting the SEI directly), in order of company name and by country,  with an indication if they offer services outside their own enterprise.  Alcyonix, as SEI accredited lead evaluators, offers its services to conduct SCAMPIs in Canada or abroad, in English or in French.