SITARA has been asked several pertinent questions and some of them find a
place here. Questions posed to SITARA & SITARA's responses to them will be posted from time to
time. If you want to submit a question, please use the Ask
SITARA Form.
QUESTION: While preparing for Peer Review Training, I came
across two things,
1) Informal Walkthrough
2) Structured Walkthrough
I am not very much clear the difference between these two. Could you
please help me out on this.
ANSWER: Informal Walkthrough: This is a reasonable mechanism to
adopt during a preparation to conduct a formal or structured walkthrough
or peer review. In such informal walkthrough's, the author and the
moderator will do spot checks, look for obvious mistakes which can be
categorized as 'oversight errors' to ensure that the work product is ready
for a peer-review. Informal walkthrough's generally does not involve all
the peer review team members and are used primarily to identify the
'obvious errors'. Informal walkthrough's can also be used for closing out
all the identified issues from a structured peer-review which identifies a
follow-up action plan. Here again, the author and moderator with one or
two key peer-reviewers can informally establish that all the issues from a
structured peer-review are closed to the satisfaction of fulfilling the
'critical-to-X' parameter where X is product quality (in the case of a
peer-review). Informal walkthrough's can use brainstorming to identify
potential alternative solutions which can help to improve the overall
quality of the work product.
Structured Walkthrough is rigorous process based on a team of
reviewers with assigned perspectives. Normally it is a team of about 5 to
7 people who will review the work product keeping their assigned
perspective in focus - testing, quality, design, end-user or customer.
These perspectives are normally assigned to individuals based on their
experience and skill profiles to help in looking at the work product from
all angles within the lifecycle to identify, isolate, minimize and
eliminate the impact due to the error in the work product. Structured
walkthrough's also have a pre-defined duration of about a maximum of 2
hours per session and are scheduled in such a manner as to look for
potential sources of mistakes (which are not obvious). Purpose of a
Structured walkthrough is to critique the existing work product without
making recommendations of alternative solutions or better ways of doing
the same thing. There is a formal closure of a structured walkthrough with
feedback to the author on the status of the work product - accepted as is,
conditionally accepted with a list of recommendations and an action plan
or rejected. Which means, the author along with a closure action team will
have to look at how best to implement the peer-review recommendations
within an assigned time. If the work product does not call for a
re-review, then Informal walkthrough may then be used to explore if all
issues are closed.
QUESTION:
We are implementing CMM Level
5 in our company. Abt Defect Prevention KPA, I request for clarification
to my doubts. While finding the root causes for the defects, during causal
analysis, to what depth we need to go? because, the root causes for
certain common defects which I have come across are not really 'the root
causes' for the defects.
Also, how ODC helps for Defect
prevention activities? Please suggest some articles/books/sites for ODC.
ANSWER: In your question to SITARA, you say that the causes
you are encountering are not really 'root-causes'. Here is what I would
think the reasons could be ...
Identification of root causes of defects in Defect Prevention has to
actually grow on top of effective 'Peer Reviews'. My suggestion to you
would be to actually come up with a peer review record giving a list of
all the peer review defect/error exposures. You can then classify the
errors into 'common cause' or 'special cause' based on whether the reasons
for the error/defect was 'due to the process' or 'assignable to reasons
outside the process'.
Common causes most definitely lend themselves to effective
root-cause analysis because the problem is in the process. Whereas special
causes are those where the error was due to assignable circumstances -
which may or may not be repeatable. In other words, there is a greater
possibility for 'false alarms' and 'non-reproducible' reasons surrounding
special causes. You need to know whether a special cause error/defect is a
'false alarm' or not - using professional judgement. If it is not a 'false
alarm' then figure out if it can be 'reproduced'. If it is not
reproducible, then just ignore it!
It is only now that it is worth your while to address both common
and special causes using 'root-cause analysis' by examining the potential
reasons for why the error or defect occurred - 'provided you can minimize
its impact in the future'. If you dont think you can minimize the impact
(either as rework, modification or reject) due to a reported error, then
treat it as a 'non-productive' error and stop the causal analysis.
Concepts behind ODC can definitely be applied depending upon how
critical your application is. If your cost of poor quality and cost of
quality measures are reasonable (less than 10% of total effort), then my
suggestion is to leave ODC alone! Treat every change as a defect - since
any modification requires rework and possibly results in waste when you
calculate CoPQ and CoQ.
QUESTION:
In QPM, we control the Project defined software process withing control
limits (capability levels) and take corrective actions appropriately for
the process metrics goals. We also derive the Organizational process
capability and performance baselines.
We understand that the quality of the software product (or work product)
depends on the quality of the process.
then why do we need SQM to define and control the product quality goals,
as long as we are controling the processes used to produce the software
product.
My understanding is- We need to measure and control the product quality charecterstics (requirement of stated customer / end
user) specially if the the current capability / performance of the
organization's / project's defined process MAY NOT be good enough to
achieve the product quality requirements??..
Will you please clarify...
ANSWER: The way to look at Level 4 KPAs of Software CMM is that QPM
and SQM are two faces of the same coin. In SQM you establish process,
product and service quality goals based on the 'critical to X' - where X
is cost, quality, scope or time - project constraints. These goals will
also have a significant influence from tailoring if the process is not
the same organizational process. And QPM takes a calibrated look at the
actual accomplishments.
In QPM you 'will have to' establish process capability baselines
using SPC techniques. Therefore you will use some form of control limits
based on process stability analysis to ascertain that you have insulated
potential influences of 'special causes of variation'. However, the
degree of impact from 'common causes' of process variation that are
inherent in the intrinsic ability of the process will not be quite under
control. The process capability baselines are for 'the tailored process'
which results from using the practices of Integrated Software
Management.
SQM involves establishing achievable and realisable product quality
goals based on the organizational experience with ISM and SPE. One of
the other important uses of SQM is to actually understand Cost of Poor
Quality and reduce it. As to why do you need SQM and QPM: yes, quality
of the process is positively corelated to the quality of the product.
And, yes again, you will need to do something about your product quality
if the current capability of the defined process is not good. What is
critical to note is that there is a large contribution of 'individual
abilities' to work with the process that dictates how well or how badly
the process variations are controlled. To that degree QPM and SQM
establish a foundation of practices that are necessary for 'effective
Defect Prevention' at Level 5. Without these practices, estalishing a
common basis to conduct studies on productivity improvements and cycle
time reduction would also be very difficult.
In a nutshell: QPM and SQM enables reduction to 'cost of poor
quality'. And today my understanding is that - Level 5 Key Process Areas
has to enable reduction of 'cost of quality' while enabling better cycle
time reduction and productivity improvement. That is the only objective
way to understand the 'common causes' of variation are brought under
statistical control.
QUESTION:
My company is ISO certified and are currently planning to go in for CMM certification. when we talk of process improvement.....which helps in improve the quality of the eliverable, I heard people saying, for a level 5 reached company, it takes two days to change a line of code, which is just a matter of few seconds generally. Does CMM
certification help in timely delivery? - what is your opinion about this statement?
ANSWER: Process like anything else sets the pitch for predictable delivery. That does not mean that it takes care of all the
uncertainties of the "real world"! We will still have to deal with realistic responses to quixotic requests.
Here is our point of view. If a level 5 company cannot deal with the vagaries of a Level 1 company; and does not know how to operate at a level of satisfaction that the customer wants/demands - it is a waste to have a process that is so very inflexible! Having said that, a true level 5 company can and will be able to meet customer expectations by adopting a suite of processes that are a true response to the situation that is being resolved. Process is most definitely not the ONLY solution that a company needs. There is no replacement to the
"gray matter" and technical expertise in a company - that has to be a given; then process is a good tool to establish consistency in delivery.
Bottom Line: Having a process is like having the "fulcrum"; you
still need the power to lift the load.
QUESTION: What are the
prerequisites of any software company to achieve CMM Level 2 and 3?
ANSWER: The most important
prerequisite is to institutionalize all the relevant key process areas at
the target level of process maturity. In this case, at Level 2 or Level 3.
For a level 2 maturity, the organization must institutionalize practices
around the 6 KPAs comprising Requirements Management, Configuration
Management, Software Quality Assurance, Software Subcontract Management,
Software Project Planning, Software Project Tracking &
Oversight. For an organizational process maturity at Level 3, in
addition to the above 6, the 7 Key Process Areas of Organizational Process
Focus, Organizational Process Definition, Training Program, Software
Product Engineering, Inter Group Coordination, Integrated Software
Management and Peer Reviews needs to be institutionalized through
appropriate practices. Then, there are assessment related prerequisites
that an organization needs to fulfill. Example, The3-Day Introduction to
CMM or an equivalent training must be imparted to the members of the
assessment team besides the mandatory Assessment Team Training using a
licensed assessment kit. For a legitimate CBA-IPI to result, it has to be
led by an SEI authorized Lead Assessor.
QUESTION: Is there any specific quantity requirement for minimum number of
projects/customers/employees for CMM level 2 and 3?
ANSWER: The more the better!
While there are no prescribed rules for a number, the most reasonable
thing to expect on an assessment is to have at least 4-6 representative
projects selected at random by the lead assessor. These projects
must have the typical characteristics of the business. It must represent
the business interests of an organization and at the same time have
sufficient process and project assets to demonstrate the use of the
organizational process and therefore an indirect measure of the process
capability. At Level 3, the process maturity assessment engages the entire
organization. Therefore, it is more than likely that at least 60-80% of
the organization will be called forth to represent in interviews with the
Assessment Team that is led by the Lead Assessor. The degree of coherence
and the level of detail described by the practitioners of the
organizational process determines the final outcome of the assessment.
QUESTION: What is the minimum timeframe for achieving Level 2? Same for
Level 3?
ANSWER: Depending upon how
prepared the organization is for change management, how geared the
organizational process is for adapting best practices, and the level of
proficiency of its process champions, Level 3 can result at a minimum of
12 months and at a maximum, may take forever! There are several other
factors concerning sponsorship, mechanisms to train and the basic
raw-material that forms the talent base of an organization that will also
determine how quickly an organization can move up the value chain. It is
very important to mention that, a level on the CMM scale is only an
abbreviation to the detailed process infrastructure that has to be in
place.
QUESTION: How many projects are required to be assessed for Level 2 and 3?
ANSWER: What is assessed is
the process capability and therefore the organizational maturity. ALL
projects must be following the process. At level 2 there is going to be a
large variation on how well these practices are performed. At level 3, the
variation on process adherence and internalizing the process culture is
very negligible. Therefore, the question of how many projects are sampled
by the lead assessor has a
contextual basis to it and is based on sound engineering and professional judgment.
QUESTION: What is the cost of
assessment for Level 2? Same for Level 3?
ANSWER: SITARA's pricing
does NOT depend upon the targeted maturity level. And, the pricing is very
much dependent upon several factors such as single site or multi site
assessments, prior experience of the organization on CBA-IPIs, additional
training that might become necessary for the assessment team to have a
credible assessment, the number of days of onsite and offsite activity for
the lead assessor and so on. Therefore the pricing varies from case to
case.
QUESTION: Is it advisable to go for
CMM Level 3 directly or to first go for Level 2 and then to Level 3?
ANSWER: You can always
target a Level 3 even before reaching a Level 2. And, in our opinion that
is the wisest thing to do! However, the challenges at level 3 are quite
steep and if the organization has no mechanisms at level 2 in place,
shooting for Level 3 is a management risk that the sponsor needs to
address. Shoot for the moon, if you miss you will be among the stars ...
is not advisable in the case of software process improvement because, the
entire program might lose its meaning and significance if the only thing
that matters is a "level". Detailed questions have been put
together by SITARA for the benefit of the sponsor and his team to pick the
right approach. You may look at it here.
QUESTION: Can we get some case
studies report for information?
ANSWER: The best possible
source of this information is www.sei.cmu.edu.
