Software Metrics A Rigorous and Practical Approach Edition 3

1. 1Fix the entity and the operating environment first. Reliability applies only to executable code, so name the exact build and the conditions of use (the specified environment) and the period of time (the specified period). A reliability figure without these three is not interpretable.
2. 2Define 'failure' before you collect anything, and keep it separate from 'fault' and 'change'. Reliability is observed through failures (departures from required behavior during execution), not through counts of internal faults. Record failures, faults and changes as separate incident-report data, as in the data collection chapter.
3. 3Log every observed failure with its time of occurrence, in operation or in testing. The probability of the next failure is a random variable, and the failure-time record is the raw data from which its distribution is built.
4. 4Choose the reliability measure as a property of that distribution: the mean (mean time to failure), the median, or the rate of occurrence of failures. State which one you report and keep it fixed across releases.
5. 5Specify a complete prediction system, not just a model. A model such as Jelinski-Moranda (exponential time to the ith failure, with N initial faults and α) needs (a) a procedure for estimating its unknown parameters, e.g., Maximum Likelihood Estimation from observed failure times, and (b) a procedure for interpreting the results, e.g., probabilistic margins of error.
6. 6Estimate the model parameters from the failure times you collected, then compute the predicted measure (e.g., MTTF for the next failure). Under Jelinski-Moranda the assumption is that each fault is removed when its failure is observed, which lowers the failure rate by ϕ = 1/α each time. Check that your process actually matches that assumption.
7. 7Treat the estimate as one output of one prediction system. Using the same model with different parameter-estimation procedures gives different results, so record which procedure produced each number before comparing across releases or projects.
8. 8Do not substitute internal process counts (faults found in formal testing, MTTF during testing) for operational reliability. Treat them as pitfalls: they say something about testing, and the link to reliability in the field is not established by the count alone. Validate any prediction against later observed failures.

The source's own worked case is the Jelinski-Moranda model (Example 3.13), which assumes an exponential distribution for the time of the ith failure. Its mean, the MTTFi, depends on N, the number of faults assumed to be initially in the program, and on 1/α, the 'size' of a fault, i.e., the rate at which it causes a failure. Faults are assumed to be removed on observation of a failure, and each removal reduces the rate of occurrence of failures by ϕ = 1/α. N and α are unknown, so you observe a series of failure times and estimate them with Maximum Likelihood Estimation. At that point you have a prediction system, not just a model: the model, the estimation procedure, and a way to interpret the results. The source makes the point with a non-software parallel (Example 2.13): predicting a London-to-Bristol journey cost as ab/c gives different answers depending on whether a, b, c come from a friend or from the automobile association. The same holds for reliability. Change the estimation procedure and the predicted MTTF changes, so the procedure must be reported alongside the number.

• If someone offers a fault count or testing-phase MTTF as 'the reliability', then treat it as an internal process measure with known pitfalls and ask for failure observations in the specified environment.
• If two reliability figures come from the same model but different parameter-estimation procedures, then do not compare them directly; they are outputs of different prediction systems.
• If the code is not executable (requirements, design documents), then reliability is not a relevant attribute for it; measure size or structure instead.
• If you must say something about future reliability, then state it as a distribution measure (mean, median, rate of occurrence of failures) estimated from past failures, and acknowledge it is a prediction from past observations.
• If your process does not remove a fault when a failure is observed, then check the model's assumption before trusting Jelinski-Moranda-style estimates.

• Counting faults instead of observing failures: the report quotes defects found or fixed as reliability. The tell is that no operating time or environment is stated anywhere.
• Quoting a model without a prediction system: a formula and a number appear, but nothing says how N and α (or equivalent parameters) were estimated. The tell is that no one can reproduce the figure.
• Mixing definitions of failure, fault and change in incident reports, so failure data are contaminated. The tell is that one problem report is counted under several categories.
• Treating a prediction as an observation: future reliability inferred from past failures is reported without margin of error. The tell is a single point value with no interpretation procedure.
• Assuming the model's fix assumption holds when faults are not cleanly removed after each failure. The tell is failure rates that do not fall as the model expects.

• ▸Entity (executable build/version): name the exact code being assessed.
• ▸Specified environment: describe the operating conditions and usage the figure applies to.
• ▸Specified period: state the time window or operating time over which reliability is claimed.
• ▸Failure definition: write the observable behavior that counts as a failure, kept separate from faults and changes.
• ▸Failure log source and times: where failure times are recorded and whether from operation or testing.
• ▸Reliability measure: choose mean time to failure, median, or rate of occurrence of failures.
• ▸Model: name it (e.g., Jelinski-Moranda) and list its assumptions, such as fault removal on each observed failure.
• ▸Parameter estimation procedure: e.g., Maximum Likelihood Estimation, with the data used.
• ▸Interpretation and validation: how you express margin of error and how you will compare the prediction with later observed failures.

1. 1Fix the attribute as external before you measure anything. Maintainability is the effort or time to do a maintenance task. It shows up only when someone modifies running software, and it cannot be read off the code directly.
2. 2Split maintenance into the three task types in the definition: correcting faults, improving performance, and adapting to a changed environment. Track effort or elapsed time separately for each type, because one pooled average hides which kind of change is costly.
3. 3Frame the measure with Goal-Question-Metric. Write the goal (for example, understand why some modules take longer to change), the questions it implies, and only then the metrics. The passages give GQM as the way to decide what to measure.
4. 4For every fault-correction task, record its mechanism in three parts: creation (when the fault entered), detection (how it was found), and correction (what was actually done). Log the effort or time against it. This follows the mechanism record used in the source's fault example.
5. 5Because maintainability cannot be measured until an operational version exists, collect internal structural metrics earlier as predictors. Take control-flow measures such as cyclomatic complexity from flowgraphs. Take coupling, cohesion, modularity, information flow and design-pattern use from design models such as UML.
6. 6Treat the link from internal metrics to maintenance effort as a prediction system, and validate it. Once real task effort data exists, check whether modules the metrics flagged as hard to maintain actually took more effort. If they did not, drop that metric as a predictor.
7. 7Use several structural measures, not one composite score. The source states there is no single overall 'software complexity' measure.
8. 8Use the data to reduce uncertainty, which is the source's definition of measurement (Hubbard). Even a few observed tasks per module tell you more than none. Use them to decide which parts of the current product to upgrade or improve.

The source's one concrete fault record concerns an up-arrow key on a machine. Its mechanism is recorded as creation during code development, detection by diagnosis of an operational failure, and correction by removing the up-arrow key from all machines and taping over the hole. As a maintenance record, this shows what the effort measure must capture. The correction was a physical workaround rather than a code change, so the task type is 'fault correction' but the effort was spent on a different kind of intervention. The passages give no effort figures, so none are invented here. In practice you would log the diagnosis time (detection) and the time spent applying the workaround across all machines (correction) as two separate numbers. You would also note that the fault was created in code development, which ties it back to the module whose structure might have predicted it. Repeating this record across many tasks gives the effort data needed to validate whether structural metrics predict maintenance cost.

• If the code is not yet operational, then do not claim a maintainability measurement. Report structural metrics as predictors only, and label them as such.
• If a module's structural metrics (for example, high cyclomatic complexity or heavy coupling) are bad but its logged maintenance effort is low, then distrust the metric as a predictor for your environment, and revalidate it against the effort data.
• If someone proposes one composite complexity or maintainability score, then ask for the component measures and test each against effort, since no single overall measure exists.
• If you cannot define how to measure maintainability directly, then propose a measure anyway, such as effort per task type. Proposing it opens the debate that leads to understanding, and a few observations already reduce uncertainty.
• If effort differs sharply between fault correction, performance improvement and adaptation tasks, then analyze each separately and do not average them.

• Treating maintainability as an internal code property. The tell is a dashboard of code metrics with no maintenance effort or time data behind it.
• Claiming measurement before the system runs. The tell is a 'maintainability score' reported for code with no operational version and no maintenance history.
• Skipping validation of the predictor. The tell is that complexity thresholds are enforced but no one has compared flagged modules against actual change effort.
• Relying on one composite number. The tell is a single index that cannot be broken back into coupling, cohesion or control-flow components.
• Logging only that a fault was fixed, not its creation, detection and correction mechanism. The tell is records that cannot distinguish a code change from a workaround, as in the taped-over key.

• ▸Task ID and module: identify the change and the module or component it touches.
• ▸Task type: correction of a fault, performance improvement, or adaptation to a changed environment.
• ▸Creation: where and when the fault or need arose (for example, during code development).
• ▸Detection: how it was found (for example, diagnosis of an operational failure).
• ▸Correction/Action taken: what was actually done, noting whether it was a code change or a workaround.
• ▸Effort or elapsed time: the measured effort for the task, recorded in consistent units.
• ▸Structural predictors at the time: cyclomatic complexity, coupling, cohesion and design-pattern use for the module, taken from flowgraphs or UML.
• ▸Predicted vs. actual: what the structural metrics suggested against the effort observed.
• ▸Decision: what to upgrade, improve, or re-validate as a result.

1. 1Name the entity before you count anything. In the source's framework, effort is a resource and process measure, so state which process (for example inspection, integration testing, or the whole development process) and which period the labor belongs to. A vague entity produces numbers nobody can compare.
2. 2Set the goal with Goal-Question-Metric. Write the goal (for example, 'evaluate the cost of inspections against the benefits'), the questions it implies, and only then the effort metric. This stops you collecting hours that no decision uses.
3. 3Fix the unit and the boundary: person-hours or person-months, and which roles and activities count. Record effort directly from time or staffing records. The source treats the number of personnel working in a specified period as a direct resource measure, so keep period and headcount explicit.
4. 4Record effort per sub-process, not only in total. The source's testing example splits unit, integration, system, and acceptance testing so each component can be judged for cost-effectiveness. Do the same for your process.
5. 5Pair effort with an output or outcome measure to derive a usable ratio. The AT&T pattern is effort per thousand lines of code reviewed. Another is effort or cost per error found. Label each as a derived measure (a calculation), not a direct one, and keep the raw effort and the denominator alongside it.
6. 6Combine the ratio with a benefit measure, such as faults discovered, to make a cost–benefit judgment. Effort alone says nothing about whether the labor was well spent.
7. 7If you use effort as the outcome of a prediction system, validate it. Compare predicted effort with actual effort on completed work before trusting it for planning, and state the uncertainty in the estimate. Software work happens under changing requirements and unpredictable staff turnover.
8. 8If an attribute you want (for example stability of the design process) is not well enough understood for numeric measurement, rate it on an ordinal scale and say so. Do not feed ordinal ratings into arithmetic that assumes ratio-scale values.

The source's AT&T case asks: are software inspections worth their cost? Managers needed to weigh what inspections cost against the benefits. The entity is the inspection process. The direct measures are effort expended (labor on inspections) and lines of code reviewed. The derived measure is effort per thousand lines of code reviewed, so E hours over K KLOC gives E/K. That ratio alone only describes cost. The managers also measured the number of faults discovered during the inspections, F. With F they could judge the cost of finding each fault (E/F) against the benefit received, which is the cost–benefit analysis the source describes. The lesson is that effort becomes decision-grade when you normalize it by size reviewed and set it against a result. Keep E, K, and F separately so the ratios can be recomputed and challenged.

• If someone hands you a single total of person-hours for a whole project, then ask for it split by sub-process before using it. Cost-effectiveness is judged per component.
• If effort is being compared across projects or periods, then require a normalizing size or output measure (such as KLOC reviewed or errors found) and check that it is defined the same way each time.
• If an effort figure is a ratio or formula result, then call it a calculation or derived measure and keep the underlying direct measures stored with it.
• If you plan to use an effort model to forecast, then validate it against actuals from finished work first, and report the forecast with its uncertainty rather than as a single figure.
• If the attribute is subjective or poorly understood, then use an ordinal rating and avoid averaging or ratio comparisons.

• Collecting hours with no goal. The tell: nobody can say which decision the data feeds, and the ratios are never used.
• Reporting effort as one project-level lump. The tell: you cannot say which sub-process (unit, integration, system, acceptance) was cost-effective.
• Treating effort per KLOC as proof of quality or productivity. The tell: no fault or benefit measure sits beside it, so cheap and ineffective inspections look identical to good ones.
• Inconsistent boundaries for who and what counts as effort, or for the period. The tell: staffing counts and hours do not reconcile between reports.
• Using an unvalidated prediction as a commitment. The tell: estimates are never compared with actuals, and uncertainty from changing requirements or staff departures is not stated.

• ▸Goal (GQM): one sentence stating the decision this effort data supports.
• ▸Entity and process: the sub-process or component measured (for example integration testing).
• ▸Period: start and end dates covered.
• ▸Effort (direct): person-hours or person-months, with the roles and activities included.
• ▸Personnel count: number of staff working in the period.
• ▸Size or output denominator: for example KLOC reviewed or errors found, with its definition.
• ▸Derived ratios: effort per denominator and effort per fault, each marked as calculated.
• ▸Benefit measure: for example faults discovered, so cost can be set against it.
• ▸Predicted vs actual effort and stated uncertainty: forecast, observed value, and the reasons for the gap.

1. 1Name the entity and the artifact first: code, design, or specification. Size is measured statically from that artifact, so no execution data enters. The book's Chapter 8 treats code size (8.2), design size (8.3) and specification size (8.4) separately, so state which one you mean.
2. 2Pick the unit to match the artifact and the question: lines of code (LOC), function points (FP), or number of classes. Chapter 8 also covers Halstead's approach, alternative code size measures and COCOMO II. Write down why this unit fits, and do not switch units mid-study.
3. 3Write an explicit counting rule before you count anything, because size is a direct measure and you must define it precisely. For LOC, the book's section on counting lines of code is the place to settle what counts as a line. Record the rule so a second person gets the same number from the same artifact.
4. 4Decide how nontextual or external code is handled (section 8.2.4 covers this). Generated, reused or imported material either goes in or out under a stated rule. Report the rule with the number.
5. 5Choose the lifecycle point you measure at. Code size exists only late. Design size, specification size and functional size (FP, COCOMO II) can exist earlier. Match the size type to when the decision is made.
6. 6Use size as a normalizer only after its definition is fixed. The book's own hypothesis is 'code produced using Scrum will have fewer defects per thousand lines of code than code produced using the XP method.' Count LOC identically for both groups before you compare ratios.
7. 7Treat any ratio built on size (defects per KLOC, effort per LOC) as a derived measure. State what it stands in for and what it might misrepresent. The book warns that a change in a surrogate is not the same as a change in the underlying factor.
8. 8Never present size as 'complexity.' The book states there is no single overall 'software complexity' measure (9.5), and structure is measured separately in Chapter 9.

The book's Example 4.2 gives the hypothesis 'Using the Scrum method produces better quality software than using the XP method.' It is untestable because 'quality' is not defined measurably. The fix defines quality as defects and restates the hypothesis: Scrum code will have fewer defects per thousand lines of code than XP code. Size is the denominator. Applied to this case: (1) the entity is the delivered code of each project; (2) the unit is LOC; (3) one counting rule is written and applied to both sets, including how reused or external code is treated; (4) defects are counted, and the count is divided by KLOC. The source then adds two caveats. Many defects found in testing may mean thorough testing or sloppy development. Delivered LOC may also not reflect the effort needed to complete the product. So the size figure makes the comparison testable, but you still have to read it as a surrogate-based ratio and not a direct verdict on quality or effort.

• If two products are compared on any ratio (defects, effort, cost) per size, then use one counting rule and one unit for both, and state it with the result.
• If the decision is needed before code exists, then use design size, specification size or functional size (function points, COCOMO II) and not LOC.
• If code includes nontextual, reused or external material, then apply and disclose an explicit inclusion rule (section 8.2.4) before comparing totals.
• If someone infers effort or quality from size, then treat it as a surrogate and ask what else could move the ratio, for example thorough testing versus sloppy development.
• If someone wants one number for 'how complex' a product is, then decline to use size for that and move to the structural measures of Chapter 9.

• Measuring what is easy in place of the attribute you care about (Krantz's 'misplaced operationalism'). The tell: LOC is quoted as if it were effort or quality and no model links them.
• Undefined counting rule. The tell: two people count the same artifact and get different totals, or the unit is just 'LOC' with no definition.
• Reading defects per KLOC as pure quality. The tell: a high defect count is called bad without asking whether testing was simply thorough.
• Mixing size types or units across products. The tell: one project is in LOC, another in function points, and they are compared directly.
• Folding generated, reused or external code in or out silently. The tell: size jumps or drops between releases with no change in the real work.

• ▸Entity measured: name the product and artifact (code, design or specification) and its version.
• ▸Question being answered: state the decision or hypothesis the size figure feeds.
• ▸Unit: LOC, function points, number of classes, or another; give the reason for the choice.
• ▸Counting rule: write the exact rule (what counts as a unit) so a second person reproduces the number.
• ▸Nontextual / external / reused material: state whether it is included and how it is counted.
• ▸Lifecycle point of measurement: say when in the lifecycle the number is taken and why that is early enough for the decision.
• ▸Size value and who counted: record the figure, the counter and the date.
• ▸Derived ratio (if any): name the ratio (e.g., defects per KLOC) and the factor it stands in for.
• ▸Known distortions: list what the ratio could misrepresent, e.g., testing thoroughness or effort not reflected in delivered LOC.

1. 1Write the objective first, in the form 'the structure measure predicts external attribute X', where X is maintainability, reliability, or testability. The source stresses that general reasons for measuring are not enough and that engineers need specific, clearly stated objectives. Without X you are collecting numbers, not measuring.
2. 2Split structure into the Chapter 9 aspects and pick one attribute per aspect you care about: structural complexity, length, coupling, cohesion, or a custom attribute. Do not merge them yet. Each is a separate attribute with its own properties.
3. 3Choose the level of the artifact. For code units, use the control-flow level: build the flowgraph and measure it with cyclomatic complexity, and with essential complexity for how structured the unit is. For design, use the design-level attributes: global modularity, morphology, tree impurity, internal reuse, and information flow. For object-oriented code, use object-oriented coupling, cohesion, length, reuse, and design-pattern use.
4. 4Model the artifact before counting. Control flow needs a flowgraph of the unit, and design needs a module or information-flow model. Write down the counting rules (what is a node, what is a connection) so two people get the same value from the same artifact. This is the representation condition: the number must preserve the empirical relation, so a unit judged more tangled must score higher.
5. 5Where testing is the goal, derive test-oriented measures from the same structure: the minimum number of test cases from the code structure, the test effectiveness ratio, and the information-flow test coverage measures. Structure then drives test planning, not just reporting.
6. 6Report the attributes as a profile, never one blended score. The source's section title says there is no single overall 'software complexity' measure. If management insists on a composite, state the weights and expect them to be contested, and validate the composite separately.
7. 7Validate against the external attribute. Collect the structure values and the observed outcome (defects, maintenance effort, test effort) for the same components. Test whether they relate, using the Chapter 4 and 6 empirical methods. Do not treat a measure as predictive until this is done.
8. 8Repeat the measurement at fixed points as the product changes. Use the values to flag components for review, redesign, or extra testing, and re-check the validation whenever the architecture or practices change.

Illustrative scenario, not from the source: the numbers are placeholders, so substitute your own. A maintainer wants to know where changes are most likely to be costly. Following the steps, the objective is 'structure predicts maintainability'. The attributes chosen are control-flow complexity inside units and coupling between modules. The maintainer builds a flowgraph for each unit and records its cyclomatic complexity, and counts the connections between modules from the design. Two modules look alike on lines of code, but one has a high cyclomatic value and heavy coupling to many other modules while the other has neither. A single complexity score would hide this. The profile shows the first module is tangled internally and also widely connected, so a change there is likely to ripple. The maintainer then checks recorded maintenance effort for all modules against the two attributes. If effort follows them, they are validated for this environment and used to rank modules for refactoring and extra testing. If it does not, the measures are dropped for this purpose or the definitions are revised.

• If someone asks for one 'complexity' number, then refuse the single score and deliver a profile of complexity, length, coupling, and cohesion, because no single overall software complexity measure exists.
• If the question concerns a single function or program unit, then use control-flow measures (cyclomatic and essential complexity from the flowgraph). If it concerns how modules relate, then use design-level measures (modularity, tree impurity, information flow). If the code is object-oriented, then use the object-oriented coupling and cohesion measures.
• If the goal is test planning, then use the structure-derived test measures (minimum number of test cases, test effectiveness ratio, information-flow coverage) rather than size alone.
• If you must combine measures into a composite, then document each weight and validate the composite separately, because controversy comes from both the components and the weights.
• If a structure measure has not been checked against an external outcome such as maintainability, reliability, or testability, then use it only as a descriptive indicator, not as a predictor.

• Treating cyclomatic number as the whole of 'complexity'. The tell: the same single figure appears in every report, and modules with heavy coupling or low cohesion pass review.
• Blending attributes into one weighted index. The tell: nobody can explain the weights, and arguments are about the formula, not the code.
• Measuring without an external target. The tell: dashboards exist, but no one can state which decision a value changes, so there is no stated objective.
• Inconsistent counting rules. The tell: two analysts or tools give different values for the same unit, so the measure does not satisfy the representation condition.
• Assuming a measure validated in one setting transfers to another. The tell: thresholds copied from elsewhere flag the wrong components and are never rechecked against local outcomes.

• ▸Objective: state which external attribute (maintainability, reliability, testability) the structure measure should predict, and the decision it informs.
• ▸Artifact and level: code unit, design, or object-oriented class set, and which model represents it (flowgraph, module structure, class relations).
• ▸Structural aspect: complexity, length, coupling, cohesion, or custom. Name one attribute per row.
• ▸Measure and counting rules: name the measure (e.g., cyclomatic complexity, information flow) and define exactly what is counted.
• ▸Test-oriented measure: record the minimum number of test cases, test effectiveness ratio, or coverage measure, if testing is the goal.
• ▸External outcome data: list the observed outcome you will compare against (defects, change effort, test effort) and its source.
• ▸Validation result: record what the structure-versus-outcome comparison showed and the method used.
• ▸Composite (if any): list component measures and weights, with the reason for each weight.
• ▸Action and review date: state which components get review, redesign, or extra testing, and when you re-measure.

1. 1Start from a goal, not a maturity level. Use the Goal-Question-Metric paradigm (Fenton and Bieman, section 3.2.1): write the goal (e.g., make inspection and testing predictable), the questions that show whether you are meeting it, and only then the metrics. A CMMI rating is not a goal. The source says CMMI relies on metrics, so the metrics have to exist before any assessment.
2. 2Classify every candidate measure using the book's framework (section 3.1): process, product (external or internal attributes), or resource. For this construct, favor process measures for design, coding, inspection and testing activities. Pair each with the product attribute it is supposed to influence, such as defect-based quality or test coverage.
3. 3Make the process visible before you measure it. The second edition added 'process visibility' to the framework. Write down which activities exist (design, coding, inspection, testing), what each produces, and where output is recorded. A step that leaves no record cannot be measured, so mark it as a visibility gap.
4. 4Fix data collection first (Chapter 5). Define good data, then separate incident-report records into failures, faults and changes (section 5.2) and keep them in distinct fields. Use a collection form or tool, and check the reliability of the collection procedure itself (section 5.4) before you trust any trend.
5. 5Select quality outcome measures that test whether the process works: defect-density and other defect-count measures (section 10.2.1), plus test coverage and a test effectiveness ratio (section 9.2.3). Use these to check that a more mature-looking process produces better results.
6. 6Combine GQM with process maturity (section 3.2.3). Use the maturity model to decide which measures are feasible now. In practice this means measuring only what the current level of process visibility can support, and adding measures as visibility improves.
7. 7Evaluate any process change with a baseline or sister-project design (section 4.3) rather than by opinion. Compare a project that used the changed process against a baseline or a parallel project. Record the confounders you cannot control, since the book lists threats to validity as a distinct topic.
8. 8Validate before you manage by the numbers (sections 3.4 and 3.3.7). Check that each measure captures the attribute you claim it does, and that any prediction system is tested against actual outcomes. Do this before targets or rewards attach to a metric.

The passages contain no numeric case, so this is an illustrative walk-through of the book's framework, not data from it. Goal: make inspection and testing effective and predictable on one product line. Questions: Are inspections happening and recorded? Do faults found late exceed faults found early? Does test coverage correlate with fewer post-release failures? Metrics follow from the questions: inspections held versus planned (process), faults recorded per phase of detection (from incident data, with faults kept separate from failures), a test coverage measure, and post-release defect density (external quality). Step one is a visibility audit, and it shows that inspections leave no record. The first action is therefore a collection form, not a metric target. Only after two or three releases of consistent data do you compare projects with and without inspection, using a baseline or sister-project comparison. At that point you can say whether process changes moved defect density, and you can describe the process's maturity with evidence rather than assertion.

• If an activity (design review, code inspection, test) leaves no recorded output, then treat it as a visibility gap and fix the record-keeping before defining any metric for it.
• If a proposed metric cannot be traced to a GQM question, then drop it, because it will produce data nobody can interpret.
• If failures, faults and changes are logged under one 'problem' category, then separate them before computing any defect-based measure, since the book treats them as distinct data types.
• If you claim a process change improved quality, then require a baseline or sister-project comparison and name the validity threats, rather than relying on before/after anecdotes.
• If a metric is about to be tied to targets or evaluation, then confirm the measure is validated for the attribute it claims to represent, to avoid managing by an unvalidated number.

• Treating a CMMI level as the goal. The tell: teams can describe their documented process but cannot show a defect, coverage or effort measure that changed as a result.
• Measuring without process visibility. The tell: metrics are estimated after the fact or reconstructed from memory, and no one can say where the data originated.
• Mixing failures, faults and changes in incident data. The tell: defect counts swing with reporting habits rather than with product or process changes, and two people count the same data differently.
• Using one coverage or defect number as proof of process quality. The tell: the number improves while post-release problems do not, which means the measure was never validated against the external outcome.
• Comparing projects with no control for differences. The tell: an 'improvement' appears only on a project with a different size, team or domain, and no baseline or sister project supports it.

• ▸Goal: one sentence stating the process-quality aim and the product or project it applies to.
• ▸Activity under review: design, coding, inspection or testing.
• ▸Visibility check: what record does this activity produce, and where is it stored? Write 'none' if nothing.
• ▸Question: what must be answered to know whether the goal is met?
• ▸Metric and class: name the measure and mark it as process, product (internal or external), or resource.
• ▸Data source and collection method: form or tool, who records, and how often.
• ▸Incident data separation: confirm failures, faults and changes are logged in separate fields (yes/no).
• ▸Comparison design: baseline, sister project or partitioned project, and the confounders you cannot control.
• ▸Validation note: how you checked that the measure represents the intended attribute, and the result.

1. 1Define fault-proneness operationally before measuring anything. The source defines it as failures during testing or operation that are traced to module faults. Record faults, failures, and changes as they occur, and trace each failure to the module that holds the fault.
2. 2Pick the unit of analysis (module or class) and the observation window. Briand et al. counted faults reported by users over a 1-year period. Fix your window in advance and use it for every component.
3. 3Choose candidate internal predictors from the product's structure: coupling (especially import coupling), cohesion, size, and design-evolution measures (changes to class attributes, methods, and associations/generalizations). Briand et al. applied 28 coupling and 10 cohesion measures to one industrial system.
4. 4Decide whether you are confirming a theory or exploring a relationship. For a theory (for example, relative defect proneness: defect density is higher in small modules than in large ones), state the hypothesis first. For exploration, run a case study and follow up with later studies.
5. 5Control confounders. Programmer experience, application type, and development environment can all mask or fake an effect, so hold them constant or record them.
6. 6Normalize carefully. Defect density is defects per thousand lines of code. Compare density when testing a size theory, and raw counts when ranking components for attention. Do not mix the two.
7. 7Validate each predictor against a direct, valid measure rather than a surrogate. If a second measure is itself a proxy, the correlation can mislead.
8. 8Treat the prediction as system-specific. Class 5 prediction systems (internal structure predicting external attributes) tend to work only on the system where the parameters were fitted. Refit or re-validate on each new system before trusting it.

Briand, Wüst, Ikonomovski, and Lounis explored which design properties are common in the most error-prone modules of one industrial software system. They applied 28 coupling measures and 10 cohesion measures to the modules. Fault proneness came from faults reported by users over a 1-year period. Coupling, particularly import coupling, showed the strongest relationship with fault proneness. Applied to your own system, you would compute import coupling per module, count user-reported faults per module over a fixed window, and rank modules on both. You would then direct inspection and testing at the highly coupled, fault-heavy modules first. Because Class 5 predictions tend to hold only on the system where they were fitted, you would confirm the pattern on your own data before generalizing from this study.

• If a predictor is an internal structural attribute (coupling, size) and you plan to use it on a different system, then re-validate it there. Class 5 predictions tend to work only on the system where they were fitted.
• If you are comparing modules of different sizes, then use defect density, not raw fault counts. Under relative defect proneness, small modules show higher density.
• If you must pick between candidate measures, then favor those with strong empirical links, such as import coupling in Briand et al., over measures chosen by intuition.
• If a measure is validated only by correlating it with another measure, then check that the second one is direct and valid. Surrogates mislead.
• If a module has poor internal attributes (large, unstructured), then treat it as likely but not certain to have more faults. Use it to prioritize attention, not to declare it faulty.

• Treating a model fitted on one system as general. The tell: strong correlation in the source system, none on a new one.
• Ranking modules by raw fault count, so large modules always look worst. The tell: fault-proneness rankings that simply track lines of code, while density tells a different story.
• Counting faults inconsistently, mixing failures, faults, and changes. The tell: the same defect counted twice, or a failure with no traced fault.
• Validating a complexity or coupling measure against a surrogate. The tell: the check does not follow a direct measure of the factor.
• Reading a correlation as a guarantee. The tell: modules with good structure are assumed fault-free, or poorly structured ones are assumed faulty, with no fault data behind either.

• ▸Component ID and unit type: name the module or class under study.
• ▸Observation window: state the start and end dates for fault counting (for example, 1 year of user reports).
• ▸Fault source: list where faults come from (user reports, bug tracker, commit messages) and how each is traced to the component.
• ▸Size (KLOC): record the size used to compute defect density.
• ▸Fault count and defect density: enter faults found and faults per KLOC.
• ▸Import coupling / other coupling value: record the measured coupling for the component.
• ▸Cohesion or design-evolution measure: record the value and the measure used.
• ▸Confounders held or recorded: note experience, application type, and environment.
• ▸Validation note: state whether the predictor was checked against a direct measure and on which system.

1. 1Classify comprehensibility correctly before measuring anything. In Table 3.1 it is an external attribute, a property of how a product behaves for the people who read it. It is not an internal attribute you can count. Write it as the target attribute and keep it separate from any counts you collect.
2. 2Name the entity under study: requirements (use case models and scenarios), design (design models), or code. The table lists different internal attributes for each, so the entity decides what you can examine.
3. 3Pick candidate internal attributes from the list for that entity, one at a time. Requirements: size, reuse, modularity, redundancy, functionality, syntactic correctness. Designs: size, reuse, modularity, coupling, cohesiveness, functionality. Code: size, reuse, modularity, coupling, functionality, algorithmic complexity, control-flow structuredness. Record each as its own measure.
4. 4Do not add the attributes into one 'software complexity' score. A single measure that stands for comprehensibility, correctness, maintainability, reliability and testability mixes conflicting goals. Report the attributes side by side so you can see which one drives a comprehension problem.
5. 5Test the size assumption explicitly. A size-driven measure says that adding code can never lower complexity (Weyuker property 5). A comprehensibility-driven view allows that a program can become easier to understand as you see more of it. Decide which view your measure serves and state it.
6. 6Treat the link between each internal attribute and comprehensibility as a hypothesis, not a fact. The source warns that judging external attributes from internal ones can lead to invalid conclusions. Look for direct ways to observe the external attribute itself, such as how readily maintainers understand the artifact, and compare your internal counts against that.
7. 7Keep the same measures and re-take them as the system evolves from one version to the next. The source says changes in product attributes must be tracked over time. A one-off snapshot cannot show whether understanding is getting easier or harder.
8. 8Prefer attribute-level findings over verdicts. A statement like 'coupling in this design is high and the maintainers report difficulty' can be acted on. A statement like 'complexity is 47' cannot, and it may satisfy formal properties while still not being a sensible measure (Cherniavsky and Smith).

Fenton and Bieman examine two of Weyuker's proposed properties for any 'good' complexity metric. Property 5 says M(P) ≤ M(P;Q) and M(Q) ≤ M(P;Q), so adding code never decreases complexity. That fits a size-based view. But it conflicts with the widely held belief that a program can sometimes be understood more easily as you see more of it, so a measure based mainly on comprehensibility cannot satisfy property 5. Property 6 says two bodies P and Q with M(P) = M(Q) can give M(P;R) ≠ M(Q;R) when each is joined to the same R. That depends on how the pieces interact, which is a comprehensibility concern and has little to do with size. The authors conclude that a single measure covering both size and low comprehensibility cannot satisfy both properties. Zuse backs this formally: property 5 requires a ratio scale for M and property 6 excludes it. The practical consequence is to measure size and the comprehension-related attributes as separate things instead of merging them into one number.

• If someone proposes one composite 'complexity' or 'quality' number to stand for comprehensibility, then split it into separate internal attributes. Conflicting goals make a single value unsound.
• If your measure must never fall when code is added (property 5), then it is a size measure and not a comprehensibility measure. Do not report it as comprehensibility.
• If you only have internal counts (size, coupling, nesting), then label the result a hypothesis about comprehensibility. Do not report it as comprehensibility, and look for a direct observation of understanding to check it against.
• If a candidate metric passes a list of formal properties, then still ask whether it is a sensible measure. Cherniavsky and Smith built one that satisfied all of Weyuker's properties and was not.
• If the system has changed versions, then re-measure the same attributes on the same entities before drawing conclusions about trend.

• Treating comprehensibility as an internal attribute: the tell is a report that 'measures comprehensibility' using only line counts or module counts.
• Collapsing attributes into one number: the tell is a single complexity or quality score with no breakdown, where a high value cannot be traced to a cause.
• Equating size with difficulty: the tell is an assumption that every added line lowers understanding, even where more context makes the code clearer.
• Confusing measures of different kinds: the source gives the example of measuring 'quality', an external attribute, by faults found in testing, an internal process attribute. The tell is a conclusion that jumps from a process count to a product property.
• Measuring once and never tracking change: the tell is a single baseline figure with no comparison across versions.

• ▸Entity: requirements, design, or code, and which artifact or version.
• ▸Stakeholder: who must understand it (developer, maintainer, other) and for what task.
• ▸Internal attributes selected: pick from the Table 3.1 list for this entity and measure each separately.
• ▸Measured values per attribute: record each value on its own line, with no combined score.
• ▸Size assumption: state whether more content is expected to raise, lower, or not affect comprehension.
• ▸Direct evidence of understanding: how you observed the external attribute itself, rather than inferring it from counts.
• ▸Hypothesis and gap: which attribute you think drives comprehension, and where counts and observed understanding disagree.
• ▸Version and date: the system version measured, so you can compare against earlier and later records.
• ▸Change since last record: which attributes moved and what the direct evidence shows about the change.