Measuring and managing performance in organizations (Austin, Robert D. (Robert Daniel), 1962-)

1. 1Name the single most-watched indicator in each role and ask what the cheapest way to raise it would be if the person ignored the real goal. In Blau's employment agency, the count of interviews completed was the only evidence the supervisor had, so interviewers minimized time spent locating jobs for clients. The cheapest path to the number is your first suspect.
2. 2List the dimensions of true performance that the measure does not capture (quality, fit, long-term results, non-key measures). Arrow's point applies: when the index is output rather than profit, people are tempted to waste inputs. Each unmeasured dimension is where effort will drain away.
3. 3Look for falsification risk wherever a measure is self-reported and the reporter's rating depends on it. Vietnam field officers carefully managed what they reported and often inflated counts. Ask who produces the number, who verifies it, and whether the producer is also the one rewarded.
4. 4Run a record-versus-reality audit on a sample. Compare the reported figure to an independent source, such as the raw slips, client outcomes, or customer feedback. Blau's replication found agents destroyed interviewing slips showing no referral at day's end. The audit needs access to source records, not only the summarized indicator.
5. 5Check for distortion without falsification: a number that is true but obtained by the wrong behavior. Agents referred clients to jobs they were ill suited for to raise referrals and placements. Ask whether the outcome behind the number (a suitable job) is still occurring.
6. 6Track measure versus true value over time. Early on, both rise because workers do not understand the system and targets are modest. Gaming appears as targets ratchet up through quotas or competition between coworkers. If the measure keeps climbing while the true output stalls, you are seeing the divergence in Austin's Fig. 2.1 pattern.
7. 7Do not respond by adding more indicators. Blau's office managers replaced one indicator with eight, and the replication showed more sophisticated dysfunction. Supervisors then fell back on a few key measures, and non-key measures were neglected. Treat gaming as something to manage, not eliminate: Ridgway and McGregor concluded no measurement system can preclude dysfunctional behavior.
8. 8Assume people will treat even informational measures as motivational. Ridgway notes that measures are read as definitions of what matters on the job. Unless trust is unusually high, a promise that data will not be used for evaluation is not credible, so expect people to prepare for the evaluation anyway.

Blau's study of two government employment agencies is the cleanest case. Interviewers were evaluated on the number of interviews completed, the only evidence the supervisor had, so they spent little time locating jobs for clients, which defeated the agency's purpose of finding jobs during a period of job scarcity. Managers noticed few placements and replaced the single indicator with eight, and performance appeared to improve. A later replication showed much of that improvement was more sophisticated dysfunction. To lift the indicator 'percentage of interviews that resulted in job referrals,' agents destroyed, at the end of the day, the interviewing slips showing that no referral had been made. Others sent clients to jobs they were ill suited for to increase referrals, and used several devices to raise placements. Supervisors then judged eight indicators too complex and relied on a few key measures, so effort moved toward those and away from the non-key ones. The lesson is that each redesign invited a new, subtler form of gaming.

• If a measure is self-reported and tied to the reporter's reward, then require an independent source (raw records, customer confirmation) before you trust it.
• If a measure has risen steadily while customer-facing results are flat, then assume gaming or distortion and audit the behavior behind the number before raising targets.
• If you are tempted to add more indicators to stop gaming, then expect supervisors to collapse them into a few key measures and expect neglect of the rest; fix the specific distortion or the incentive tie instead.
• If a measure is meant to be purely informational, then assume it will be read as an evaluation signal unless trust between workers and managers is unusually strong, and design as if it will be gamed.
• If targets or quotas are being raised or coworkers are being ranked against each other, then expect the shortcuts to become more severe and tighten verification at that point.

• Treating gaming as only falsification: you check for faked numbers but miss genuine numbers produced by the wrong behavior, such as referring clients to unsuitable jobs. The tell is clean records alongside poor outcomes for the people the work is meant to serve.
• Responding to gaming with more indicators: the multi-measure system is judged too complex, and key measures take over. The tell is that most of the indicators go unread and effort concentrates on two or three.
• Reading early improvement as proof the system works: both the measure and true output rise at first, because targets are modest and workers still follow the designers' intent. The tell is a gain that fades or reverses as targets ratchet up.
• Trusting summarized reports without source checks: an inflated count looks like any other count. The tell is that numbers hold up well on paper but cannot be reconciled to underlying records or field conditions.
• Declaring a measure 'informational only' and assuming behavior will stay neutral: workers still treat it as a statement of what matters. The tell is behavior that bends toward the metric even though no rewards were announced.

• ▸Measure and role: name the indicator and the person or group whose rating or reward it affects.
• ▸Cheapest route to the number: describe the lowest-effort behavior that would raise it regardless of true performance.
• ▸Unmeasured dimensions: list what the measure leaves out (quality, fit, long-term results, other tasks) that effort could be drained from.
• ▸Who reports it: state who produces the figure and whether that person benefits from it being high.
• ▸Independent source: identify the raw record or outside check that can verify the reported number.
• ▸Audit result: record the gap found between the reported figure and the independent source in a sample.
• ▸Distortion check: note any cases where the number is true but the behavior behind it hurt the real goal.
• ▸Trend comparison: compare the trend in the measure against the trend in true output or customer results over the same period.
• ▸Response chosen: state the fix (verification, dropping or reshaping the measure, reducing the reward tie), and confirm it does not simply add more indicators.

1. 1Name the customer separately from the principal. List who finally judges the value of the work (in Blau's employment agency: both job seekers and the firms seeking workers). Your own tastes as manager do not define value, and the principal cannot directly observe true value.
2. 2List the job's activities and keep only the critical ones, meaning no valuable output is possible without at least some effort on them. Use the General Electric test for each area: will continued failure here block the mission even if all other key areas go well? A yes marks a key area.
3. 3Run the second test GE's question omits: have all key areas been identified? A measurement set is not complete until this is answered yes. Check specifically for quality-related dimensions, since the passages note these are often hard to measure and there is no general link between ease of measurement and criticalness.
4. 4Write down the customer's preferences over effort pairs or sets. Apply 'more is better' (extra effort on one activity, others held constant, is preferred) and 'zero effort is awful' (any critical activity at zero yields no value). Then sketch the contour map of customer value to locate the best-mix path.
5. 5Compare the current measures and rewards to the best-mix path. For each measure, ask whether raising it moves effort along the path or away from it. Pay particular attention to employees who respond by pushing effort into the measured activity at the expense of the unmeasured one.
6. 6Decide the supervision mode by cost: if every critical dimension is measurable at acceptable cost, full supervision is possible. If even one critical dimension is too expensive to measure, full supervision fails, and you are in partial supervision. Do not judge this by average or median measurement cost.
7. 7If you are in partial supervision, treat the measured subset as a source of distortion. Weigh using it for pay against dropping incentives on it, or against delegatory management that relies on internal motivation, and check the factors that lower delegation costs (see the decision rules).
8. 8Run a recurring check for gaming of the indicators. Look for activity that makes the number rise while customer value does not, such as destroyed records or inflated ratios, and treat any sign of it as evidence the allocation has left the best-mix path.

Blau's employment office, as described by Ridgway, counted eight quantities per interviewer, including ratios of referrals to interviews, placements to interviews, and placements to referrals. The customers are both the people seeking work and the firms seeking workers. The simplified job has two critical activities, interviewing and referral, and if referral ceases, nothing of value emerges however many interviews are done. The customer prefers (5, 4) to (5, 3) or (4, 4), and treats (x, 0) and (0, y) as the worst allocations. Reading the system through the best-mix path, the measures reward placements-to-interviews, so interviewers pushed their effort toward making that ratio look good. They went so far as to destroy records of interviews that did not end in placements. The ratio rose while customer value did not, which is incentive distortion: effort was allocated according to the measure, not the customer's preferred mix. The lesson is that raising the number is not evidence of alignment. You must ask whether the effort mix moved toward what the job seeker and the hiring firm actually value.

• If a measure is chosen because it is easy to count, then check whether any critical dimension was left out; ease of measurement has no general link to criticalness.
• If even one critical dimension is too costly to measure, then do not claim full supervision; treat it as partial supervision, regardless of low average measurement cost.
• If the GE-style key-area question is answered yes for an area, then confirm the second question (all key areas identified) before trusting the set to pay or direct people.
• If the people judging value (customers) differ from the person designing measures (principal), then do not assume the available measure represents true value; treat it as a proxy that can be subverted.
• If delegation costs look low (small organization, long relationships, frequent interaction, culturally similar staff), then lean toward delegatory management instead of tight measure-based incentives.

• Measuring only the easy dimensions: the tell is that ratios improve while quality-related or customer-facing outcomes stay flat or worsen.
• Treating more total effort as success: the tell is that people work harder but on the wrong activity mix, as when interviewing earnestly continues while employer contact lapses.
• Letting the principal's preferences stand in for the customer's: the tell is that a measure looks right internally but nobody has checked what the final purchaser values.
• Stopping after the GE test: the tell is a list of confirmed key areas with no check that every key area is on it.
• Ignoring signs of gaming: the tell is record-keeping behavior that serves the indicator, such as discarding unsuccessful interviews, which signals that people are optimizing the number over the mix.

• ▸Job/role audited: name the role and the principal and agent in it.
• ▸Customer(s): list who finally judges the value of the output, kept separate from the principal.
• ▸Critical activities: list each activity where zero effort means zero value.
• ▸Key-area test (GE question): for each activity, answer yes/no to whether continued failure blocks the mission.
• ▸Completeness check: have all key areas been identified? Note any quality-related dimension that is hard to measure.
• ▸Customer's preferred mix: describe the effort combination the customer would value most, and which direction is 'more is better'.
• ▸Current measures and rewards: list each indicator and which activity it pushes effort toward.
• ▸Observed allocation vs. best mix: record where effort actually goes and the gap.
• ▸Supervision mode: full or partial, plus the one critical dimension that is too costly to measure, if any.
• ▸Evidence of gaming: note any behavior that raises the indicator without raising customer value.

1. 1Write the true performance goal in one plain sentence, as the end customer and the organization would state it. Use the form 'measure <parameter> in the hope of improving <goal>' and fill in the goal first. Example: long-term profitability, not 'productivity'. If you can only name the parameter, you have not yet defined true performance.
2. 2List every surrogate currently standing in for that goal. The passages say you settle for a surrogate because you cannot measure all aspects of progress: it 'seems to represent the goal closely and is simple enough to measure.' Name each one, and note what it leaves out.
3. 3Classify each measure as motivational or informational. Motivational measures are tied to rewards, such as bonuses, merit pay, promotion odds or dismissal, and are meant to change behavior. Informational measures are valued for the logistical, status and research information they convey, and are most representative when people behave as if the measurement system did not exist. A measure that is both is a conflict risk.
4. 4For each motivational measure, run the 'raise the number, not the goal' test. Ask what a person could do to push the parameter up while the goal stays flat or worsens. Check two things: how much effort goes in, and how that effort is allocated across task activities. Both determine the eventual value delivered.
5. 5Look for the verification gap. If the employee's private job knowledge makes it hard to tell whether they are fulfilling their part of the contract, expect opportunism to be possible. Record where the measure is the only evidence of performance, because that is where it is easiest to satisfy the letter while missing the spirit.
6. 6Check for known dysfunction patterns in your own setting: easy cases worked first, complaints unrecorded or posted late, instruction narrowed to the test, cosmetic compliance instead of function. If your measure invites a pattern like these, assume it is already happening and look for evidence.
7. 7Pair each surrogate with a direct, even if rough, look at the goal itself. Use customer outcomes, qualitative review and downstream results. Treat divergence between the two as the signal, since 'values of <parameter> go up comfortingly, but the values of <goal> get worse' is exactly how dysfunction shows up.
8. 8Decide the role of each measure explicitly. Keep informational measures out of reward decisions. Where you must use a motivational measure, treat it as imperfect alignment, and re-examine it whenever the parameter and the goal move in different directions.

Take the produce grading standards described in the source. The intended goal is quality and safety of the produce reaching the market. The measured surrogate is physical appearance. The reform argument is that the standards no longer guarantee quality and safety and, because they rest almost entirely on appearance, encourage use of potentially dangerous pesticides. A 1989 E.P.A. draft report said many applications 'do not increase yields but simply serve the purpose of maintaining cosmetic appearance of fruits and vegetables from minute defects, such as surface scarring or blemishes, or slight amounts of insect fragments.' Run the method on it. Goal: safe, good produce. Parameter: cosmetic appearance. The raise-the-number test exposes the move of spraying to remove blemishes, which improves the parameter and may worsen the goal. The source's verdict is that the measured standards 'interfered with the achievement of their intended effect.' A parallel case is the federal law enforcement agency in Blau's study. Agents with case quotas worked easy-to-solve cases first and neglected bigger, more important cases. The count rose while the true value of the work did not.

• If you can only name the parameter and not the goal it stands for, stop and define the goal before adding or keeping the measure.
• If a measure is tied to pay, promotion or dismissal, assume it will change behavior, and test whether the changed behavior serves the goal before you rely on it.
• If you need the measure to describe what is really happening, keep it informational and away from rewards, because people must behave as if it did not exist.
• If the parameter improves while customer-facing or goal-level results flatten or worsen, treat it as dysfunction and not as noise. Investigate the allocation of effort before praising the number.
• If performance can only be verified through the measure itself, expect opportunism to be possible and add a direct check on the goal.

• Treating dysfunction as a rare anomaly. The passages say it is closer to the rule: anything measured is likely to show at least some. The tell is surprise when a gamed metric turns up.
• Mistaking the surrogate for the goal. The tell is that people describe success in terms of the metric (quota met, clearance rate, test scores) with no reference to the customer or organizational outcome.
• Measuring effort quantity but not allocation. The tell is more total activity with easy or visible work done first and harder, more important work neglected, as in the agents with case quotas.
• Forcing one measure to serve both motivational and informational purposes. The tell is that data once reliable becomes unrepresentative after rewards are attached, because people now act with the measurement in mind.
• Assuming measurement is quick and easy. The tell is a measure installed without asking what behavior it rewards, or what it costs in distorted effort.

• ▸True performance goal: state in one plain sentence what the end customer and the organization actually value.
• ▸Measured parameter: name the surrogate currently used in its place.
• ▸What the parameter omits: list the aspects of the goal the surrogate cannot see.
• ▸Measure type: motivational (tied to rewards or penalties), informational, or both. If both, flag the conflict.
• ▸Raise-the-number moves: list actions that would push the parameter up without improving the goal, covering both effort level and effort allocation.
• ▸Verification gap: describe what in the employee's private job knowledge keeps you from verifying the spirit of the work.
• ▸Direct goal evidence: name the rough, non-surrogate look at the goal you will use as a cross-check.
• ▸Divergence signal: state what pattern of parameter-up, goal-flat-or-down triggers a review.
• ▸Decision: keep, make informational only, redesign, or drop. Give the date to revisit.

1. 1Declare the intended use of each measure in writing before collection starts: motivational (meant to change the behavior of the people measured) or informational (process refinement or coordination). Informational data is most representative when people behave as if the measurement system did not exist, so classify first.
2. 2Sort every measure into the informational form it serves. Process refinement measures reveal the detailed structure of a process, in the Taylor experiment style. Coordination measures support short-term flow and schedule management, such as paper stock levels or how far ahead or behind schedule a software project is running. Keep the question each one answers to a single sentence.
3. 3Run a reaction test on each informational measure. Ask who sees it, who decides rewards or punishments, and whether those people could read the measure as a definition of what matters. Ridgway's point is that even purely informational measures are interpreted as signals of what is important. If the answer is yes, treat the data as contaminated by reaction until shown otherwise.
4. 4Check whether a price, grade, or rating attached to the measure invites reactions even though no one intended it. In the farm-produce grading example, grades were meant as a common language for buyers and sellers, but higher prices for some grades made buyers and farmers react anyway. Look for any attached consequence the design does not preclude.
5. 5Separate true characteristics from substitutes. For each metric, write down the true quality or output you care about and the proxy actually measured. In Ishikawa's newsprint example, the true characteristic is tear resistance on a rotary press, which can only be tested destructively. Weight and thickness are the substitutes, and paper acceptable on both can still tear.
6. 6List the critical dimensions of effort or output, meaning those where no valuable output exists without effort on them. Confirm each is measured. In the Blau employment agency, interviews were counted but referral calls to employers were not, even though no value is created without referral. An unmeasured critical dimension means the data misrepresents the process.
7. 7Where a measure serves both purposes, segregate the information by intended use. Use separate data streams, restricted access, or both for the informational stream, so that motivational uses are precluded by design rather than by good intentions.
8. 8Run a drift check on a recurring schedule. Compare the measure against direct observation of the underlying process, and look for changes in behavior after the measure was introduced, such as the Soviet nail factories shifting from many small nails to a few huge ones when the target changed from count to weight.

Take the Blau employment agency example from the passages. Agents are measured on counts of interviews conducted, a proxy for the true output, which is value to the organization (placing people in jobs). The measure is informational in intent, but the agents read it as a definition of the job. Interviews rise, so the dashboard looks healthy. Value is not additive across interview and referral activity: if agents conduct many interviews but never contact prospective employers, the agency delivers no value. Referral is a critical dimension of effort that goes unmeasured. The interview count therefore reports accurately on the thing it counts and misrepresents the process as a whole. Informational validity is restored by adding a referral measure, such as calls to prospective employers, and by checking whether the interview count still correlates with satisfaction in the employment office. The correlation justifies the proxy only while behavior has not been bent toward the proxy.

• If a measure is meant to be informational and the measured people can see it or know who sees it, then assume it influences behavior and either hide it from reward decisions or discount its validity.
• If a true characteristic can only be tested destructively or cannot be observed, then use substitute characteristics, but record that they are imperfect indicators and verify them periodically against the true characteristic.
• If a critical dimension of effort is not measured, then do not trust the other measures on that process, because dysfunction becomes possible.
• If a measure serves both motivational and informational purposes, then segregate the information by intended use and treat the informational stream as the one at risk.
• If the target definition changes, such as count versus weight, then expect the output to shift to match the target, and recheck what the number now represents.

• Assuming a measure is purely informational because that was the intention. The tell: the people measured talk about the number as what the boss wants, or grades and prices attach to it, as with produce ratings.
• Measuring the proxy and treating it as the true quality. The tell: all metrics are in range yet the true characteristic fails, like newsprint acceptable in weight and thickness that tears on the press. This is Merton's displacement of goals, where an instrumental value becomes terminal.
• Measuring inputs or one effort dimension while skipping a critical one. The tell: activity counts climb while the outcome that matters stays flat, as with interviews rising and no referrals.
• Combining motivational and informational uses in one system and expecting both to hold. The tell: justifications mix phrases like improved accountability with better understanding of processes, and the data shifts once rewards are linked to it.
• Mistaking compliance with a target for accurate data. The tell: quotas are met exactly while the output is useless, as with factories producing only size-7-left boots.

• ▸Measure name and owner: name the metric and the person accountable for its collection.
• ▸Intended use: state motivational, process refinement, or coordination, in one sentence.
• ▸True characteristic or output: describe what you actually want to know, apart from what is counted.
• ▸Proxy actually measured: name the substitute and the evidence (such as an observed correlation) that links it to the true characteristic.
• ▸Critical dimensions of effort: list every dimension without which no valuable output exists, and mark each measured or unmeasured.
• ▸Visibility and consequences: record who sees the measure, who controls rewards or punishments, and any price, grade, or rating attached to it.
• ▸Reaction signals observed: note any changes in behavior since introduction, such as shifts in output mix or gaming.
• ▸Segregation safeguard: describe how informational use is separated from motivational use (separate stream, restricted access).
• ▸Verification check and date: specify the direct observation or test against the true characteristic, and when it will next be run.

1. 1Inventory every measure that carries a consequence. List each bonus, merit-pay formula, promotion criterion, performance rating and dismissal trigger, then tie each to the measure that feeds it. The source defines motivational measurement as any attempt to reward strong performance as determined by an established measurement system, and that includes less tangible rewards such as a higher probability of promotion. Include measures built for another purpose, because a manager may feel a planning measure affects their career 'profitability' even when it was never meant to.
2. 2Classify each measure by intended use: motivational (meant to provoke greater effort toward organizational goals) or informational (valued for the logistical, status and research information it conveys). Motivational measures are meant to change behavior. Informational measures must not change it, because they are most representative when people act as if the system did not exist. Mark any measure that serves both, since the source says forcing the two to be compatible causes dysfunction.
3. 3For each motivational measure, write down the true output you actually want (value to the organization, the product's real quality) and the substitute characteristic being measured. Ishikawa's newsprint is the template: tear resistance on a rotary press is the true characteristic, and it can only be tested destructively, so weight and thickness stand in for it. Record the substitute-to-true gap explicitly.
4. 4Check the direction of the implication. Observed correlation usually runs one way: strong performance implies the measure is in a favorable range, but a favorable measure does not imply strong performance. Test each measure with the reverse question: 'If this number is in the target range, what could still be wrong?' Write down at least one way to hit the number without producing the true output.
5. 5Flag input-for-output substitutions. If the measure counts effort (interviews performed) or a proxy for a result (on-time delivery) and the reward pays on it, the person being paid has a reason to maximize the proxy. Add the cost side: a measure on output alone invites waste of inputs, and a measure on one dimension of appearance invites actions that improve the appearance rather than the produce.
6. 6Surface your behavioral assumptions. Ask whether the plan assumes people will not perform unless contingently rewarded (the rational self-interest view) or that internal motivation matters. Reliance on external rewards and objective targets implies conservative assumptions about human motivation, namely that internal motivation is inadequate. If your system says it trusts people but pays purely on counts, the assumptions and the design conflict.
7. 7Set the reliance level per role. Where the measure tracks the true output closely, a tight reward link is defensible. Where only imperfect substitutes exist, loosen the link, supplement the measure with judgment, or move the measure to informational use only. Segregate information by intended use so data gathered for learning is not also used to pay.
8. 8Record the decision and the reasoning in the tool below, and revisit it when the measure, the role or the reward changes.

Ishikawa's newsprint case shows the structure. The true characteristic is resistance to tearing on a rotary press, and the only test is destructive: actually printing on the paper. A purchaser therefore relies on weight and thickness. Paper acceptable in both can still tear on the press, so the substitute is an imperfect indicator. Now move the same logic to people. An employment office's true output, value to the organization, is intangible, so the organization counts interviews performed. The correlation is real: satisfaction with the office usually means many interviews are occurring. But the implication fails in reverse, because a large number of interviews does not imply general satisfaction with the office. If bonuses or promotions attach to the interview count, the count becomes the goal rather than a stand-in for one, which is what Merton calls displacement of goals, where an instrumental value becomes a terminal value. The audit step is to write the true output (satisfaction, value), the substitute (interview count), and the one-way test: it is possible to hit the count and still fail the true output. The remedy is not necessarily to drop the count, but to avoid tying heavy rewards to it alone.

• If the true output can be measured directly and cheaply, then a contingent reward on it is defensible; if only a substitute can be measured, then keep the reward link weak or add judgment.
• If a measure is meant to supply accurate information about how work really goes, then do not attach rewards to it, because people being measured will stop behaving as if the system did not exist.
• If the measure counts inputs or effort while the organization wants output, then expect effort to be directed at the count and treat that as a design risk rather than a character flaw.
• If the measure covers one dimension of a multi-dimensional outcome (appearance of produce, delivery date of an airplane), then expect effort to shift to that dimension at the expense of the rest.
• If your design assumes people will not perform without contingent rewards, then state that assumption explicitly and test it, since social scientists and managers have often questioned it.

• Treating a one-way correlation as an equivalence. The tell: someone says 'satisfied customers have high on-time delivery, so high on-time delivery means satisfied customers.' The source's counterexample is on-time delivery that does not guarantee satisfaction with a new airplane.
• Letting the substitute become the goal. The tell: people discuss the count (interviews, weight, thickness) rather than the output it was meant to indicate, and hitting the target ends the conversation.
• Rewarding output alone and ignoring inputs, or the reverse. The tell: a manager paid on output rather than profits is wasteful of inputs, and an effort-count target produces busy activity without value.
• Assuming a non-incentive measure is harmless. The tell: a measure introduced for planning gets treated by managers as a career signal because they see an effect on their prospects.
• Adopting rational self-interest as a default because it is simple. The tell: the design is clean and mathematically tidy, but nobody has checked whether people actually act only for net monetary benefit.

• ▸Measure and owner: name the measure and who is held to it.
• ▸Reward link: state exactly which bonus, rating, promotion or penalty depends on it.
• ▸Intended use: motivational, informational, or both; if both, note the conflict.
• ▸True output: describe what the organization actually wants, in plain words.
• ▸Substitute characteristic: describe what is actually counted or weighed in its place.
• ▸Forward test: does strong true performance push the measure into the target range? Give the evidence.
• ▸Reverse test: describe a way to hit the target while the true output is poor.
• ▸Behavioral assumption: does the design assume internal motivation is inadequate? Note whether that is justified.
• ▸Decision: keep, loosen the link, add judgment, or make informational only; give the date for review.

1. 1List every dimension of effort the job involves, using the job's real work rather than the existing scorecard. Blau's employment agency agents, for example, conduct interviews and also make referral activities such as calls to prospective employers. Start from the work, not from what is already counted.
2. 2Run the criticality test on each dimension: if an agent devotes zero effort to it, can any valuable output still be created? If no valuable output exists, the dimension is critical. If value survives, it is not critical. In the can-opener example, a designer who spends no effort on stylishness still produces a can opener that opens cans and may be marketable.
3. 3Check value structure: ask whether value is additive or depends jointly on effort across tasks. In the agency, value is not the sum of interview value and referral value. Many interviews with no referrals yield no value. Treat any dimension that works like referral as a must-measure, not a nice-to-have.
4. 4For each critical dimension, record whether a signal exists and whether it is feasible and accurate. Signals may arrive individually rather than collectively, and for some tasks no signal is available. Do not count a dimension as measured because a proxy exists. Count it only if the signal accurately reflects effort on that dimension.
5. 5Compute coverage: critical dimensions with a feasible, accurate measure divided by all critical dimensions. Any gap means partial supervision, the condition where dysfunction arises. True output depends jointly on effort across all tasks and cannot be measured directly or used to set payment, so you cannot treat an output total as covering the gaps.
6. 6Do not rely on balance or the number of measures. Experts such as Chew, Eccles, and Kaplan and Norton recommend multiple measures across areas of performance, and balance, but most do not stress missing no critical dimension. A balanced scorecard can still omit a critical dimension.
7. 7For each uncovered critical dimension, choose one of two actions: redesign work or the measure so the dimension becomes feasibly measurable, or stop using measurement-based management for that job. Use the passages' criteria: if measurement on critical dimensions is likely to be inexpensive (repetitive, well-established production), measurement is viable. If measurement cost is high on some dimensions, as in creative advertising work, delegation is the recommended method.
8. 8Revisit the audit when the work changes. A job moving from repetitive and well-established to novel changes measurement feasibility, so re-score each critical dimension.

Blau's employment agency is the passages' own case. Agents do two things: conduct interviews and make referral calls to prospective employers. Interviews are easy to count. Referral effort is harder to signal. Value is non-additive: an agent who conducts many interviews but performs no referral activities provides no value, because no valuable output exists without referral. Referral therefore passes the criticality test and interviewing alone is not enough. A system that measures only interviews leaves a critical dimension unmeasured, which is partial supervision, so effort drifts toward what is counted. Coverage is one of two critical dimensions, or 50 percent at best. Contrast the can opener: stylishness is valued by customers but fails the criticality test, since an unstylish opener still opens cans. Leaving it unmeasured does not create the same exposure. The audit therefore demands measuring referral and permits skipping stylishness.

• If a dimension yields zero valuable output when effort on it is zero, then it is critical and must be measured accurately, or the job should not be run on measurement alone.
• If value is non-additive across tasks (as with interviews and referrals), then reject any scorecard that measures only the easy task, however many measures it has.
• If a critical dimension cannot be measured feasibly and accurately, then you have partial supervision, so expect dysfunction. Redesign the work for measurement or shift to delegatory management.
• If the work is repetitive and well-established (the No. 2 pencils and rivets examples), then measurement on critical dimensions is likely inexpensive and measurement-based management is workable.
• If the work depends on creativity and originality (the advertising agency), then full supervision is unlikely, so prefer delegation.

• Treating balance as completeness: the scorecard has many measures across areas, yet no one has checked them against critical dimensions. The tell is that the design rationale mentions balance and breadth but never names what happens if effort on a given dimension drops to zero.
• Measuring the easy dimension and calling it the job: interviews get counted, referrals do not. The tell is that reported activity rises while valuable output does not.
• Mislabeling a non-critical dimension as critical, or a critical one as optional: effort goes into measuring stylishness while referral goes untracked. The tell is that nobody can state what output survives if effort on the dimension is zero.
• Substituting total output for dimension-level measures: true output depends jointly on effort across all tasks and cannot be measured directly. The tell is that pay or evaluation rests on one aggregate figure with no view into task-level effort.
• Treating a one-time audit as permanent: work shifts from routine to novel and coverage quietly erodes. The tell is that the measures predate a change in the nature of the job.

• ▸Job or role: name the position being measured.
• ▸Effort dimensions: list every distinct kind of effort in the actual work.
• ▸Zero-effort test: for each dimension, state what valuable output exists if effort is zero.
• ▸Critical? (Y/N): mark Y only if no valuable output remains.
• ▸Value structure: note whether value from this dimension is additive or depends jointly on others.
• ▸Current measure/signal: name the metric or signal for each dimension, or write none.
• ▸Feasible and accurate? (Y/N): record whether the signal accurately reflects effort on that dimension.
• ▸Coverage ratio: count critical dimensions with Y divided by total critical dimensions.
• ▸Action for each gap: choose redesign for measurement, or delegatory management, with a one-line reason.

1. 1Name the true output first. Write down what the customer actually values (for the employment office: placements and satisfied clients; for newsprint: resistance to tearing on the press). If the only direct test is destructive or intangible, you are in delegatory territory. Measurement can only reach substitutes, and substitutes are imperfect: the implication runs from good performance to good measures, rarely back.
2. 2List the proxy measures you would otherwise use (interview counts, weight, thickness, on-time delivery) and test each in both directions. Ask: does strong performance usually produce a good number? Does a good number guarantee strong performance? Where the second answer is no, that measure cannot carry the weight of control.
3. 3Check that the agent's utilities can be aligned with customer value. The model's no-distortion result holds only if the agent's values already point toward the customer and he treats all effort as equivalent. If he dreads one activity (the source's example is interviewing versus referral work), his unsupervised allocation can drift off the best-mix path. Fix or acknowledge that before relying on delegation.
4. 4Choose levers that change preferences, not payoffs. Delegatory management does two things: it lowers the felt disutility of effort (enthusiasm for the mission, the persuasiveness of the leader) and it raises the value the agent places on customer satisfaction. Concrete levers named in the source: time taken from production to build group identity, and lifetime employment. Barnard's version: build a culture of beliefs and values in which people act in the name of the institution because they identify with it.
5. 5Price the investment. Delegation costs the principal real production time and commitments, and the cost grows at an increasing rate as you push the agent's allocation outward. Buy organizational identification up to the point where the cost of the next increment equals the value of the extra customer value it yields. Stop there.
6. 6Do not layer measurement-based incentives on top to squeeze out more. External motivators tend to crowd out internal motivation (Frey; Deci and Ryan; McGraw), and a measurement-based incentive carries an implicit message of distrust. Treat the two methods as alternatives and choose one per activity.
7. 7State the assumption you are rejecting. Reliance on measured incentives rests on rational self-interest, the belief that internal motivation is inadequate or irrelevant. If your own experience says people identify with the institution and sacrifice for it, write that down as the explicit premise for delegating.
8. 8Run delegation unsupervised and read the result. The source's argument is that delegation alone moves effort along the best-mix path and cannot produce measurement-style dysfunction. Judge it by customer value actually delivered, not by a proxy count.

The source's running case is an employment office. A measurement-based manager counts interviews, a substitute for the true output (value to the organization). The source notes that satisfaction with the office usually means many interviews are occurring, but a large interview count does not imply general satisfaction. Counting therefore pulls effort toward interviewing and away from referral work, an effort mix that distorts customer value. The delegatory alternative changes the agent's utility weights instead. The leader's persuasiveness and the mission lower the agent's disutility for effort and raise the value he places on customer satisfaction. In the model, his unsupervised allocation moves outward along the best-mix path, so interviews and referrals rise in proportion with no distortion. The principal pays for this in time spent on group identity and in commitments such as lifetime employment. She buys identification only until the cost of the next increment equals the value it adds. She does not then add a per-interview bonus, because that would signal distrust and risk crowding out the motivation she just paid for. Note that this is the book's model illustration, not a measured case.

• If the true output is intangible or can only be tested destructively (tear resistance, value to the organization), then rely on delegation and treat any measure as an imperfect substitute, not as the target.
• If agent utilities are aligned with customer value and the agent regards all effort as equivalent, then delegation alone keeps the allocation on the best-mix path, so no measurement is needed to prevent distortion.
• If the marginal cost of further identification-building (time, commitments) exceeds the marginal customer value it yields, then stop investing. Do not escalate with more culture-building, and do not patch the gap with measurement-based incentives.
• If you are tempted to add measured incentives on top of a trust-based approach, then expect crowding-out and a message of distrust. Choose one method for that activity.
• If you believe people act only for net monetary benefit, then you are on the measurement-based path. Say so explicitly, and accept that you are assuming internal motivation is inadequate.

• Treating the substitute as the goal. A team optimizes weight-and-thickness-style indicators while the true characteristic degrades. The tell: the numbers are green and the customer is still unhappy.
• Stacking incentives on a culture program. Leadership talks mission and vision, then ties pay to counts. The tell: enthusiasm drops after the scorecard arrives, and people ask what is being checked.
• Assuming delegation is free. Leaders promise identity and security without budgeting the production time and commitments. The tell: the cost keeps rising with no stated stopping point or benefit comparison.
• Delegating where agent utilities are not aligned. People dislike part of the work, so effort piles into the favored activity even with a shared mission. The tell: some tasks are done well and others are consistently neglected despite high morale.
• Using rational self-interest as an unexamined default. Contingent bonuses are the only lever considered. The tell: every proposal to improve cooperation is a payment schedule, and none addresses preferences, values, or trust.

• ▸Activity: the work being managed.
• ▸True output: what the customer actually values from it, in plain words.
• ▸Available proxy measures: list each, and note whether good performance implies a good number and whether a good number implies good performance.
• ▸Agent utility alignment: does the agent already value customer satisfaction? Does he dread any component of the work (such as interviewing versus referrals)?
• ▸Preference levers chosen: mission, leader communication, group identity time, employment commitments. Note which shift disutility for effort and which raise value on the customer.
• ▸Cost of each lever: production time and commitments, and how cost rises as you push further.
• ▸Stopping point: the level at which the next increment costs as much as the customer value it adds.
• ▸Measurement-based incentives present? (Y/N) If yes, note the risk of crowding-out and the distrust message, and decide which method wins here.
• ▸Evidence review: how you will judge customer value delivered without turning a proxy into the target.

1. 1Write down the model you are implicitly using. The Ross-Holmström model assumes the agent is motivated only by fondness for money and dislike of work. If your incentive design rests on that assumption, you have assigned internal motivation a value of zero. State this explicitly so you can test it against your own people.
2. 2Define internal motivation in observable terms before you try to protect it. Use the three components: the wish to perform well (pride in workmanship), the wish to contribute to organizational goals (identification with the group's mission), and the wish to provide value to the customer. For each, name one behavior you would see with no reward attached. Examples: reworking a deliverable nobody inspects, or flagging a defect that hurts your own numbers.
3. 3Classify every existing measure as motivational or informational. Motivational measures are meant to provoke more effort, for example bonuses, merit pay or promotion links. Informational measures are meant to convey logistical, status or research information. Informational measures are most accurate when people behave as if the measurement system did not exist, so keep them unlinked from reward and discipline.
4. 4For each motivational measure, write the spirit of the intention in one sentence and the letter (the number) in another. Dysfunction is a violation of the spirit even while the letter is met. If the two sentences describe different things, that measure is a candidate for dysfunction.
5. 5Ask the Schwab question of each measure: if people push this number up, what could be getting smaller or worse that the measure does not see? The source asks whether Bethlehem Steel workers made ingot smaller or lower in quality to hit a higher count. Name the specific unmeasured dimension (quality, customer value, effort allocation) and check it directly.
6. 6Check effort allocation, not only effort amount. In the Blau example, employment agents can work hard at interviewing, but nothing of value results if prospective employers are never contacted. List the task activities behind the outcome and ask whether the measure pulls effort toward some and away from others.
7. 7Compare two management postures for each role: measurement-based management and delegatory management. The source's table of contents lists both and notes a conflict between them, but the supplied text does not describe how delegation works. Decide role by role which you are using, and do not stack a heavy measurement-and-reward layer on top of a role you intend to delegate.
8. 8Record where people's behavior contradicts the money-and-effort-aversion assumption, such as extra effort without reward or concern for the customer beyond the measure. Treat it as evidence that internal motivation exists in that role, and review any proposed measure for its likely effect on that behavior before you launch it.

The source gives two accounts that look alike. At Bethlehem Steel, Charles Schwab chalked '78' on the hearth after counting ingot. The next day it read '80', then '85', and output kept rising. The common moral is that measuring focused workers on an important dimension and they excelled. At TWA in 1968, managers facing a major loss after sizable 1967 profits changed accounting methods and profits rose. Measurement again focused people on one dimension and the number went up. But the true mission was not served: time spent on accounting changes was wasted, and the profit figure became ambiguous to outsiders. Applying the construct, the difference you cannot see in the numbers is what the people were motivated by. The source raises the doubt itself: were the steelworkers making ingot smaller or worse in their haste? The passages do not say. The test is to look past the headline number, check the unmeasured dimension (ingot quality, or whether TWA's profit gain reflected real performance), and ask whether people were pursuing the mission or only the number. A rising metric alone cannot answer that.

• If a measure is meant to be informational (planning, process refinement, coordination), then keep it away from rewards and sanctions, because behavior change makes the data less representative.
• If a motivational measure can be met in letter while violating the spirit of the organizational intention (the TWA pattern), then redesign or drop it before relying on it to drive effort.
• If a measure captures only some of the activities that create value (the Blau contacts-versus-interviews case), then check how effort is split across activities before you reward the total.
• If your incentive logic assumes people work only for money and dislike work, then treat it as a simplification the source says behavioral scientists and real managers reject, and test it against what your people actually do.
• If observable pride in workmanship and mission identification are already present in a role, then weigh delegatory management against measurement-based management for that role instead of defaulting to more measurement.

• Assuming a rising number proves better performance. The tell: output rises but nobody has checked quality or customer value, as with the unchecked ingot size and quality.
• Using the Schwab story as proof that measurement always motivates. The tell: the only evidence cited is the increase in the measured quantity itself.
• Hitting the letter while missing the spirit. The tell: results are achieved by changing definitions or accounting rather than the work, as in TWA's profit change.
• Importing the economic model's picture of the worker (motivated only by money, averse to work) into a real team. The tell: the design contains no place for pride, mission or customer concern, and managers describe staff only in terms of payoffs.
• Mixing informational and motivational uses in one measure. The tell: people who feed data into a planning or process measure start managing how it looks, and the data stops matching events.

• ▸Measure name and owner: name the measure and who is responsible for it.
• ▸Intended use: motivational (changes effort), informational (logistics, status, research), or both; if both, flag it.
• ▸Spirit of the intention (one sentence): what the organization actually wants achieved.
• ▸Letter of the measure (one sentence): what number or threshold counts as success.
• ▸Gap test: could the letter be met while the spirit is violated? Describe how.
• ▸Unmeasured dimension at risk: what could get worse (quality, customer value, task coverage) if people push the number?
• ▸Effort allocation check: list the task activities behind the outcome and mark which the measure rewards or ignores.
• ▸Internal motivation evidence: observed behavior showing pride in workmanship, mission identification or customer concern with no reward attached.
• ▸Management posture decision: measurement-based or delegatory for this role, with a one-line reason.

1. 1Start from the premise that your stated intent is not believed. The passages say designers cannot guarantee how measurement information will be used once a system has been in place for a while. Unless trust is greater than usual, claims that data will be used only one way are not credible. Treat trust as something you build and test before you rely on any informational measure.
2. 2Write down the intended use of each measure: informational (understanding the work) or motivational (rewards and punishments). Then write down the use a worker could reasonably fear. The gap between the two is the trust deficit you have to close. Informational data such as 'is the project behind schedule' can be used either way regardless of intent.
3. 3Read your measurement system the way a worker does. Ridgway's point is that even purely informational measures are interpreted as definitions of the important parts of the job. List what each measure signals as important, and who could see it. Workers have a justifiable interest in knowing what the people who control rewards care about.
4. 4Remove the implicit message of distrust where you can. The existence of a measurement system conveys distrust, and external rewards can crowd out internal motivation. For each measure, ask whether the work is already done out of pride or workmanship. If it is, weigh the cost of measuring it, because measurement-based and delegatory management conflict.
5. 5Make your good faith costly to fake and visible to check. Liker et al. find Japanese firms deliberately create reciprocal vulnerabilities with suppliers, so that everyone is exposed and no one exploits. Look for a version in your setting. Decide what information you will expose to workers, or what you will forgo using against them, so that you are vulnerable too.
6. 6Pursue Barnard's route in parallel: honest efforts to promote trust and identification with the organization, not only incentive design. The aim is a culture where people act for the institution because they identify with it. Plan this as ongoing leadership work, not a one-time announcement.
7. 7Watch for the trust-failure signature: measures that rise while real performance does not. The TWA and Bethlehem Steel stories show output numbers moving for reasons that may have nothing to do with the mission. Check periodically whether the measured quantity and the true goal have separated. When they have, assume the validity problem comes from workers' reactions and review the trust conditions first.
8. 8Report the same pattern after any measure is introduced: who learned what the measure means, and who changed behavior in response. Use this to decide whether a measure stays informational, becomes motivational, or is dropped.

Charles Schwab walked through his Bethlehem Steel plant, counted the ingot a crew had poured, and chalked '78' on the hearth without saying anything. The next day '78' was crossed out and '80' written beside it, then '85' the day after. Output rose. The measure was silent about its use, so the crew supplied the meaning themselves, reading the number as something the boss cared about and competing against it. That is Ridgway's point in action: a measure is read as a definition of what matters. The passage also raises the doubt a trust-minded manager has to confront: were employees making ingot smaller or lower in quality in their haste to impress Schwab? Nothing in the story shows Schwab could answer that, because the number was the only channel between him and the crew. TWA shows the same pattern in accounting form: managers facing a loss extended the depreciable life of assets, the profit measure rose, and true performance did not. In both cases the measure moved while the mission may not have. A manager applying this construct would state the use of the count in advance, say what would not be done with it, and check quality alongside it. Without trust, even those commitments would be discounted.

• If you cannot credibly commit to how a measure will be used, then assume workers will treat it as motivational and design for that, rather than relying on your stated intent.
• If the work is already driven by pride, craft, or collegial enjoyment, then be wary of adding measurement-based rewards, because they can crowd out internal motivation and signal distrust.
• If a measure is meant to be purely informational, then make that credible with visible constraints on its use, because declarations alone are not believed unless trust is greater than usual.
• If a measured number improves while you cannot confirm the underlying mission improved (the TWA and Bethlehem pattern), then treat the data as suspect and review trust and incentives before acting on it.
• If you measure activity in one area, then check for spillover: Frey finds it can damage internal motivation in other areas too.

• Announcing 'this is only for information' and expecting belief. The tell: workers act as if the data will be used for evaluation anyway, preparing for that possibility.
• Ignoring what the measure design itself communicates. The tell: people optimize whatever the system seems to define as important, including accounting changes that raise the number but not performance.
• Layering incentives onto already-motivated work. The tell: people stop showing the pride and enjoyment they previously showed, as in DeMarco's remark that extrinsics shove the intrinsics aside.
• Assuming trust is only a moral matter and adding it last. Austin's argument is that trust and honesty are more efficient than verification in many contexts, so it is a design input for measurement validity.
• Measuring in ways that foster internal competition. The tell: comparability breeds unfriendliness between workers and erodes common purpose (Kohn).

• ▸Measure and owner: name the measure and who collects it.
• ▸Intended use: informational or motivational, stated in one sentence.
• ▸Feared use: the worst reasonable use a worker could assume, given who sees the data.
• ▸What the measure signals: what workers would infer you consider important (Ridgway test).
• ▸Distrust message: what the existence of this measure implies about management's view of the worker.
• ▸Internal motivation at risk: is the work already done from pride or workmanship? If yes, note the crowding-out risk.
• ▸Reciprocal exposure: what management exposes or forgoes using, so that vulnerability is mutual.
• ▸Validity check: how you will detect the number rising while real performance does not.
• ▸Spillover and competition check: effects on other work areas and on cooperation among those compared.