Handbook of Marketing Scales Multi-Item Measures for Marketing and Consumer Behavior Research

1. 1Write the construct definition first. State what the domain includes, what it excludes, and the a priori dimensionality. You cannot say whether an item is surplus until the domain is fixed.
2. 2Classify breadth. A wide domain, or one conceptualized as multidimensional, needs more items. A very narrowly defined construct may be measurable with a single item, per the Bergkvist and Rossiter (2007), Drolet and Morrison (2001), and Wanous, Reichers, and Hudy (1997) evidence. Record the call as a hypothesis, not a preference.
3. 3Generate a deliberately larger pool than you plan to keep. Have judges with literature expertise screen it, then run several pilot tests on samples from the relevant population to trim and refine. Brevity is reached by trimming a pool that covers the domain, not by writing few items at the start.
4. 4Write each item to be short and simple. Remove jargon, ambiguous or difficult wording, and double-barreled structure. Shorter, simpler items are easier to respond to and more reliable, so tightening wording cuts burden without cutting coverage.
5. 5Check dimensionality before and after every cut. Unidimensionality is a prerequisite to reliability and validity. Use confirmatory factor analysis, where several multi-item factors and their relations can be specified, and review fit indices, cross-loadings, correlated errors, and method factors. The empirical structure must still match the theorized structure.
6. 6Apply explicit retention rules and log each deletion with its reason. Use a loading threshold (the MAS retained items loading 0.40 or higher on a factor) and a theoretical-overlap test (a factor that duplicates another's content is removed).
7. 7After trimming, re-check reliability. Internal consistency is the usual estimate, and test-retest is offered by fewer than half of the handbook's scales. Then re-check construct validity: convergent, discriminant, nomological, and known-group. A shorter scale has to pass the same tests as the longer one.
8. 8Report the final length alongside the content-coverage argument. State why this number of items covers this domain, and why the construct is broad or narrow enough to justify it.

The Money Attitude Scale (Yamauchi and Templer 1982) shows trimming driven by the domain. The authors defined three content areas: security, retention, and power-prestige. They generated 62 items to reflect them. Factor analyses trimmed the pool to 34 items on five factors, keeping only items with loadings of 0.40 or above; those five factors accounted for 33.6% of the variance. The five factors were power-prestige, retention-time, distrust, quality, and anxiety. The quality factor was then deleted because of theoretical overlap with power-prestige. The final scale has 29 items on four factors, scored by summing within each dimension, with an optional total across all 29 items. The path ran from 62 to 34 to 29, and every cut had a stated statistical or conceptual reason. The contrast case is the Product Retention Tendency scale (Haws et al. 2010), which uses four items on a 7-point agree scale for a narrow construct. Breadth sets length: a four-factor money-attitude domain needs many items, and a tight retention tendency needs few.

• If the construct is narrowly defined and concrete, then test a single-item measure against a multi-item version before committing to the longer form.
• If the construct is wide or multidimensional, then keep enough items to cover every dimension, and do not cut to a target length.
• If an item loads below your stated threshold (0.40 in the MAS example) on its intended factor, then it is a deletion candidate. Confirm that dropping it leaves the domain covered.
• If a whole factor overlaps theoretically with another factor, then delete or merge it. This is what happened to the MAS quality factor and power-prestige.
• If an item is long, jargon-laden, ambiguous, or double-barreled, then rewrite it before deleting it. Simpler wording is the cheaper way to shorten the respondent's task.

• Cutting to hit a length target. The tell is that a defined subdomain or dimension now has no item.
• Using a single item for a broad or multidimensional construct. The tell is that the item can be read several ways and the theorized dimensions cannot be recovered.
• Trimming on loadings alone without re-running dimensionality. The tell is that the factor structure after trimming no longer matches the theorized structure.
• Starting with too small a pool. The tell is that expert judges and pilot tests leave you nothing to trim, and coverage gaps appear only at the validation stage.
• Shortening the scale and skipping the validity checks. The tell is that reliability is reported only as internal consistency, with no convergent, discriminant, nomological, or known-group evidence for the short form.

• ▸Construct definition: state what the domain includes and excludes in one or two sentences.
• ▸Hypothesized dimensionality: uni- or multidimensional, and the named dimensions.
• ▸Breadth call: wide, moderate, or narrow, and whether a single item is a candidate.
• ▸Initial pool size: number of items generated, and who judged them.
• ▸Item ID and wording: one row per item, with a note on jargon, ambiguity, or double-barreling.
• ▸Loading and retention threshold: the empirical cutoff used (e.g., 0.40) and each item's result.
• ▸Deletion reason: statistical, theoretical overlap, wording, or none, for every dropped item.
• ▸Post-trim dimensionality check: method used (e.g., CFA), fit indices, cross-loadings, and whether the structure matches the theory.
• ▸Final length and coverage argument: final item count per dimension, with a sentence on why it covers the domain.

1. 1Name the population of interest before you draw any sample. Write down who will use the scale in the future (the whole population, or a given population of interest). The source states the prime consideration is the applicability of the scale and its norms to respondents who are likely to use them later.
2. 2Treat college students as one sample, not the only sample. The source does not advocate discontinuing student samples. It recommends going beyond them to samples more representative of the population as a whole or of the population of interest.
3. 3Use students where they are cheap and fast: pilot testing and item refinement. Then validate on a non-student sample. In the Consumer's Need for Uniqueness (CNFU) work, the student sample refined the item set to the final 31 items, and a mail sample of adult consumers was used for further validation.
4. 4Run the pilot tests on samples from relevant populations, not only convenient ones. The source recommends several pilot tests to trim and refine the item pool, and the samples for those tests should come from the relevant populations.
5. 5Collect the same reliability evidence in every sample and compare it side by side. For CNFU, internal consistency was 0.94 in the student sample and 0.95 in the consumer mail sample. Similar values across samples support generalizing; a gap tells you the scale behaves differently outside students.
6. 6Check stability in the non-student sample too. CNFU test-retest correlations came from consumer mail subsamples re-surveyed 1 and 2 years later: 0.81 (n=84) and 0.73 (n=346).
7. 7Add known-group samples drawn from the populations where the construct should differ. CNFU used tattoo artists (n=39), low-rider auto owners (n=22), and a medievalist reenactment group (n=21), among others. Scores that differ in the predicted direction in those groups support validity beyond students.
8. 8Report the sample mix explicitly for every study (students vs. nonstudents vs. panel, and n). The Temporal Focus Scale shows the format: graduate students, a student/nonstudent mix, students, and a national online survey panel. Readers can then judge where the norms apply.

The Consumer's Need for Uniqueness (CNFU) scale (Tian, Bearden, and Hunter 2001) shows the pattern. An initial pool of 93 items was cut to 62 after two stages of judging for content validity, and redundant items were then removed, leaving 15 per dimension. The student sample (273 undergraduate business students) was used to refine the set to the final 31 items, deleting items with corrected item-to-total correlations below 0.50 or higher correlations with the other two factors. A second large sample of 621 adult consumers was collected by mail after telephone pre-contact. Internal consistency was 0.94 in the student sample and 0.95 in the consumer mail sample, so the scale held up outside the classroom. Subsamples from the consumer mail surveys, 1 and 2 years later, gave test-retest correlations of 0.81 (n=84) and 0.73 (n=346). Five known-group samples (tattoo artists n=39, low-rider owners n=22, medieval reenactors n=21, and others) added evidence from groups expected to differ. The student sample did the refinement work, and the non-student samples carried the generalization claim.

• If every sample in your validation is college students, then do not claim general-population norms; add at least one sample from the population of interest, or limit the claim explicitly to students.
• If students are the only practical sample for item refinement, then use them for that stage and validate reliability and structure on a non-student sample before finalizing.
• If alpha or factor structure differs materially between student and non-student samples, then treat norms as sample-specific and report them separately.
• If the scale will be used on a specific group (e.g., salespeople in a given industry), then sample that group, because applicability to likely future users is the prime criterion.
• If you have a theory about groups that should differ on the construct, then recruit them as known-group samples, even small ones, to test validity outside students.

• Student-only development: every study in the scale's record uses undergraduates. The tell is a sample table with no nonstudent row, yet claims about consumers in general.
• Using students for everything, including validation, because they are available. The tell is that refinement and validation come from the same convenience pool, so the reliability estimates cannot show generalization.
• Overcorrecting by discarding student data. The source says it does not advocate dropping student samples. The tell is that cheap early-stage item work is skipped and the pilot budget goes to an expensive broad sample too soon.
• Reporting norms without saying who they came from. The tell is a mean or cutoff with no description of the sample mix (students vs. nonstudents, n), so users cannot judge whether it applies to them.
• Assuming a result generalizes because alpha is high in one sample. The tell is a single reliability figure with no second-sample comparison, such as CNFU's 0.94 vs. 0.95 across student and consumer samples.

• ▸Population of interest: who will use this scale and its norms in the future?
• ▸Study/stage: pilot, item refinement, dimensionality, reliability, validity, or known-group.
• ▸Sample type and n: students, adult consumers, panel, occupational group, with the count.
• ▸Recruitment method: how respondents were reached (mail, online panel, classroom) and any pre-contact.
• ▸Match to population: how closely this sample resembles the population of interest, and the main gaps.
• ▸Reliability in this sample: coefficient alpha and any test-retest value, with the interval.
• ▸Comparison with student sample: differences in reliability, factor structure, or means.
• ▸Known-group samples used: groups expected to differ and the predicted direction of difference.
• ▸Scope of claim: where norms and relations can and cannot be generalized, given the above.

1. 1Define the construct and its domain before touching data. Write down what the construct means in each target culture, because the source warns that applying a measure abroad without checking 'the meaning of the construct' leads to 'erroneous inferences of a construct's meaning, applicability, and relation to other constructs.' If the construct has no plausible counterpart in a culture, stop there.
2. 2Trace the scale's origin. Record the countries and sample types it was built and validated in. The Handbook flags that many scales come from US researchers and student samples. Treat a scale with only US or single-country development evidence as unvalidated elsewhere, however good its reliability.
3. 3Check content validity in each culture. Have local judges rate every item against the construct definition, as the LTO developers did when marketing faculty judged items against a definition of national culture. Flag items whose wording, idiom or social meaning does not carry over, and rewrite or drop them before fielding.
4. 4Run confirmatory factor analysis separately in each country sample. Specify the hypothesized dimensionality (for example, LTO's tradition and planning factors, four items each). Compare fit, cross-loadings, correlated errors and method factors across countries. A scale that is unidimensional or two-factor in one country but not in another is not equivalent.
5. 5Run formal equivalence tests using the procedures the Handbook points to: Steenkamp & Baumgartner 1998, Durvasula et al. 2006, Hult et al. 2008, Rossiter 2002, and Weathers & Sharma 2008. Do this before comparing means or structural relations across countries.
6. 6Test for response set bias by country. The Handbook names acquiescence (yea-saying and nay-saying), extreme response style and social desirability as sources of scores that do not reflect the construct. Examine whether these styles differ across samples and whether they contaminate the comparison. Use the remedies in the cited work (Baumgartner & Steenkamp 2001; de Jong et al. 2007; Steenkamp et al. 2010).
7. 7Match samples. Prefer samples that represent the population of interest in each country, not student-only samples in one place and managers in another. The Handbook's standard is applicability 'to respondents who are likely to use' the scale norms. Record sample type per country so a sampling difference is not mistaken for a cultural one.
8. 8Report country-specific norms: means, item means and standard deviations per group. Interpret cross-country mean gaps only after equivalence is supported, and publish the norms so users have a frame of reference.

Campbell et al. (1996) report Self-Concept Clarity (SCC) means for Canadian and Japanese subsamples of Study 3. The scale has 12 items on 5-point scales. Canadian means are 41.72, 39.30 and 38.02. Japanese means are 34.41, 35.01 and 34.35. The Canadian samples score 7.31, 4.29 and 3.67 points higher. A naive reading says Japanese respondents have less clear self-concepts. Under the Handbook's logic, that reading is allowed only if the SCC items mean the same thing and are answered the same way in both groups. The article's subtitle, 'Cultural Boundaries,' signals that the authors treated culture as a limit on the construct. Several items ask about self-description varying from day to day or conflicting beliefs about oneself. You would check whether the same one-factor structure holds in each sample. You would check whether item wording is read equivalently. You would also check whether response styles such as acquiescence or extreme responding differ, since items 6 and 11 are worded in the opposite direction from the other ten. The gap is a finding about the construct only if those checks pass. Otherwise it may reflect measurement differences.

• If a scale was developed and validated in only one country, then treat cross-national use as untested and run the equivalence procedures before using it elsewhere.
• If the CFA factor structure differs across countries (dimensionality, cross-loadings, correlated errors), then do not compare means or structural paths. Revise the scale or analyze each culture separately.
• If equivalence is not established, then limit conclusions to within-country relationships. Do not interpret a cross-country mean gap as a true difference in the construct.
• If response style indicators (acquiescence, extreme responding, social desirability) differ by country, then address them before comparing scores. A gap may be a style effect, not a construct effect.
• If the country samples differ in type (students in one, general adults in another), then fix the sampling or report the confound before attributing differences to culture.

• Translating or administering the scale and treating the job as done. The tell: no analysis in the report assesses whether the construct's meaning or the measure is equivalent in the new country.
• Reading raw mean differences as cultural differences, like the Canadian-Japanese SCC gap. The tell: a conclusion about 'higher' or 'lower' in a culture with no equivalence test and no country norms behind it.
• Assuming the original factor structure holds because reliability is acceptable abroad. The tell: Cronbach's alpha is reported but no per-country CFA appears. Reliability does not establish dimensionality.
• Ignoring response style. The tell: cross-country differences concentrate in scale endpoints or in agreement with both positively and negatively worded items, and no style check was run.
• Comparing non-matched samples across countries. The tell: one country sample is students or a convenience sample and another is a broader population, and cultural differences are claimed anyway.

• ▸Construct and target cultures: name the construct and list each country or culture being compared.
• ▸Origin of scale: record where and in which sample types the scale was developed and validated.
• ▸Construct meaning check: note how the construct's meaning and applicability were confirmed in each culture, and by whom.
• ▸Content validity review: record the local judges, the definition used, and items flagged or revised per culture.
• ▸Dimensionality per country: record the hypothesized factors and CFA results (fit, cross-loadings, correlated errors, method factors) for each sample.
• ▸Equivalence procedure used: name the method followed (for example, Steenkamp & Baumgartner 1998) and the outcome.
• ▸Response set bias check: record acquiescence, extreme response and social desirability results by country, and any correction applied.
• ▸Sample comparability: record sample type and size per country and any mismatches.
• ▸Norms by country: list scale means, item means and standard deviations for each group.
• ▸Inference limit: state what may and may not be concluded across countries given the results above.

1. 1Name the bias risk before fielding. The passages list three response-set sources: acquiescence/disacquiescence (yea-saying and nay-saying), extreme response style (disproportionate use of the high or low scale endpoints), and social desirability (making a good impression). Write down which of the three your construct, wording, and sample make plausible.
2. 2Audit item keying and item content. Mark every item as agree=high or agree=low, then mark whether agreeing sounds good or bad. A scale whose items all point the same way and all sound good cannot separate content from going along with anything that sounds good. Reverse-key or rebalance where the construct permits.
3. 3Flag socially loaded constructs. Respondents may purposefully score low on undesirable characteristics (the passage's example is selfishness) and high on desirable ones (altruism). If your construct sits at either pole, plan a social desirability check and name which scale or procedure you will use.
4. 4Map your data collection modes. The passage warns that the tendency to use scale extremes may be more endemic to one mode (web, mail, phone) than another. If modes are mixed, treat any mode difference in means, variances, or covariances as a possible artifact until tested.
5. 5Apply an assessment procedure instead of eyeballing. The passages point to de Jong et al. (2007) and Weijters et al. (2008) for assessing various response styles, including extreme response style. Paulhus (1991) and Robinson et al. (1991) are the cited sources for recommended acquiescence procedures. The passage says elimination of acquiescence has no easy answers, so plan to assess and report it, not to claim it is gone.
6. 6Test the relationships, not only the means. Social desirability can bias relationships among variables, especially variables with a higher propensity for desirable responding (Manning, Bearden, and Tian 2009; Mick 1996; Steenkamp et al. 2010). Check whether your key correlations change after the bias estimate is accounted for.
7. 7Report normative information. A raw score is not informative about a person's position on the construct because scale units are often interval and unfamiliar. Report scale means, item means, and standard deviations per sample group, per mode, and per country so a reader can see where bias would shift them.
8. 8For cross-national use, establish measure equivalence first (Steenkamp and Baumgartner 1998). Response style differences can otherwise be mistaken for construct differences.

Take the Long-Term Orientation scale (Bearden, Money, and Nevins 2006), eight items such as 'Respect for tradition is important to me,' 'I plan for the long term,' 'I work hard for success in the future,' and 'Persistence is important to me.' Every item is worded so that agreement signals high LTO, and each states something that sounds admirable. A yea-saying respondent, or one trying to look good, would therefore score high without holding the orientation. That is the response set problem: the score no longer reflects how the respondent stands on the construct. The passage notes the scale was validated with known-group comparisons (younger vs. older respondents, U.S. vs. Japanese respondents). Those are the comparisons where response style and cultural differences in endpoint use can masquerade as construct differences. The expert response is to audit keying and desirability as above, then run the de Jong et al. (2007) or Weijters et al. (2008) response-style assessment on the cross-group data. Finally, report means and SDs by group and check that the hypothesized group difference and the moderation result persist once the response-style estimate is accounted for. This example is the analyst's inference from the printed items, not a finding reported in the source.

• If all items are keyed in one direction and all sound favorable, then assume acquiescence and desirability confounds are possible and test for them before interpreting high scores.
• If the construct is a socially desirable or undesirable trait (altruism, selfishness), then include a social desirability scale or procedure and check whether it changes the construct's correlations.
• If data come from more than one mode or country, then do not compare raw means, variances, or covariances until you have assessed extreme response style and equivalence.
• If a prior study argues social desirability effects are overstated (Moorman and Podsakoff 1992), then still test: others find the effect can bias relationships among variables for high-propensity constructs.
• If you cannot run a bias procedure, then report scale and item means and SDs by group and state the limitation instead of treating raw scores as clean.

• Treating a high alpha as proof of clean measurement. Consistent yea-saying also raises inter-item correlations. The tell is a uniformly agree-keyed scale with strong internal consistency and no bias check.
• Pooling modes without checking. The tell is a mode difference in group means or variances that shrinks or vanishes once response style is assessed.
• Dismissing social desirability because one paper says it is overstated. The tell is a trait-like, morally loaded construct (e.g., altruism) with no desirability check and a surprisingly clean correlation pattern.
• Reporting only raw scores. The tell is a results section with no means or SDs by group, so readers cannot see whether endpoint use differs.
• Reading cross-cultural differences as substantive. The tell is a country gap on a favorable-sounding scale with no equivalence or response-style test.

• ▸Construct and scale: name the scale and list the number of items.
• ▸Keying map: count items where agree=high vs. agree=low; flag a one-direction scale.
• ▸Desirability of content: mark each item as sounding favorable, unfavorable, or neutral, and note any selfishness/altruism-type poles.
• ▸Collection modes and groups: list each mode (web, mail, phone) and each country or sample group in the data.
• ▸Bias types in play: tick acquiescence, extreme response style, social desirability, and give one sentence of evidence for each tick.
• ▸Assessment procedure chosen: cite the method used (e.g., de Jong et al. 2007; Weijters et al. 2008; Paulhus 1991; Robinson et al. 1991) and the social desirability scale or procedure.
• ▸Group means and SDs: record scale and item means and standard deviations per mode and per group.
• ▸Relationship check: record key correlations before and after accounting for the bias estimate, and note any change.
• ▸Conclusion and limitation: state whether scores are interpretable as-is, and write the residual limitation to report.

1. 1Pull the original source article for each candidate scale and record its publication year. Age sets how much citation weight you can fairly demand, because the Handbook notes that recent scales have not had time to be heavily cited.
2. 2Count citations to the original article in two sources, the Social Science Citation Index and Google Scholar, as the Handbook editors did. Log both counts and the date you checked, since the counts differ by source.
3. 3Apply the Handbook's age-and-count screen: a scale in existence at least 20 years with fewer than 50 total citations is a candidate for deletion, meaning you drop it from consideration. Treat this as a screen for review, not an automatic verdict.
4. 4Check length as a second, independent criterion. The Handbook cites a trend toward short scales that are both reliable and valid (De Jong, Steenkamp, and Veldkamp 2009; Netemeyer, Pullig, and Bearden 2002; Richins 2004; Stanton et al. 2002). Flag any scale that is excessively long or cumbersome.
5. 5Combine the two criteria scale by scale. A scale that is both lightly cited and excessively long is the strongest drop. A scale failing only one criterion needs a judgment call.
6. 6Weigh adoption evidence beyond the raw count: look for repeated use across multiple studies. The Handbook treats citation count as a proxy for importance and usefulness, not as proof of quality, so confirm the scale's psychometrics (dimensionality, reliability, construct validity) separately.
7. 7For recent scales (published roughly 1998-2010 in the Handbook's addition window), do not penalize low counts. Judge them on development quality and fit with your construct, and note the limitation in your write-up.
8. 8Document your own judgment calls explicitly. The editors state that their judgment 'came heavily into play' in applying both criteria, so record why you kept or dropped each borderline scale.

The Handbook of Marketing Scales (3rd ed., Bearden, Netemeyer, and Haws 2011) applied this construct to decide which second-edition scales to delete. For every second-edition scale, the editors looked at the citation count of the article in which the scale first appeared, using both the Social Science Citation Index and Google Scholar as a proxy for importance and usefulness to researchers. They factored in publication year because recent scales have had less time to accumulate citations. Their stated rule: a scale in existence at least 20 years with fewer than 50 total citations was a candidate for deletion. They then added a second screen on length, citing evidence favoring short scales that are reliable and valid. For most deleted scales, a low citation count and excessive length coincided. The editors also say their own judgment 'came heavily into play,' so the rule guided decisions rather than mechanically making them.

• If a scale is 20 or more years old and has fewer than 50 total citations, then treat it as a deletion candidate and look for a better-adopted alternative.
• If a scale is recent, then do not read a low citation count as low usefulness; evaluate development quality and fit instead.
• If a scale is both lightly cited and excessively long, then drop it; if it fails only one test, then decide case by case and record the reason.
• If two scales measure the same construct, then prefer the one that is more widely adopted across studies, provided its reliability and validity evidence holds.
• If the Social Science Citation Index and Google Scholar counts diverge, then report both and base the decision on the pattern, not on the more favorable number.

• Treating citation count as a quality verdict: a heavily cited scale is chosen without checking dimensionality, reliability, or validity. The tell is that you cannot cite its psychometric evidence.
• Ignoring scale age: a 2009 scale is rejected for having few citations. The tell is that you never recorded the publication year beside the count.
• Relying on a single citation source: the count comes from one database only. The tell is that you have no second count or retrieval date.
• Letting length drive the decision while ignoring adoption, or the reverse. The tell is that one criterion is absent from your notes.
• Applying the rule mechanically and hiding judgment calls. The tell is that borderline scales have no written rationale.

• ▸Scale name and original source: record the scale name and the citation of the article where it first appeared.
• ▸Publication year and age: enter the year and compute years in existence, to compare against the 20-year threshold.
• ▸Citation count, Social Science Citation Index: enter the count and the date retrieved.
• ▸Citation count, Google Scholar: enter the count and the date retrieved.
• ▸Meets deletion screen (20+ years and fewer than 50 citations)?: answer yes or no, using the larger of the two counts.
• ▸Number of items and length concern: enter the item count and note whether it is excessively long or cumbersome.
• ▸Evidence of repeated use across studies: list the other studies that used the scale, beyond the citation count.
• ▸Psychometric check: note dimensionality, reliability, and construct validity evidence, since adoption alone is not enough.
• ▸Decision and judgment rationale: write keep, drop, or review, with one sentence on why.

1. 1Write the theoretical definition before drafting any item. The passages require a 'solid theoretical definition' with the domain 'thoroughly delineated and outlined', so the definition comes first and the items follow from it.
2. 2Make the definition do three jobs in writing: state what is included in the domain, state what is excluded from it, and state the a priori dimensionality. If one of the three is missing, the delineation is incomplete.
3. 3Derive the definition, the domain boundaries, and the dimensionality from a thorough review of the existing literature. Record which sources set each boundary so every inclusion and exclusion has a traceable origin.
4. 4Add expert opinion where you can ('ideally, expert opinion'). Have judges with expertise in the literature screen the draft definition and the item pool against it. Have them also check whether you have specified dimensionality correctly.
5. 5Commit to uni- or multidimensionality before collecting data. A construct's domain 'can be hypothesized as uni- or multidimensional', and the scale or subscales must reflect that hypothesis. Name each hypothesized dimension and say what it covers.
6. 6Let the definition set the item budget. Wide-domain or multidimensional constructs need more items. Very narrowly defined constructs may be adequately served by a single item. Choose the count only after you know how broad or narrow the construct is.
7. 7Write items to be representative of the defined construct: brief, jargon-free, unambiguous, and not double-barreled. Check that each one looks consistent with the theoretical domain (content and face validity).
8. 8Test the empirical factor structure against the dimensionality you theorized. Confirmatory factor analysis is the 'somewhat agreed' technique. A mismatch means revisiting the definition or the items, not only the statistics.

The Long-Term Orientation (LTO) scale (Bearden, Money, and Nevins 2006) shows definition driving dimensionality. The construct is defined with tradition and planning as its two dimensions. An initial pool of 59 items came from 292 MBA-graduate responses and was supplemented with items from similar cultural-values measures. The pool was edited to remove double-barreled, redundant, ambiguous, and misleading statements. Marketing faculty in the AMA Global SIG then judged the items against a general definition of national culture, which left nine items. Factor analyses on four country samples yielded eight items, four per factor, matching the hypothesized planning and tradition structure. Confirmatory factor analyses on a two-country sample confirmed the two-factor structure. The factors correlated 0.28 to 0.44, so they are related but distinct. Average alphas were 0.77 for tradition and 0.60 for planning, a reminder that matching the intended dimensionality does not guarantee equal reliability across dimensions.

• If you cannot state what the construct excludes, then the definition is unfinished. Do not write items until you can.
• If the construct is broad or multidimensional, then plan more items per dimension. If it is very narrow, then consider a single item and weigh length against parsimony.
• If you hypothesize multiple dimensions, then specify them before data collection. Treat a factor structure that departs from them as a signal to revisit the definition, not as a result to rationalize.
• If expert judges say an item does not fit the stated domain, then drop or rewrite it, even if it has good statistical properties.
• If an item is double-barreled, jargon-laden, or ambiguous, then fix or delete it before pilot testing. Brevity and simplicity improve reliability.

• Defining only what the construct is. The tell is that nobody can say which neighboring concepts are out of scope, and items drift into adjacent domains.
• Letting factor analysis decide dimensionality after the fact. The tell is a write-up that describes the factors only once the data show them, with no earlier hypothesis.
• Skipping the literature review and expert screening. The tell is a definition with no cited sources and no judges who evaluated the items against it.
• Fixing item count by convention instead of by domain breadth. The tell is a broad construct covered by a few items, or a narrow one padded with redundant items.
• Writing long, jargon-filled, or double-barreled items. The tell is that respondents misread them and reliability suffers.

• ▸Construct name and one-sentence theoretical definition: state the definition in your own words and cite its source.
• ▸Included in domain: list the facets, behaviors, or perceptions that belong in the construct.
• ▸Excluded from domain: list the neighboring concepts and facets that do not belong, with a reason for each.
• ▸A priori dimensionality: state unidimensional or multidimensional. If multidimensional, name each dimension and define it.
• ▸Literature basis: list the sources that support each boundary and each dimension.
• ▸Expert review: record who the judges are, what they assessed (definition, domain, items), and what changed as a result.
• ▸Item budget rationale: state how broad or narrow the domain is and the item count it justifies per dimension.
• ▸Item screening checklist: confirm each item is brief, jargon-free, unambiguous, not double-barreled, and consistent with the domain.
• ▸Dimensionality test plan: name the confirmatory factor analysis you will run and the hypothesized structure it will test.

1. 1Write the construct definition and its theoretical base first, before any item exists. The handbook's first inclusion test is 'a reasonable theoretical base and/or conceptual definition.' Name the dimensions you expect and state what the construct excludes.
2. 2Generate items from the target population, not only from your desk. The Consumer Self-Confidence (CSC) scale began with an item-generation sample of 43 adults. Pair this with items derived a priori from the definition, so each expected dimension is covered.
3. 3Make the pool larger than the final scale. Allow for losses at every later stage. The CSC pool shrank from the screened 97 items to 39 final items.
4. 4Edit the raw pool for wording defects before judging. The CSC authors deleted redundant, leading, and ambiguous items. Remove duplicates, items that suggest an answer, and items open to more than one reading.
5. 5Run a judgmental screen for item representativeness. Have judges rate each remaining item for relevance to the definition and for which dimension it belongs to. Fix the retention rule before reading ratings. CSC's screening produced the 97-item pool carried into data collection.
6. 6Pilot the screened pool on real samples and purify it statistically. CSC used corrected item-to-total correlations, average factor loadings, and interitem correlations on two nonstudent adult samples (N = 221 and 204). That cut the pool to 39 items.
7. 7Confirm dimensionality on a fresh sample. CSC ran confirmatory factor analyses on Study 3 (252 undergraduates), which gave a six-factor correlated model. Construct reliabilities ran from 0.81 to 0.88. Do not call a pool content-valid until the structure you theorized appears in new data.
8. 8Add validity evidence beyond content. CSC reported test-retest correlations of 0.60 to 0.84 (Study 4), convergence with single-item self-ratings, spouse convergence (Study 5), and impression-management tests. Content validity alone does not meet the handbook's standard, which excludes scales justified by face validity alone.

The Consumer Self-Confidence scale (Bearden, Hardesty, and Rose 2001) shows the full sequence. The authors started with an item-generation sample of 43 adults. They deleted redundant, leading, and ambiguous items, then applied judgmental screening for item representativeness, which left 97 items. Studies 1 and 2 (221 and 204 nonstudent adults) supplied data for purification. Corrected item-to-total correlations, average factor loadings, and interitem correlations cut the pool to 39 items. Study 3 (252 undergraduates) confirmed a six-factor correlated model with construct reliabilities of 0.81 to 0.88. Study 4 added test-retest correlations of 0.60 to 0.84 and convergence with single-item self-ratings. The tell of rigor is the order: judgment came before statistics, and the structure was confirmed on a different sample from the one used to prune items. The passage does not report how many judges rated the items or what cutoffs they used, so set and record those yourself.

• If an item is redundant, leading, or ambiguous, delete it before judging, so judges rate only usable wording.
• If judges find an item does not represent any dimension of your definition, drop it, even when its statistics look strong.
• If a dimension ends up with few judged-representative items after screening, return to generation for that dimension. Do not rely on statistical pruning to fix coverage.
• If the confirmatory structure from the new sample does not match the dimensions you theorized, revisit the construct definition and items. Do not relabel the factors after the fact.
• If your only support for a multi-item scale is that it looks right, treat it as unvalidated. The handbook excludes face-validity-only scales.

• Writing items without a written definition. The tell: you cannot say which dimension an item belongs to, and the factors are named only after the analysis.
• Generating items only from the researcher's own thinking. The tell: the pool reads like the definition restated, with no wording from the target population.
• Letting statistics replace judgment. The tell: items are chosen only by item-to-total correlations, and a narrow cluster of near-duplicate items survives.
• Purifying and confirming on the same data. The tell: strong fit in the sample used for pruning, with no separate confirmatory sample.
• Starting with too small a pool. The tell: after the screening and purification losses, a dimension has too few items to measure reliably.

• ▸Construct definition: write the definition and its theoretical base in one or two sentences.
• ▸Expected dimensions: list each dimension and what it excludes.
• ▸Generation sources: record the population sample (size, who) and the a priori theory-derived items.
• ▸Raw pool count: number of items generated per dimension.
• ▸Wording edits: count and list items deleted as redundant, leading, or ambiguous.
• ▸Judging protocol: who judged, what they rated (relevance, dimension), and the retention rule set in advance.
• ▸Post-screen pool count: items retained per dimension after judgmental screening.
• ▸Purification statistics and sample: corrected item-to-total, loadings, interitem correlations, and N.
• ▸Confirmation and extra validity: fresh-sample structure result, reliability, test-retest, convergent evidence.

1. 1State the hypothesized dimensionality before collecting data. Write down what the construct's domain includes, what it excludes, and whether it is uni- or multidimensional (the passages call this the a priori dimensionality), derived from a literature review and ideally expert opinion.
2. 2Map every item to its hypothesized factor in a written item-to-dimension table, and flag reverse-scored items. If the source never specified item-to-dimension membership or reverse coding (as with Malhotra 1981), record that gap, because you cannot test a structure that was never specified.
3. 3Run item analysis and exploratory factor analysis on a development sample to check whether the number and composition of factors resemble the hypothesized structure. Treat this as a screening step, not the final verdict.
4. 4Specify the multi-item factors and the relations among them in a confirmatory factor analysis on a separate sample. The passages call CFA the somewhat agreed-on technique for assessing dimensionality.
5. 5Evaluate the CFA against several criteria, not one: fit indices, within- and across-factor correlated measurement errors, degree of cross-loading, and presence of method factors.
6. 6Compare the empirical structure with the theorized one. If they diverge (for example, 8 hypothesized dimensions yield 7 retained), revise the scale or the theory, and report the change.
7. 7Establish dimensionality before reliability and validity work. Unidimensionality is a prerequisite for both, so compute alphas and composite scores per factor only after the structure holds.
8. 8Score to match the structure: sum items within each dimension for dimension indices, and form an overall composite only if the construct is specified to support one.

Take Netemeyer, Burton, and Lichtenstein's (1995) vanity scale. The construct is defined with four trait aspects: excessive concern for physical appearance, a positive (perhaps inflated) view of physical appearance, excessive concern for personal achievements, and a positive (perhaps inflated) view of personal achievements. The a priori structure is four distinct dimensions, with 5, 6, 5, and 5 items respectively (21 items), all on 7-point Likert scales and summed within dimensions. To test it, you specify a four-factor CFA with each item loading only on its own dimension. You then check fit indices, cross-loadings (especially between the two physical-appearance factors, which share content), and correlated errors. A single summed 'vanity' score would be defensible only if the evidence supported it. Contrast Malhotra's (1981) semantic-differential self-concept scale. It is considered multidimensional, but items were not assigned to dimensions and reverse coding was not stated, so a researcher cannot run the CFA without first reconstructing that specification.

• If the construct is theorized as multidimensional, then test a multi-factor CFA with items assigned to factors, and do not average all items into one score unless the structure supports it.
• If EFA and the theorized structure disagree, then do not lean on alpha to rescue the scale; revisit the dimensionality or the item set, since unidimensionality precedes reliability.
• If CFA fit is acceptable but cross-loadings or correlated errors are present, then treat the structure as provisional and examine those items before reporting dimension scores.
• If the construct is very narrowly defined, then consider that a short scale or even a single item may suffice, and weigh length against parsimony.
• If the source scale lacks item-to-dimension assignments or reverse-coding information, then reconstruct and document them before any factor test, and state that the structure is unverified.

• Reporting only coefficient alpha as dimensionality evidence. Tell: a high alpha is cited, but no factor analysis appears and no one checked whether the items form one factor.
• Skipping an a priori specification and letting EFA define the construct. Tell: the dimension names appear only after the analysis and match no earlier definition.
• Judging CFA by a single fit index. Tell: no mention of cross-loadings, correlated errors, or method factors.
• Silently dropping or merging dimensions when the data disagree. Tell: the final scale has fewer dimensions than the theory, with no explanation (cf. the eight hypothesized vs. seven retained job-satisfaction dimensions).
• Using a multidimensional scale whose structure is undocumented, then summing everything. Tell: items lack dimension labels or reverse-coding instructions, and a single total is computed anyway.

• ▸Construct definition: state what the domain includes and excludes in one or two sentences.
• ▸Hypothesized dimensionality: uni- or multidimensional, and the number and names of dimensions.
• ▸Item-to-dimension map: list each item, its assigned factor, and whether it is reverse-scored (write N/A if the source omits it).
• ▸Samples used: EFA/development sample and separate CFA sample, with sizes.
• ▸EFA result: number of factors found and whether the pattern matches the hypothesis.
• ▸CFA model and fit indices: specified factors, inter-factor relations, and fit values reported.
• ▸Problem diagnostics: cross-loading items, correlated errors within/across factors, and any method factor.
• ▸Match verdict: does the empirical structure reflect the theorized one (yes/partial/no), and what was changed.
• ▸Scoring decision: dimension sums only, or overall composite, with the justification.

1. 1Define the construct's domain and breadth first. Reliability depends on how broad or narrow the construct is. Wide or multidimensional constructs need more items. Very narrowly defined constructs may be adequately measured by a single item (Bergkvist and Rossiter 2007; Drolet and Morrison 2001; Wanous et al. 1997).
2. 2Check dimensionality before computing any reliability coefficient. Unidimensionality is a prerequisite to reliability and validity. Run a confirmatory factor analysis that specifies the hypothesized factors. Inspect fit indices, within- and across-factor correlated measurement errors, cross-loadings, and method factors. Confirm the empirical structure matches the theorized structure.
3. 3Compute internal consistency separately for each factor or subscale, not once across the whole instrument. Items in a scale or subscale should show high internal consistency. Report coefficients such as alpha and composite reliability, plus item-to-total correlations. Examine the item-level correlations to find weak items, not just the summary coefficient.
4. 4Draft items for reliability. Shorter, simpler items that are easy to process and understand are generally more reliable. Keep each item representative of the construct. Remove jargon, ambiguous or difficult wording, and double-barreled items.
5. 5Balance length against parsimony. Scale brevity is a growing concern, so do not add items to inflate a coefficient. Add items only when the content domain requires them, and cut only items that are redundant or defective.
6. 6Plan test-retest reliability before data collection. Administer the same items to the same respondents at two points in time and correlate the scores. Test-retest is the most commonly skipped half of reliability: less than half of the handbook's scales report it. Decide the interval and the retest sample in advance.
7. 7Collect the evidence on a sample that represents the population of interest, not students alone. The question is whether the reliability and norms apply to the respondents who will use the scale. Report means, percentages, and variances as normative information, because a raw score means little without them.
8. 8When a scale is borrowed or used in another country or culture, assess measurement equivalence before comparing reliability or scores. Do not assume the U.S. coefficient carries over. Then read the original source article and make your own evaluation, because a handbook listing does not ensure quality.

You need a trait measure for a consumer study and shortlist Need for Cognition (Cacioppo and Petty 1982) and Consumer Self-Confidence (Bearden, Hardesty, and Rose 2001). Both appear in the handbook's Traits chapter. The handbook warns that each write-up reflects its source article and that inclusion does not guarantee quality, so you go to each original article. For each scale you record whether it reports internal consistency, whether it reports test-retest, and whether the factor structure was confirmed. Suppose one source reports only an internal consistency coefficient and the other reports both. This is the common pattern, since the overwhelming majority of handbook scales offer internal consistency and under half offer test-retest. The first scale has a gap you must close. You plan a two-wave retest in your own sample, and you run a confirmatory factor analysis first so the alpha you report applies to a unidimensional factor. If your respondents are a general-population panel and the source used students, you note that the published coefficients do not transfer automatically. You report your own alpha, item-to-total correlations, retest correlation, means, and variances.

• If a source reports only internal consistency, then plan test-retest assessment yourself, since stability over time is the commonly missing half.
• If the confirmatory factor analysis does not reproduce the theorized dimensionality, then do not report a single overall coefficient. Resolve the structure, or report reliability per factor.
• If the construct is narrowly defined, then consider a single item or very few items rather than lengthening the scale to chase a coefficient.
• If the construct is broad or multidimensional, then expect to need more items and report reliability for each subscale.
• If the original validation sample was students only, or the scale is used in another culture, then re-estimate reliability and norms in the target population and check measurement equivalence before relying on published values.

• Computing alpha on a multidimensional item pool. The tell: one summary coefficient with no factor analysis behind it, and a factor structure that differs from the theorized one.
• Treating internal consistency as the whole of reliability. The tell: the report contains no stability evidence, so respondent scores over time are unknown.
• Inflating length to raise the coefficient. The tell: redundant, near-identical items, longer surveys, and no justification tied to the construct's domain.
• Trusting published coefficients without reading the source. The tell: you cite the handbook entry alone, even though its detail varies with the original article.
• Transferring reliability and norms across populations or countries unchecked. The tell: student-derived values are applied to general or foreign samples, and raw scores are reported with no means or variances.

• ▸Construct and domain breadth: state the definition and whether it is narrow, broad, or multidimensional.
• ▸Hypothesized dimensionality: list the expected factors or subscales and the items assigned to each.
• ▸CFA evidence: record fit indices, correlated errors, cross-loadings, method factors, and whether the structure matches the theory.
• ▸Internal consistency by factor: record the alpha or composite reliability and the item-to-total correlations, flagging weak items.
• ▸Item quality check: note any jargon, ambiguous wording, double-barreled items, or items that are not brief.
• ▸Test-retest plan or result: give the interval, the sample, and the correlation (or mark 'not assessed').
• ▸Sample representativeness: state who the respondents were and whether they match the target users.
• ▸Normative information: record means, percentages, and variances for the sample.
• ▸Cross-national or cross-group equivalence: record the equivalence procedure used, or mark 'not assessed' with a justification.

1. 1Start from the construct definition, not the statistics. Before any validity test, write what the construct includes, what it excludes, and its a priori dimensionality. The source says this definition should come from a thorough literature review and, ideally, expert opinion. Every later validity claim is judged against this document.
2. 2Confirm the prerequisites first: content/face validity, dimensionality, reliability. The source treats unidimensionality as a prerequisite to reliability and validity, and construct validity as what you assess 'beyond' these. Run a confirmatory factor analysis, check that the empirical factor structure matches the theorized one, and report internal consistency (plus test-retest where possible) before you interpret any validity correlation.
3. 3Build a validity map before collecting data. List (a) a second measure of the same construct (convergent), (b) measures of similar but conceptually different constructs (discriminant), (c) constructs your theory says should relate to yours, with the predicted direction (nomological), and (d) groups expected to differ on the construct (known-group). Write the predictions down before you look at the correlations.
4. 4Test convergent validity. Correlate your scale with another measure designed to capture the same construct. The Campbell and Fiske (1959) rule is that the correlation should be statistically significant and 'sufficiently large to encourage further examination of validity.' The source says convergence is found if the two measures are highly correlated.
5. 5Test discriminant validity. Correlate your scale with measures of similar but conceptually different constructs. A low to moderate correlation is often taken as evidence of discriminance. If you can use maximally different methods (e.g., self-report vs. observational), build a multitrait-multimethod (MTMM) matrix and check the three Campbell and Fiske criteria: (1) validity-diagonal entries exceed correlations in the same row and column of the heteromethod block; (2) convergent coefficients exceed the heterotrait-monomethod triangles; (3) the pattern of correlations is the same across the heterotrait triangles.
6. 6Test nomological validity. Check whether constructs that theory says are related are empirically related, i.e., their measures correlate significantly in the predicted direction. Judge each result against the prediction you wrote down in step 3, not against whatever came out significant.
7. 7If you have the data and need more rigor, move the tests into structural equations. The source notes that MTMM via structural equations partitions measure variance into trait, method, and error variance. Convergent and discriminant validity can then be assessed with chi-square maximum likelihood tests and related fit statistics, and nomological relationships can be modeled the same way.
8. 8Report the pattern, not a single number. The Handbook summarizes validity as a pattern of results with example findings when an article ran many tests. Present your evidence in that form: reliability, convergent, discriminant and nomological results, each tied to its prediction.

The source gives no coefficients for any scale, so this applies its procedure to two scales it lists, with no numbers invented. Suppose you adopt Need to Evaluate (Jarvis and Petty 1996) and want construct-validity evidence. First, confirm the prerequisites: the item set is unidimensional in a confirmatory factor analysis and internal consistency is reported. Second, choose Need for Cognition (Cacioppo and Petty 1982), also in the Handbook, as the discriminant comparator. It is a similar but conceptually different construct, so you predict a low-to-moderate correlation and record that prediction in advance. Third, for convergence you need a second measure designed to capture need to evaluate; if one exists, you predict a high, significant correlation. Fourth, you write down which theoretically related constructs should relate to need to evaluate, with the predicted sign, to define the nomological test. If the discriminant correlation is as high as the convergent one, the scale fails the second Campbell and Fiske criterion (convergent coefficients exceed heterotrait correlations), and the scale is not distinct from its neighbor. The Handbook's own screen runs on the same logic: a multi-item measure supported by face validity alone is excluded.

• If the CFA structure does not match the theorized dimensionality, then stop and fix the scale before running any convergent, discriminant or nomological test, because unidimensionality is a prerequisite to reliability and validity.
• If the convergent correlation is significant but not 'sufficiently large' to justify further validity work, then treat the two measures as not yet shown to capture the same construct, and re-examine item content and the definition.
• If a similar-but-different construct correlates with your scale as highly as your convergent measure does, then you lack discriminant validity. Low-to-moderate is the target, and your convergent coefficients should exceed the heterotrait values.
• If you can obtain maximally different methods (self-report vs. observational), then use an MTMM matrix. If you only have one method, then expect method variance to inflate correlations and say so, or partition trait, method and error variance with structural equations.
• If a nomological prediction fails (the relation is non-significant or in the wrong direction), then question the theory network or the scale. Do not quietly drop that construct from the report.

• Treating face validity as construct validity. The tell is a validity section that says only that the items 'look right.' The Handbook excludes multi-item measures supported by face validity alone.
• Reporting only reliability (e.g., internal consistency) as validity. The tell is a high coefficient alpha with no external correlations. Reliability and the other validity types are separate steps, and construct validity is assessed beyond them.
• Testing convergence but not discrimination. The tell is that every reported correlation is with a close relative of the construct. You never show the scale is distinct from similar but conceptually different constructs.
• Fitting predictions after seeing the data. The tell is a nomological network that matches the results exactly, with no failed predictions. Nomological validity means predictions from a theory are confirmed, so they must be stated first.
• Applying the Campbell and Fiske rules mechanically with a single method. The tell is MTMM language with no maximally different methods. The source calls the criteria vague and problematic, and its own MTMM treatment 'extremely brief and over-simplified,' so consult the original and the critical updates it cites.

• ▸Construct definition: what the construct includes, excludes, and its a priori dimensionality.
• ▸Prerequisites status: content/face validity, CFA fit and dimensionality match, internal consistency, test-retest if available.
• ▸Convergent measure(s): the second measure of the same construct, and the predicted correlation (significant and sufficiently large).
• ▸Discriminant comparator(s): similar but conceptually different constructs, with the predicted low-to-moderate correlation.
• ▸MTMM design: methods used (should be maximally different) and result on each of the three Campbell and Fiske criteria.
• ▸Nomological predictions: each theoretically related construct and the predicted direction, written before data collection.
• ▸Known-group predictions: groups expected to differ on the construct and the expected direction of difference.
• ▸Observed results: actual correlations or fit statistics for each row above, next to the prediction.
• ▸Verdict and action: supported, partially supported or not supported for each type of evidence, and the revision to make.