wiki-query
PApaperclipai/paperclip102k3,122 new stars in the last 7 days
assess-construct-validity
Assess whether a benchmark measures its claimed capability using content, convergent, discriminant, and confound analysis.
- Star
- 503
- Category
- Science
- License
- Apache-2.0
- Last commit
- Sep 29, 2026
skills/assess-construct-validity/SKILL.md
View on GitHubassess-construct-validity
Purpose
Assess whether a benchmark or evaluation construct measures the claimed capability rather than content cues, confounds, or unrelated skill.
Input contract
required: [construct_claim, benchmark_specification, evaluation_records]
optional: [content_analysis, convergent_measures, discriminant_measures, confound_hypotheses]
constraints: [each validity judgment requires an observable indicator, comparison basis, and provenance]
Procedure
- State the target construct and map benchmark tasks, labels, and metrics to its intended components.
- Check content coverage and plausible construct-irrelevant cues against the benchmark specification.
- Compare convergent and discriminant evidence where available, preserving missing comparisons.
- Test confound hypotheses with controlled contrasts or artifact probes and record residual uncertainty.
If construct validity depends on whether the operationalization covers the intended domain, consider map-coverage-space as the next tactic.
Output contract
produces: [construct_map, content_validity_assessment, convergent_discriminant_evidence, confound_report, validity_judgment]
delta_fields: [findings, evidence_updates, uncertainties, decisions, open_questions]
Quality gates
- Every validity claim cites a task, measure, contrast, or artifact observation.
- Content coverage, convergence, discrimination, and confounds are reported separately.
- A missing diagnostic is marked unresolved rather than treated as evidence of validity.
Failure and counterexamples
Do not infer construct validity from a high score, face validity, or agreement with another measure that shares the same confound.
Provenance map
resolved: construct-validity-assessment
Similar Skills
Official
clinical-case-report
NEnexu-io/open-design101k1,146 new stars in the last 7 days
academic-paper-review
BYbytedance/deer-flow84k389 new stars in the last 7 days
systematic-literature-review
BYbytedance/deer-flow84k389 new stars in the last 7 days
directory-submissions
COcoreyhaines31/marketingskills55k1,019 new stars in the last 7 days
academic-paper
IMImbad0202/academic-research-skills52k1,065 new stars in the last 7 days