跳到正文
FunCoding

assess-construct-validity

Assess whether a benchmark measures its claimed capability using content, convergent, discriminant, and confound analysis.

Star
503
分类
科研
许可证
Apache-2.0
最近提交
2026年9月29日

skills/assess-construct-validity/SKILL.md

在 GitHub 上查看

assess-construct-validity

Purpose

Assess whether a benchmark or evaluation construct measures the claimed capability rather than content cues, confounds, or unrelated skill.

Input contract

required: [construct_claim, benchmark_specification, evaluation_records]
optional: [content_analysis, convergent_measures, discriminant_measures, confound_hypotheses]
constraints: [each validity judgment requires an observable indicator, comparison basis, and provenance]

Procedure

  1. State the target construct and map benchmark tasks, labels, and metrics to its intended components.
  2. Check content coverage and plausible construct-irrelevant cues against the benchmark specification.
  3. Compare convergent and discriminant evidence where available, preserving missing comparisons.
  4. Test confound hypotheses with controlled contrasts or artifact probes and record residual uncertainty.

If construct validity depends on whether the operationalization covers the intended domain, consider map-coverage-space as the next tactic.

Output contract

produces: [construct_map, content_validity_assessment, convergent_discriminant_evidence, confound_report, validity_judgment]
delta_fields: [findings, evidence_updates, uncertainties, decisions, open_questions]

Quality gates

  • Every validity claim cites a task, measure, contrast, or artifact observation.
  • Content coverage, convergence, discrimination, and confounds are reported separately.
  • A missing diagnostic is marked unresolved rather than treated as evidence of validity.

Failure and counterexamples

Do not infer construct validity from a high score, face validity, or agreement with another measure that shares the same confound.

Provenance map

  • resolved: construct-validity-assessment

相似的 Skill

wiki-query

PApaperclipai/paperclip102k近 7 天新增 3,122 个 star

官方

clinical-case-report

NEnexu-io/open-design101k近 7 天新增 1,146 个 star

academic-paper-review

BYbytedance/deer-flow84k近 7 天新增 389 个 star

systematic-literature-review

BYbytedance/deer-flow84k近 7 天新增 389 个 star

directory-submissions

COcoreyhaines31/marketingskills55k近 7 天新增 1,019 个 star

academic-paper

IMImbad0202/academic-research-skills52k近 7 天新增 1,065 个 star