dsh-statcheck
当前版本 0.1.4。
Recompute the statistics a manuscript reports — and flag the ones that do not add up.
It is arithmetic, not accusation: each finding shows the reported statistic, the p value it actually implies, and the exact characters it came from.
简体中文 · Español · Português · हिन्दी
What it does
- Parses APA-style reports:
t(df), F(df1, df2), χ²(df), r(df), z, each with a nearby p = …, p < … or p > … and an optional 95% CI [lo, hi].
- Recomputes the p value from the reported statistic with zero-dependency distributions (two-tailed t/r/z, upper-tail F/χ²).
- Flags self-inconsistency with the reported value, the recomputed value, the character range and a stable issue code.
- Checks the CI against significance (a significant p with a CI that includes 0 is a contradiction).
- Emits Markdown and SARIF 2.1.0, and registers the analyzed text as a REP v1 evidence item whose checks are the findings.
Structure: parse → recompute → judge; findings[] and summary are the two outputs, and stat_report renders them as Markdown and SARIF 2.1.0. The token labels inside the diagram are the real tool, check and issue names, so the same picture serves all five languages.
What it does not do
- No data re-analysis. It cannot know whether the reported statistic is appropriate for the design.
- No misconduct claims. A finding means “the reported p does not follow from the reported statistic”. It is a candidate for human review, never a verdict about authors.
- No PDF/OCR. Extract text first (
dsh-doc-anchor) and pass text or a .txt/.md/.tex/.html path.
- No model/assumption checks, no multiple-comparison correction, no sample-size inference.
- No silent tolerance widening. The rounding rule is fixed and documented; a flagged value stays flagged.
Compatibility
|
|
| DeepSeek Harness |
dsh-v0.2.3-alpha.1 (verified 2026-11-22); peer line >=0.2.1-0 <0.3.0 || >=0.2.2-0 <0.3.0 |
| Node |
^22.19.0 || >=24.9.0 (tested on 24.22.1), ESM only |
| Peers |
@perrylink/dsh-research-core ^0.1.0, @deepseek-ai/cordis ^4.0.5-alpha.1, @deepseek-ai/dsh-tools, @deepseek-ai/schemastery ^3.18.5-alpha.1 |
| Runtime dependencies |
zero |
dsh plugin --profile <name> add @perrylink/dsh-research-core dsh-statcheck
dsh --profile <name> --dump-config | grep dsh-statcheck
dsh plugin --profile <name> remove dsh-statcheck
Tools
stat_check
{ text?, path?, persist?, maxFindings? } → { ok, summary, findings[], source, evidenceId?, recordPath? }.
Each finding: statistic, df, text, charStart, charEnd, reported { statistic, p, pKind, pValue, ci? }, recomputed { p }, status (consistent/inconsistent/unparsed), issue? (p-value/p-threshold/ci/format/df), severity, detail.
stat_recompute
{ statistic: 't'|'F'|'chi2'|'r'|'z', value, df[], p? } → { ok, p, judgment? }.
t/r/z are two-tailed; F/χ² upper-tail. p uses the same rounding rule as stat_check.
stat_report
{ findings, text? } → { ok, markdown, sarif, summary }. SARIF carries one error result per inconsistent finding. Pass the original manuscript text to map each finding's charStart to a real SARIF startLine/startColumn; without text, the region falls back to line 1 plus a character column (the properties.charStart/charEnd are always exact).
Error codes
INVALID_INPUT · NOT_FOUND · PATH_ESCAPE · NOT_A_FILE · TOO_LARGE · OUT_OF_DOMAIN · STORE_ERROR
The rounding rule (documented, not tuned)
A reported p = x is consistent when |recomputed − x| ≤ 0.5 × 10^-decimals + 0.001.
The extra 0.001 absorbs the fact that the statistic itself is often reported rounded (e.g. t = 2.50 implies p ≈ .01855, commonly written .019). p < X is consistent when the recomputed value is below X; p > X when above.
Configuration
| Key |
Default |
Meaning |
enabled |
true |
false mounts nothing |
root |
.statcheck |
store + records root |
workspaceRoot |
process.cwd() |
manuscript paths must resolve inside it |
maxChars |
2000000 |
analyzed-character cap (over-limit fails, never truncates) |
maxFindings |
500 |
findings per call |
persist |
true |
store a snapshot + REP record |
Evaluation

Animated: the benchmark adapter's real stdout over the 30 frozen statistics cases (node bench/run-stats.mjs with BENCH_DIR pointing at the frozen dsh-research-bench checkout, no network; re-run 2026-11-21). The perfect scores describe these constructed critical-value cases only — the scope warning below still applies.
Benchmark — dsh-research-bench stats v0.1.0 (30 cases)
| metric |
value |
| inconsistent precision |
1.00 |
| inconsistent recall |
1.00 |
| inconsistent F1 |
1.00 |
| clean false-positive rate |
0.00 |
| issue accuracy when flagged |
1.00 |
| exact accuracy |
1.00 |
Raw per-case predictions: bench/results-stats.json.
Scope warning: the dataset is small, synthetic and clean (10 consistent + 20 inconsistent sentences around published critical values). A perfect score shows the parser and the recomputation agree with the critical-value labels on these constructions; it does not estimate real-manuscript accuracy. The rounding rule and issue codes are documented so a reader can disagree with a specific finding.
Tests
- 43 tests across 6 files: distribution tails against published critical values (t/F/χ²/r/z), parsing (all families, offsets, thresholds, CIs, multiple statistics, unparsed), judging (p-value, threshold, format, CI, df, negative F/χ² domains), runtime (evidence + valid REP record, re-check without an evidence conflict, findings-cap clamp, file/media type, persist=false, path escapes, size cap), tools (schemas, success/failure, SARIF incl. a null recomputed p and line/column mapping from optional text), package contract.
Known limitations (v0.1)
- One-sided tests, fractional Welch df, multi-model tables and within-text references such as “see Table 2” are not modeled.
- Only APA-style inline reports are parsed;
p values in tables are out of scope.
- CI-versus-p checking assumes a two-sided test of the same effect at α = .05.
- No correction for multiple comparisons.
- SARIF line/column mapping needs the optional
stat_report text; without it the region is line 1 + character column (the exact charStart/charEnd properties are always present).
License
Apache-2.0. No third-party runtime code.