PaperJury

arXiv 2606.16322 · Claude Code skill · MITClaude Code skill · MIT

PaperJury

Due-Process Review for Bounded LaTeX Revision

投稿前,把稿件交给闭环的「审稿 → 裁定 → 修改 → 复查」系统:先由领域 reviewer 通读全文并提出问题,再对每条意见执行完整的裁定流程。不成立的意见会被驳回;可安全修复的问题会生成最小补丁;需要研究判断的问题会交回作者。模型负责阅读、判断和起草;状态管理、投票规则、补丁应用与停止条件由确定性代码控制。 A closed-loop review → verdict → revise → verify system for pre-submission hardening of LaTeX papers. Adversarial whole-paper review first; then every complaint gets due process — invalid ones are dropped, safely fixable ones become minimal patches, and anything needing the author’s judgment is handed back untouched. The model still does the hard reading; it no longer holds the gavel or the eraser.

Yiran Wang* · Ruixuan An* · Biao Wu · Wenhao Wang  —  Vast Intelligence Lab

裁定板 · 一次真实运行The verdict board · one real run 55 个问题55 issues
26valid-fixable
10author-required
19invalid-drop

对一篇 21 页真实草稿执行 dogfood 测试:152 条 reviewer 意见去重为 55 个问题;其中 26 处安全应用、10 个交回作者、19 条驳回。查看修改前后的 PDF 与运行报告 → A dogfood run on a real 21-page draft: 152 reviewer complaints deduped into 55 issues — 26 safely applied, 10 handed back to the author, 19 dropped as invalid. Before/after PDFs & run report →

/plugin marketplace add Spark-To-Paper-Skills/paperjury/plugin install paperjury@Spark-To-Paper-Skills
审稿,重点看实验和 claim 是否站得住。# or, in English:Review this paper. Focus on whether the experiments support the claims.

在 Claude Code 中安装后,进入论文项目并直接说明需求,无需记忆命令。在 direct-edit 和 review 模式下,补丁应用前都会先交给你确认。 Install inside Claude Code, then just say what you need in your paper project — no commands to memorize. Every patch waits for your confirmation before it lands.

12 篇 held-out 论文 · Vision / NLP / ML · 盲审专家审计12 held-out papers · Vision / NLP / ML · blinded expert audit

在同一批论文、同一套预算与日志口径下,对比四个 baseline。删除线灰字是其中最强的 baseline(LLM-as-judge 循环);金色是 PaperJury。 Against four baselines on the same papers, budget envelope, and logging. The struck-through gray number is the strongest baseline (the LLM-as-judge loop); brass is PaperJury.

  1. 问题质量Issue quality Panel-relative F1 ↑ 0.519 0.656 +0.137
  2. 审计精度Audited precision Pverified ↑ · n=98 0.663 0.847 +0.184
  3. 不安全编辑率Unsafe-edit rate ESVR ↓ 0.110 0.025 低 4.4×4.4× lower

Baseline 为 LLM-as-judge 循环(四个 baseline 中综合最强);PaperJury 同时以 3.08 ± 0.67 轮确定性收敛(从未触到五轮上限)。全部数字来自 arXiv 2606.16322 的双臂专家评审,置信区间为 Wilson 95% CI。 Baseline is the LLM-as-judge loop, the strongest of the four; PaperJury also terminates deterministically in 3.08 ± 0.67 rounds, never hitting the five-round cap. All numbers from the two-arm expert study in arXiv 2606.16322; intervals are Wilson 95% CIs.

核心想法Core Idea

难的不是写出批评,而是管住批评之后发生的事。The hard part is not writing the critique — it is governing what follows from it.

一篇论文可以在论证还站不住的时候就“看起来写完了”:claim 比实验多说了半步、两节之间互相矛盾、一个数字暗示了没测过的东西。这些毛病读起来很流畅,只有把全文放在一起对抗式地读才会现形。真正的困难在后面——AI 挑出的问题里混着幻觉,真问题里又有相当一部分机器根本修不了。 A paper can look finished long before its arguments are sound: a claim half a step ahead of the experiments, two sections that quietly contradict each other, a number that implies something never measured. These flaws read fluently and only surface under adversarial whole-paper reading. The real difficulty comes after — machine-generated complaints mix in hallucinations, and many genuine issues are simply not machine-fixable.

不能直接采用所有 AI 意见Acting on everything corrupts sound papers

机器批评里有相当一部分是错的——比如凭空“发现”两段其实一致的文字互相矛盾。一个听见什么就改什么的循环,会把没病的稿子改出病来。Many machine complaints are wrong — a hallucinated contradiction between two passages that in fact agree. A loop that acts on everything it hears rewrites a sound paper into a broken one.

“修 / 不修”是错误的二分Fix-or-ignore is the wrong shape

很多成立的批评需要补实验、补数据——软件做不了,也不该替作者做。裁定空间必须容得下“问题成立,但只能交回作者”这个出口。Many correct complaints require new experiments or data — things software cannot and should not do for the author. The verdict space must include “valid, but hand it back.”

提议改动的人不能自己盖章The proposer is the worst certifier

被问“改完了吗”的模型倾向于回答“改完了”;放开手改稿的模型也控制不住自己的波及范围——一个只想收紧措辞的补丁,可能悄悄改掉隔壁的 claim。A model asked whether it is finished tends to say yes; a model editing free-hand cannot bound its own blast radius — a wording patch can silently mutate the claim next door.

“模型仍然做最难的阅读;但它不再握着法槌,也不再握着橡皮。”“The model still does the hard reading; it no longer holds the gavel or the eraser.”

—— PaperJury 的设计原则:确定性与语义严格分工(arXiv 2606.16322,§1)— The deterministic-versus-semantic split, arXiv 2606.16322, §1

现状 · judge 中心的循环Today · judge-centered loops

模型自己审、自己判、自己改、自己喊停One model reviews, judges, edits, and stops itself

哪些意见算数、什么时候结束,都押在单个语义判断上——而这种判断换个 prompt 就会变。结果是:该驳回的幻觉照样落成补丁,该停的循环停不下来。Which complaints count and when to stop both ride on a single semantic judgment — one that shifts when the prompt is rephrased. Hallucinated charges become patches; loops that should stop, don’t.

PaperJury · 确定性 vs 语义分工PaperJury · the deterministic-versus-semantic split

状态、争议程度分流、停止和补丁应用,全部收进确定性代码State, routing, stopping, and patching live in deterministic code

durable ledger 为每个问题提供跨轮次的稳定标识;争议程度分流决定哪些问题进入庭审;代码按照法定人数和多数规则判断指控是否成立;成立后,judge agent 再决定是安全修复还是交回作者。预应用检查通过后,apply-patch 会原子应用补丁并写入 journal;随后运行 compile-guard,失败则回滚并记录回滚。模型只处理需要阅读、判断和起草的有界子任务。A durable ledger gives every issue a stable cross-round identity; contestability routing decides what goes to trial; verdicts are computed in code under quorum rules; patches pass a guard chain, then apply exactly once, journaled and revertible. Models handle only bounded reading, judging, and drafting.

方法Method

确定性代码负责状态机;模型负责阅读、判断和起草。Deterministic code owns the state machine; models only read, judge, and draft.

整个流程分为三段:先通过确定性分解把稿件转换为可寻址的单元,并冻结 claim spine;再执行有界的全文评审与裁定;最后只修改被判为 valid-fixable 的问题。durable ledger 贯穿整个流程,停止谓词 τ 根据其中的状态决定是否结束。 Three stages: deterministic decomposition turns the manuscript into addressable units and freezes the claim spine; bounded whole-paper review and adjudication follow; only issues ruled valid-fixable reach guarded revision. A durable ledger and a ledger-query stopping predicate τ run through all of it.

1确定性准备Deterministic setup
  • 分解 + 锚点。把 LaTeX 稿切成章节、段落、交叉引用目标,后续一切都挂在持久锚点上,不再用随手的文本片段。Decompose + anchor. The manuscript becomes sections, passages, and cross-reference targets; everything downstream binds to persistent anchors, not ad hoc spans.
  • 冻结 claim spine。抽出论文的核心主张并冻结成受保护的骨架:修订可以修支撑、修措辞,但不许悄悄改写 claim。Freeze the claim spine. The paper’s central assertions become a protected backbone: edits may repair support or wording, never silently rewrite a claim.
  • 初始化 durable ledger。问题、证据、裁定、补丁历史,全部机器可读、跨轮次、跨会话。Initialize the durable ledger. Issues, evidence, verdicts, and patch history — machine-readable, across rounds and sessions.
2评审 + 裁定Review + adjudication
  • 有界全文评审。N = 2–4 位(默认 3)领域 reviewer 各通读全文一遍,输出带证据锚点的弱点;反略读检查不过关才触发定向重读。Bounded holistic review. N = 2–4 domain reviewers (default 3) each read the full paper once, emitting evidence-anchored weaknesses; targeted re-reads fire only when anti-skim checks fail.
  • 合并去重。重复意见并成一个问题,保留证据来源,问题身份跨轮稳定。Merge + dedupe. Duplicates collapse into one issue with provenance kept; identity stays stable across rounds.
  • 可争议性路由。确定性规则决定走向:机械/轻微问题走低成本 polish;有争议的重大问题上庭。Contestability routing. A deterministic rule sends mechanical/minor issues down the cheap polish path and contestable substantive-major ones to trial.
  • 正当程序审理。全文视角的辩方对阵去相关的局部证据陪审团;没有明显多数时通常升级到更大合议庭,但全员都因上下文不足而无法判断时直接交回作者。代码按法定人数和多数规则判断指控是否成立;成立后由 judge agent 完成后续路由。Due-process trial. A whole-paper defense argues against a decorrelated local-context jury; insufficient agreement escalates the panel; the verdict is computed in code under quorum and majority rules.
3带护栏修订 + 复查Guarded revision + verification
  • 只修 valid-fixable。起草最小补丁,按结构信号定风险等级(LOW / RISKY),风险越高审计越强。Valid-fixable only. Minimal patches are drafted and risk-rated (LOW / RISKY) from structural signals; higher risk triggers stronger audits.
  • 预应用检查。先执行锚点约束 diff 和交叉引用检查;RISKY 修改还要通过含义审计或编辑审计。检查失败的补丁会被拦截或升级处理。Guard chain. Anchor-bounded diff, cross-reference, meaning/edit audits, compile checks — fail any and the patch is blocked or escalated.
  • Exact-once 应用、日志与编译检查。apply-patch 原子应用补丁并写入 journal,随后运行 compile-guard;失败则回滚,并在 journal 中记录回滚。Exact-once apply + journal. Patches apply atomically, fully journaled, always revertible.
  • 独立复查。下一轮只读取当前稿件,不延续上一轮的模型输出,从而减少旧批评形成的惯性。Clean re-review. The next round reads the current manuscript only — never continues from earlier model outputs.

Durable ledger:跨轮次的问题记录The durable ledger: issue identity that survives

每个问题以稳定身份跨轮存续——周围的模型可以重跑,问题本身不漂移。路由、闭合、exact-once 修订全都查它。Each issue persists as the same tracked object while the models around it are re-run. Routing, closure, and exact-once revision all query it.

停止谓词 τ:由 ledger 状态决定Stopping predicate τ: the ledger decides

无人值守循环在 τ(L, Ur, Cr, r) 为真时停:看新增问题、闭合问题和硬性上限,而不是问模型「你写完了吗」。The unattended loop halts when τ(L, Ur, Cr, r) holds — new issues, closed issues, hard limits. Never by asking the model whether it feels done.

要点Takeaway

所有需要跨轮保持一致的操作,都由确定性代码执行。Anything that must behave consistently across runs is taken out of model discretion.

分解、ledger、争议程度分流、投票结果计算、停止判断和补丁应用由确定性代码处理;语义 agent 负责阅读、推理和起草。多数票判定指控成立后,judge agent 再将其路由为 valid-fixable 或 author-required。Decomposition, the ledger, routing, verdict computation, stopping, patching — every load-bearing call runs in deterministic code, while semantic agents take only the bounded subtasks that genuinely need reading, reasoning, and drafting.

正当程序Due Process

每个问题最终都会进入三种结果之一:代码先按投票规则判断指控是否成立,成立后再由 judge agent 决定后续路由。Every issue terminates in one of three verdicts — computed in code, not decreed by a model.

单个 LLM judge 的判断会随 prompt 措辞漂移,所以 PaperJury 不给任何单一语义裁判最终权力。有争议的问题要经过一场结构化的「庭审」:辩方与陪审团相互隔离、视角不同,谁也污染不了谁。 A single LLM judge shifts with prompt phrasing, so no lone semantic judge gets final authority. Contested issues go through a structured trial where defense and jury are isolated from each other, seeing the paper from deliberately different angles.

辩方The defense

全文视角,为论文辩护Whole-paper context, arguing against the charge

拿着整篇稿子反驳指控:如果「矛盾」其实在别处有解释、「过强的 claim」其实有支撑,就在这一步把它讲清楚。Armed with the full manuscript, it pushes back: if the “contradiction” is explained elsewhere or the “overclaim” is actually supported, this is where that gets said.

vs
陪审团The jury

去相关的局部证据评审Decorrelated, local-context evaluation

只看局部证据、刻意摆脱最初 reviewer 的措辞框架来评估指控;没有明显多数时通常升级到更大的合议庭,但全员都因上下文不足而无法判断时直接交回作者。It weighs localized evidence with reduced dependence on the original reviewer’s framing; insufficient agreement escalates to a larger panel.

庭审结果 oi = TrialOutcome(T(i), q, m) 由代码根据法定人数阈值 q 与多数阈值 m 算出。指控成立后,judge agent 再决定是 valid-fixable 还是 author-required。关键在于:问题成立不等于机器可修 The terminal verdict vi = Verdict(T(i), q, m) is computed in code from quorum threshold q and majority threshold m. The crux: issue validity does not imply machine editability — which is why the verdict space is three-way by construction.

invalid-drop

指控不成立,驳回The charge fails — dropped

reviewer 误读了论文,或者提了不该改的问题。驳回并记录在案,不进入修订。The reviewer misread the paper or raised something that should not change. Rejected on the record; never reaches revision.

盲审专家复核一致率 0.872Blinded expert agreement 0.872

valid-fixable

成立,且能安全地修Valid, and safely machine-editable

表达不清、claim 过强、结构不顺——不用补实验、不会改偏原意的问题,起草最小补丁进护栏链。Unclear wording, overreaching claims, broken structure — issues needing no new experiments. A minimal patch is drafted and sent through the guard chain.

盲审专家复核一致率 0.913Blinded expert agreement 0.913

author-required

成立,但必须交回作者Valid, but the author must decide

缺实验、缺 ablation、缺数据,或者需要作者的私有知识与研究判断。原样交回,绝不代笔。Missing experiments, ablations, or data — anything needing the author’s own knowledge or judgment. Handed back untouched, never ghost-written.

盲审专家复核一致率 0.860Blinded expert agreement 0.860

要点Takeaway

裁定错误集中在 valid-fixable 与 author-required 的边界上——那正是合理分歧所在。Verdict errors concentrate on the valid-fixable / author-required boundary — exactly where reasonable people disagree.

三类裁定的盲审一致率都在 0.86 以上;争议最大的 author-required 也有最高的「无法判断」占比(15.7%),说明剩下的分歧是讲得出道理的判断差异,而不是裁定机制失灵。All three classes hold above 0.86 blinded agreement; the boundary class author-required also has the highest cannot-tell rate (15.7%) — defensible disagreement, not adjudication failure.

评测设计Evaluation

双臂专家评审:先对问题,再审裁定。Two arms of expert review: match the issues, then audit the verdicts.

在 12 篇 held-out 的 LaTeX 论文(Vision / NLP / ML 各 4 篇)上,与四个 baseline 在同一套稿件、预算口径和日志下对比。问题发现质量与下游裁定质量分开度量,避免互相混淆。 On 12 held-out LaTeX papers (4 each from Vision, NLP, and ML), against four baselines under the same manuscripts, budget accounting, and logging. Issue discovery and downstream adjudication are measured separately so neither contaminates the other.

第一臂 · 问题匹配Arm 1 · Issue matching

系统发现的问题与独立的专家问题清单做段落级 + 语义匹配(按 fix-equivalence 判等),得到 panel-relative 的精度、召回与 F1。System-discovered issues are matched against independent expert issue panels at passage level and semantically, under fix-equivalence — yielding panel-relative precision, recall, and F1.

第二臂 · 盲审审计Arm 2 · Blinded audit

同一批专家在看不到系统裁定的情况下重新标注抽样问题,并审计路由决定、已应用的编辑和停止行为——得到裁定正确率、路由正确率和编辑安全性。The same experts relabel sampled issues without seeing the system’s verdicts, and audit routing decisions, applied edits, and stopping behavior — yielding verdict accuracy, routing accuracy, and edit safety.

结果Results

最高的问题质量、最高的审计精度、最安全的编辑,同时确定性收敛。Best issue quality, highest audited precision, safest edits — while terminating deterministically.

关键不在单项指标,而在能否同时守住问题质量、受治理的裁定、不伤稿件的安全修订(artifact-safe)和实际可用的成本。逐篇配对比较里,PaperJury 的 F1 在全部 12 篇上胜过每一个会出问题清单的 baseline;paper-cluster paired-bootstrap 的 ΔF1 95% 下界分别为 +0.200(vs 批评器)、+0.127(vs judge 循环)、+0.188(vs 朴素生成器),满足预注册的主要终点。 The point is not any single metric but sustaining all of it at once: issue quality, governed adjudication, artifact-safe revision, practical cost. Per-paper paired F1 favors PaperJury on all 12 papers against every issue-producing baseline; paper-cluster paired-bootstrap 95% lower bounds on ΔF1 are +0.200 (vs critic), +0.127 (vs judge loop), and +0.188 (vs naive generator) — satisfying the preregistered primary endpoint.

表 1 · 端到端主结果(12 篇论文,双臂专家评审)Table 1 · Main end-to-end results (12 papers, two-arm expert review)
方法Method Ppanel / Pverifiedn95% CI R ↑F1 ↑AccvAccr ESVR ↓KW (h)
Forward-only 重写器Forward-only rewriter n/an/an/an/an/a n/an/a0.24010.31
LLM 批评器LLM critic only 0.437 / 0.57778[0.466, 0.680]0.4620.446 n/an/an/a10.51
LLM-as-judge 循环LLM-as-judge loop 0.512 / 0.66392[0.562, 0.751]0.5330.519 0.681n/a0.1103.33 ± 1.07 (2/12)2.06
朴素无界生成器Naive unbounded generator 0.341 / 0.51192[0.410, 0.611]0.7210.459 n/an/an/a18.37
PaperJury (本文)(ours) 0.684 / 0.84798[0.763, 0.905]0.6370.656 0.8870.9130.0253.08 ± 0.67 (0/12)2.47

Ppanel:相对专家清单的重大问题精度(fix-equivalence 匹配);Pverified:经专家审计、把系统独有的成立问题计入后的修正精度(从不参与 F1);Accv / Accr:盲审专家对终局裁定 / 路由决定的一致率;ESVR:已应用编辑中违反安全的比例(需与表 3 的编辑量合读);K:收敛轮数,括号内为触到五轮上限的论文数;W:每篇小时数。n/a 表示该方法没有这项能力。95% CI 为 Pverified 的 Wilson 区间。 Ppanel: panel-relative precision of major issues under fix-equivalence matching. Pverified: audit-corrected precision crediting expert-validated system-only issues (never used inside F1). Accv / Accr: blinded expert agreement with terminal verdicts / routing. ESVR: safety-violating share of applied edits (read with Table 3’s edit volumes). K: rounds to termination, with papers hitting the five-round cap in parentheses. W: wall-clock hours per paper. n/a marks capabilities a method does not have. CIs are Wilson intervals on Pverified.

问题质量:panel-relative F1Issue quality: panel-relative F1 越长越好 · Forward-only 重写器不出问题清单,无 F1Longer is better · the forward-only rewriter produces no issue list, hence no F1
PaperJurybaselineBaselines
裁定一致率Verdict agreement0.887judge 循环:0.681Judge loop: 0.681
路由一致率Routing agreement0.913baseline 均无路由No baseline has routing
收敛轮数Rounds to termination3.08 ± 0.67从未触到五轮上限;judge 循环 2/12 触顶Never hits the five-round cap; judge loop caps on 2/12
表 2a · 分裁定类的盲审复核Table 2a · Verdict audit by terminal class
裁定类ClassnAccv95% CI无法判断Cannot-tell
invalid-drop430.872[0.733, 0.944]4 (9.3%)
valid-fixable730.913[0.823, 0.960]4 (5.5%)
author-required510.860[0.727, 0.934]8 (15.7%)

盲审专家在不知道系统裁定的情况下重新标注;误判集中在 valid-fixable ↔ author-required 边界。Blinded experts relabel without seeing the system’s verdict; confusions concentrate on the valid-fixable ↔ author-required boundary.

表 2b · 分领域切片(每个领域 4 篇;仅作诊断)Table 2b · Per-domain slices (4 papers each; diagnostic)
领域DomainF1AccvAccrESVR
Vision0.6460.8810.9040.028
NLP0.6710.9020.9290.021
ML0.6510.8780.9060.026

三个切片的全部指标与合并值相差不超过 0.03——质量不是靠某个子领域撑起来的。Every metric stays within 0.03 of the pooled values — quality is not carried by one subcommunity.

安全与成本Safety & Cost

编辑量相当,不安全编辑率相差 4.4 倍,PaperJury 约为 judge 循环的四分之一,说明护栏链带来了明确的安全收益。Similar edit volume, 4.4× fewer unsafe edits — the guard chain pays for itself.

ESVR 必须结合编辑量和覆盖率解读;如果系统几乎不修改稿件,低 ESVR 本身没有解释力。PaperJury 与 judge 循环的编辑量相当(每篇 13.4 vs 14.3 处),但护栏链在应用前拦截了 17.0% 的候选补丁(judge 循环为 3.4%)。两者的单位编辑违规率相差 4.4 倍,PaperJury 约为 judge 循环的四分之一。 ESVR is only interpretable next to edit volume and coverage — a system that barely touches the paper is “safe” in a useless way. PaperJury applies about as many edits as the judge loop (13.4 vs 14.3 per paper); the difference is a guard chain that blocks 17.0% of proposed patches before application (vs 3.4%), buying a 4.4× lower per-edit violation rate.

表 3 · 编辑安全(ESVR 的必备伴读)Table 3 · Edit safety (the mandatory companion to ESVR)
方法Method 已应用(每篇)Applied (per paper) VF 终局VF terminals 候选补丁Proposed 覆盖率Coverage 护栏拦截Guard-block 不安全Unsafe ESVR ↓
Forward-only 重写器Forward-only rewriter 254 (21.2)n/an/an/an/a610.240
LLM-as-judge 循环LLM-as-judge loop 172 (14.3)2041780.8430.034190.110
PaperJury (本文)(ours) 161 (13.4)1961940.8210.17040.025

批评器与朴素生成器不做修改,不在此表。重写器改得最多(每篇 21.2 处)也错得最狠(ESVR 0.240)。PaperJury 覆盖率略低于 judge 循环(0.821 vs 0.843)——用这一点覆盖率,换来 12 篇合计的不安全编辑从 19 处降到 4 处。 The critic-only and naive baselines apply no edits. The rewriter edits the most (21.2 per paper) and worst (ESVR 0.240). PaperJury’s coverage runs slightly below the judge loop (0.821 vs 0.843) — the price of dropping unsafe edits from 19 to 4 pooled across papers.

每篇耗时(小时)Wall-clock hours per paper 越短越好 · token 成本:PaperJury 每篇 6.76M;朴素生成器 31.4MShorter is better · token cost: PaperJury 6.76M per paper; naive generator 31.4M
PaperJurybaselineBaselines
要点Takeaway

治理的代价并不高:2.47 小时、6.76M token,换回可审计的全流程。Governance costs little: 2.47 hours and 6.76M tokens buy a fully auditable loop.

token 比穷举式批评省 4 倍多、时间快 3 倍多,用一些召回换来精度与安全的大幅提升;比便宜的单遍系统多花的时间,花在了裁定与护栏上。Over 4× cheaper in tokens and over 3× faster than exhaustive critique, trading some recall for large gains in precision and safety; the hours beyond the cheap single-pass systems are spent exactly on adjudication and guards.

消融实验Ablations

拆掉哪个部件,系统就恰好坏在那个部件的职责上。Remove a component, and the system fails exactly where that component’s job was.

在预注册的、领域均衡的 6 篇子集上逐个拆部件。消融有说服力的标准不是“拆了还能跑”,而是退化方向要与被拆部件的功能对得上——五个部件全部对上了。 Each component is removed on a preregistered, domain-balanced six-paper subset. The bar for an informative ablation is not “it still runs” but that the degradation aligns with the removed component’s function — and all five align.

表 4 · 消融(6 篇子集;Δ 相对同子集重跑的完整系统)Table 4 · Ablations (six-paper subset; Δ vs Full PaperJury re-executed on the same subset)
变体Variant F1ΔF1AccvΔAccv ESVRΔESVRKW (h)
Full PaperJury 0.6490.8810.0293.00 ± 0.63 (0/6)2.43
去掉有界评审w/o bounded review 0.572−0.0770.868−0.0130.042+0.0133.67 ± 0.524.81
去掉确定性路由w/o routing 0.636−0.0130.806−0.0750.047+0.0183.33 ± 0.52 (0/6)3.49
去掉庭审w/o trial 0.642−0.0070.728−0.1530.052+0.0233.17 ± 0.75 (0/6)2.18
去掉 claim spinew/o claim spine 0.632−0.0170.857−0.0240.112+0.0833.00 ± 0.63 (0/6)2.37
去掉护栏链w/o guard chain 0.627−0.0220.859−0.0220.181+0.1522.83 ± 0.75 (0/6)1.94

同一子集上完整系统重跑的 F1 与主实验相差在运行随机性内。“去掉有界评审”同时取消了轮数上限,故 K 无触顶口径。The full system’s subset re-execution differs from the main run within run stochasticity. “w/o bounded review” also removes the round cap, so its K has no cap fraction.

拆有界评审 → 发现能力掉最狠Drop bounded review → discovery falls hardest

−0.077F1

同时耗时翻倍(2.43 → 4.81 小时):没有边界的评审既漏得多、又跑得慢。Runtime also doubles (2.43 → 4.81 h): unbounded review misses more and runs slower.

拆庭审 → 裁定质量掉最狠Drop the trial → verdicts fall hardest

−0.153Accv

换成单遍语义判断后,恰恰是边界案例最先失守——它们正需要全文语境才能稳定裁定。With single-pass semantic judgment, borderline cases fail first — exactly the ones needing manuscript-wide context.

拆护栏链 → 安全崩得最狠Drop the guard chain → safety collapses

+0.152ESVR

ESVR 升至 0.181,同时以 1.94 小时最早结束。这说明修改缺少安全检查,并不代表修订质量更高。ESVR jumps to 0.181 while terminating fastest (1.94 h) — the signature of unchecked application, not better revision.

拆 claim spine → 主张开始漂移Drop the claim spine → claims drift

+0.083ESVR

没有被冻结的主张骨架,更多补丁越过局部修补、动到 claim 层——初筛拦不住它们。Without the frozen backbone, more patches slip past screening into claim-level alteration.

拆确定性路由 → 判得更差、花得更多Drop routing → worse verdicts, higher cost

−0.075Accv

耗时 2.43 → 3.49 小时而 F1 几乎不动:所有问题都涌进昂贵的语义裁定,分配失灵。Hours rise 2.43 → 3.49 while F1 barely moves: everything floods into expensive semantic adjudication.

合起来读Read together

5 / 5

五个部件的退化方向全部与其功能对齐——护栏链和 claim spine 是修订不伤稿件的必要条件,不是保守的摆设。All five degradations align with component function — the guard chain and claim spine are necessary conditions for artifact-safe revision, not conservative overhead.

上手使用In Practice

三种模式,一次真实 dogfood,一套诚实的降级。Three modes, one real dogfood run, and honest degradation.

PaperJury 以 Claude Code skill 的形式发布。直接描述需求即可,无需记忆命令;系统会在 direct-edit 与 review 之间选择,auto 则必须显式启用。在 direct-edit 和 review 模式下,补丁应用前都会先交给你确认。另有 Codex 版 PaperJury ships as a Claude Code skill. No commands to memorize: describe what you need and it picks the mode; every patch waits for your confirmation before landing. A Codex port exists too.

direct-edit

最常用 · 只改一处Most common · one spot

“把这段改紧一点,但别改 claim。”不开评审面板,直接起草补丁,你确认后应用。“Tighten this paragraph, but don’t change the claim.” No review panel — a patch is drafted and applied after you confirm.

review

需要全面审稿时When you want the jury

“审稿”“mock review”“只看第 4 节”。启动对抗式评审引擎,先判问题成立与否,改动逐一确认。“Review this,” “mock review,” “just Section 4.” The adversarial engine runs; issues are adjudicated first, and every change is confirmed one by one.

auto

无人值守 · 须显式开启Unattended · explicit opt-in

配置 mode: auto 作为修改策略,并用 /goal 提供可验证目标后,系统会连续运行多轮;安全修改可自动应用,高风险修改仍交回作者。With explicit /goal or mode: auto authorization, it iterates to a verifiable target; safe fixes may auto-apply, high-risk items still return to the author.

auto 策略和跨轮运行都必须显式启用。mode: auto 决定候选修改是自动应用还是进入待办队列;/goal 负责跨轮继续运行。仅放开工具权限或发送普通 prompt,都不会启动无人值守的多轮循环。详见 AGENT-GUIDE §3Auto is never inferred. Opening tool permissions and sending an ordinary prompt runs one round and stops. Unattended multi-round work requires /goal or mode: auto — see AGENT-GUIDE §3.

完整运行样例:21 页草稿,11 个预设缺陷The real run: a 21-page draft with 11 planted defects

一篇真实草稿完成了一轮 auto 模式评审:152 条 reviewer 意见去重为 55 个问题,其中 26 处安全应用、10 个交回作者、19 条驳回;修改后的稿件编译结果为 0 error / 0 warning。仓库提供修改前后的 PDF 和经过人工核对的运行报告,建议先查看样例,再决定是否用于自己的论文。A full auto-mode pass over a real draft: 152 reviewer complaints deduped into 55 issues — 26 safely applied, 10 handed to the author, 19 dropped; the revised manuscript compiles with 0 errors and 0 warnings. Before/after PDFs and a hand-verified run report live in the repo — read the sample before trusting it with your own paper.

152 → 55意见去重成问题Complaints deduped into issues
26 / 10 / 19应用 / 交回作者 / 驳回Applied / author / dropped
0 / 0改后编译 error / warningErrors / warnings after revision
1 roundauto 模式完整闭环One full auto-mode loop

方式 A · Claude Code plugin(推荐)Option A · Claude Code plugin (recommended)

/plugin marketplace add Spark-To-Paper-Skills/paperjury
/plugin install paperjury@Spark-To-Paper-Skills

方式 B · clone 成 skillOption B · clone as a skill

# macOS / Linux
git clone https://github.com/Spark-To-Paper-Skills/paperjury ~/.claude/skills/paperjury
# Windows (PowerShell)
git clone https://github.com/Spark-To-Paper-Skills/paperjury "$env:USERPROFILE\.claude\skills\paperjury"

边界写在最前面:PaperJury 是投稿前的自查工具,不能替代作者的科学判断,也不能替代 peer review。它不会编造实验、伪造结果、添加没有证据的 claim 或掩盖论文局限;凡是需要新实验、新证据或作者研究判断的问题,一律交回作者处理。Scope, stated up front: PaperJury is a pre-submission self-check — not a substitute for your scientific judgment or for peer review. It will not invent experiments, fabricate results, add unsupported claims, or paper over limitations; anything needing new experiments, new evidence, or research judgment goes back to the author.

BibTeX

引用这项工作。Cite this work.

如果 PaperJury 对你的研究或写作流程有帮助,欢迎引用这篇 arXiv 论文。 If PaperJury helps your research or writing workflow, please cite the arXiv paper.

@misc{wang2026paperjurydueprocessreviewbounded,
  title={PaperJury: Due-Process Review for Bounded LaTeX Revision},
  author={Yiran Wang and Ruixuan An and Biao Wu and Wenhao Wang},
  year={2026},
  eprint={2606.16322},
  archivePrefix={arXiv},
  primaryClass={cs.CL},
  url={https://arxiv.org/abs/2606.16322},
}