Fabricated numbers
A draft that needs a result will happily invent one. Once a made-up metric enters the text, every later revision inherits and defends it.
Overview
Every page below came out of the pipeline.
Core Idea
The model does the reasoning. The code keeps it honest.
Autonomous paper generators fail in three familiar ways: they invent numbers, they cite papers that do not exist, and they ship figures nobody can edit. Spark-to-Paper splits every stage between two actors — Claude owns the judgment (writing, research, critique, review) and deterministic Python gates own the facts. A red gate does not warn; it fails the build.
Phantom citations
Title-only stubs generated to hit a quota. They look like a bibliography and collapse at the first reviewer who checks a DOI.
Dead bitmap figures
AI image models produce rasters. A camera-ready paper needs vectors you can still edit the day before the deadline — not a flattened PNG.
Machine-checked integrity: not a style suggestion — a hard stop.
One command, run_gates.py <workdir> all, chains the
suite’s finish-line gates and exits nonzero on the first red one: citation
completeness, no-fabrication, word bands, editable-vector presence,
figure-critique traces, and a zero-error LaTeX log. A nonzero exit means
the paper is not done — no matter how good it looks.
Claude
Model reasons
Judgment-heavy work stays with the model: framing the story, planning the blueprint, reading candidate references, writing and refining prose, critiquing figures with its own vision, arguing against the draft in review.
Python
Code backstops
Deterministic tasks go to code: linting drafts, checking citation completeness, plotting from data, vectorizing figures, assembling LaTeX, compiling, and gating. Code never authors content; it only verifies it.
Layer 1 · Deterministic gates
Section shape, word bands, no-fabrication, citation completeness, vector-PDF presence, compile status. Run per stage or all at once; the first red gate stops the line.
Layer 2 · Self-review
The refine stage right-sizes every section to its band, scrubs AI tells from the prose, and re-reads each edit for the contradiction it may have introduced.
Layer 3 · Adversarial review
Isolated reviewers read the whole draft and must quote it verbatim to raise an issue; perspective-diverse skeptics then try to refute each finding. The loop runs until dry.
Layer 4 · Vision critique
Claude looks at every rendered figure with its own eyes — faithfulness to the method, semantic agreement with the equations, readability, aesthetics — and a measured geometry audit drives at least four repair rounds on every redrawn SVG.
Measured in the paper’s controlled ablation: with 36 unsupported claims injected, detection climbs from 14% for a single-pass draft to 69% with the gates, 81% adding self-review, and 92% with the full stack (arXiv:2608.11924, Table 4).
Pipeline
One input in. Eight stages later, a compiled PDF.
You drop one input — a bare idea, a structured proposal,
a proposal with real results, or a story from a previous run. Stage 0 of
ts-paper classifies it with no fixed schema, sets the one switch
the whole suite reads (results_mode), and drives the chain.
Files on disk are the contract between stages; each stage
writes an INPUT / DECISIONS / OUTPUT trace to logs/.
Stage 0 · What you drop decides the route.
| You dropped | Route | results_mode |
|---|---|---|
| A one-line idea | ts-idea2story builds the story first, then the chain |
proposal |
| A structured proposal | Straight into the chain — result cells stay blank | proposal |
| A proposal + measured results | ts-paper-data distills the numbers into results.facts.json |
data_aware |
A story.json from a prior run |
Skips idea2story, straight to planning | proposal |
Any real measured number in the input — a filled table, “achieved 0.62 HOTA” — forces the data-aware route; it is never sent down the no-numbers proposal path.
1 · Plan
One reasoning pass emits blueprint.json: title, keywords, exactly three contributions, notation, terminology, experiment design, per-section word targets.
2 · Cite
Broad WebSearch per angle, abstracts read, metadata fetched via Crossref / arXiv. Floor of 40 real references, every one mapped to the claim it supports.
3 · Write
All sections in one holistic pass — terminology stays consistent because the whole paper is in context. In proposal mode, result cells stay --.
4 · Refine
Right-size to the enforced word bands, scrub the AI tells, self-check the logic of every edit. “Right-sized” is verified in code, not by eye.
5 · Review
The adversarial panel argues against the paper: isolated reviewers, verbatim-quote anti-skim, skeptic verification, loop until dry. Fixes route back through refine.
6 · Figures
Data plots are born-vector matplotlib; schematics are PaperBanana renders reconstructed in code as native, audited vector graphics.
7 · Compile
Template-driven assembly and latexmk. Zero-error logs and a resolved bibliography required; the fix loop is bounded at three tries.
8 · Experiments (auto)
Runs automatically after the first gates-green draft: diagnoses the paper’s logic, runs only feasible experiments on real data, fills the tables, recompiles.
13 skills, by name
ts-paper
ts-idea2story
ts-paper-plan
ts-paper-cite
ts-paper-write
ts-paper-refine
ts-paper-review
ts-paper-figure
ts-paper-data
ts-figure-svg
ts-figure-optimize
ts-paper-latex
ts-paper-experiment
No numbers, ever
Forward-looking prose only. A concrete metric in a sentence — “18.3%”, “0.72 F1”, “2.5×”, even “doubles” — hard-fails the lint; bare-integer results are caught in self-review. Result tables exist with every cell literally --; the only place a dash may appear. No results figure is drawn at all, because drawing one would fabricate data.
Every number traced
Your real results are distilled into results.facts.json — the audit ground truth. Result sections switch to definitive past tense; any decimal or percent in prose that is not in the facts file fails the build. A measured-but-missing value is written TBD and never guessed.
python skills/ts-paper/scripts/run_gates.py <workdir> all # nonzero exit = NOT done
Figure Engine
AI image models make rasters. Papers need editable vectors. This engine ships both.
The engine is decided by which section a figure lives in.
Results plots draw from real data with matplotlib — numerically exact,
born vector, never from an image model. Every other figure runs a three-act
pipeline: the official PaperBanana agent team renders the
candidate image that becomes the visual target;
ts-figure-svg reconstructs it in code as editable vector graphics
— real <rect>/<path>/<text>, every label
live text — rendering each round and comparing it against the target
until structure and appearance are recovered; a geometry audit drives at
least four repair rounds. If reconstruction stays unreliable, the DrawAI
hybrid — then the approved PNG — steps in.
Never a lossy trace.
Render candidates
The official PaperBanana pipeline — Retriever → Planner → Stylist → Visualizer → Critic — produces candidate images; a distilled image-model loop stands in when no key is configured.
Learn the look
Claude looks at the chosen render and extracts its design language — palette, type scale, spacing, idiom — into a style sheet. The PNG is also the visual target: the redraw is rendered and compared against it, round by round, until its structure and appearance are recovered.
Reconstruct in code
The figure is reconstructed as editable vector graphics through code — every label live <text>, every element a real object — with the chosen render as its visual target. Reference grounding and visual review then check the figure against the manuscript itself.
Audit & repair
Each round renders the reconstruction and compares it with the target, and a geometry audit measures the drawing — overflow, text-on-text, clipped arrowheads, sub-legible type — driving repairs for at least four rounds, with no upper bound while defects remain.
Style sheet from the render
Palette, type scale, spacing and visual idiom are read off the chosen PaperBanana candidate by eye and written down as a style sheet.
Checked against the paper
Modules, edges and labels are then checked against the figure spec and the paper’s equations — reference grounding and visual review catch a garbled label or an invented link before it ships.
Real vector objects
The figure is drawn as genuine <rect>/<path>/<text> elements — every label live, editable text, with glyphs kept Times-safe so the PDF embeds fonts cleanly.
The audit measures the drawing
Each round renders the reconstruction and compares it with the reference; a gap in layout, geometry or text placement fails the round. audit_svg.py then measures the drawing — canvas or card overflow, a clipped arrowhead, text over text, sub-legible type, font-fallback glyphs — and path soup, the signature of a traced raster.
Verified on the compiled page
The SVG exports to a vector PDF that must genuinely embed its fonts — an exporter that outlines every glyph is caught — and the figure is checked on the real compiled page. The build gate requires at least four audit rounds.
No upper bound while defects remain.
Four rounds is the floor, not the ceiling — the draw–measure–repair loop keeps going until the audit comes back clean. And if native redraw is genuinely impossible, the ladder holds: DrawAI hybrid, then the approved PNG. Never a lossy trace.
- Repair loop audit rounds per figure ≥4 gate-enforced
- Audit cost extra models · API keys 0 code-only
- Fallback hybrid SSIM vs full redraw 0.670.91 reported
- Pixel tracing paths in one auto-trace 59k banned
results.facts.json, and a vector PDF written alongside every PNG with the text kept editable.
59,430
Why not just trace the PNG?
Auto-tracing one figure produced 59,430 paths, 10 MB, and not one editable label — and it was still blurry when zoomed. “No <image> element” does not mean vector. So the reconstruction draws real objects to match the render — live labels, editable shapes — never traced pixels.
3rungs
The fallback ladder
Native SVG first. If the redraw is impossible, the vendored DrawAI hybrid keeps the approved render pixel-exact under an editable text overlay (~0.91 SSIM, key-free via ModelScope). Failing that, the approved PNG goes in as-is — full richness preserved, editability deferred. Never a lossy trace.
Experiments
Stage 8 runs what the paper claims — or files a report saying why it can’t.
After the chain delivers a gates-green first draft, ts-paper-experiment
starts automatically — no one has to ask. It maps the paper’s logic,
plans the minimum set of experiments, classifies each as
necessary, feasible, or blocked, executes only what real data
and code support, and recompiles the paper with measured numbers.
If nothing can run, it writes a requirements report and leaves the tables in
proposal form — it never invents results.
Diagnose
Three reports map the territory: the paper’s logic, its claimed contributions, and the gap between claimed and existing experiments.
Plan
Each experiment is classified necessary / feasible / blocked, with metrics, baselines and commands. Named open datasets must actually be downloaded before “blocked” may be declared.
Run
Real data and code only, seeds fixed at [1, 2, 3], cheap local runs auto-approved, costly or external-data runs held for explicit user approval.
Audit & fill
Every value is recomputed from per-seed raw logs, traced to its source file, and only then written into the paper’s own result tables — nulls and honest failures included.
6
Audit reports before any number lands
Provenance, completeness, code–paper consistency, design correctness, artifact completeness, and a single truthfulness verdict. “No value may be guessed, manually invented, or written from memory.”
weakenorremove
When a claim outruns its evidence
Anything classified CLAIMED_BUT_NOT_RUN gets its claim weakened or removed — or the run stops to ask. The final step diffs the before/after paper and issues an honesty verdict, admitting when experiments weakened the original story.
The Self-Refutation Loop — named, bounded, survived.
Long research trajectories have a failure mode the paper names the Self-Refutation Loop: the system keeps judging its own evidence insufficient for the original objective, yet keeps revising the same direction without converging. Spark-to-Paper bounds it at seven experiment–critique–revision cycles; an unresolved trajectory is terminated and written up as an honest failure report — idea, methods, experiments, results, and why the evidence fell short — and the system restarts from a new idea rather than forcing success.
Guardrails, out of the box.
| Guard | Default |
|---|---|
| Random seeds | [1, 2, 3] |
| Max runtime per experiment | 6 hours |
| Traceable results required | true |
| Ask the user before | changing a contribution · deleting a core method · a new experiment type · strong claims from weak results |
| Golden rules | 27 human-approved rules; candidates are never auto-promoted |
From the skill’s bundled paper_config.yaml and
golden_rules.md. Experiments run locally; Overleaf sync is
configurable via paper_config.yaml + .env.
Writing & Review
Journal-shaped prose, with the AI scrubbed out and a courtroom at the end.
The blueprint fixes the shape before a word is written: per-section word bands, exactly three contributions, a notation table every section must reuse. The draft is written in one holistic pass so terminology never drifts; refine right-sizes it and scrubs the tells; and then a review panel does what the rest of the suite never does — argues against the paper.
Blueprint
Title of 8–14 words, 4–6 keywords, three contributions, full notation — one reasoning pass, linted before anything downstream may start.
Real references
Floor of 40, built through Crossref and arXiv metadata — authors, venue, pages, DOI. A title-only stub is forbidden; a claim with no real paper behind it goes uncited.
Holistic draft
Method first, ~2000–3000 words; intro in exactly five paragraphs; related work by theme, never chronology. Every symbol defined before use, no raw Unicode.
Right-size + de-AI
Word bands enforced in code. Comma-soup fragments become connected prose; “delve”, “leverage”, “it is worth noting” and their kin are hunted down; citations, math and labels stay untouched.
“Can’t quote = didn’t read.”
The review panel runs N isolated reviewers — theoretical soundness, experimental design, systems validity — each seeing nothing but the paper. Every issue must carry an exact verbatim quote and a closeable criterion. Each finding then faces three perspective-diverse skeptics — misreading? already addressed? out of scope? — and survives unless a majority refute it. Fresh panels re-run until a full pass finds nothing new — within a budget-capped number of rounds. Fixes are minimal, targeted, and re-gated: the linters are re-run after the edits land, because the suite derives “green” — it never forecasts it. In the paper’s blinded evaluation, issues raised this way were verified at 74% precision.
Templates · pick a venue, keep the quality.
| Template | Venue | Style | Ref floor | Status |
|---|---|---|---|---|
ts_iieta |
Traitement du Signal | two-column, numeric citations | 40 | approximation, demo-only |
neurips |
NeurIPS (community) | single-column, author-year | 30 | approximation, demo-only |
neurips_official |
NeurIPS 2025 | official .sty, fetched verbatim |
42 | official style files |
Add any venue by dropping a templates/<name>/ directory with
a template.json and the venue’s LaTeX assets — no code
changes. The suite’s hard rule: style files are user-provided or fetched
verbatim from the official source, never fabricated.
Showcase
Seven papers. Six domains. Every number traceable.
Each paper below started as a research proposal and ran the full chain — plan → cite → write → refine → review → figure → compile — then the experiment stage ran the planned experiments and wrote the measured results back into the draft. Click any card for the compiled PDF.
Conference-format samples lead the set: one in the official ICML 2025 style and two in the official NeurIPS 2025 style (preprint option); the remaining four use the Traitement du Signal journal format. References verified via WebSearch + Crossref on every paper; all figures delivered as editable vector PDFs; integrity gates passed on all seven. Two of these domains — chronic-disease screening and PM2.5 forecasting — are dissected as case studies in the paper.
Compare
The whole arc, as drop-in skills.
The heavy autonomous scientists match the breadth — but ship as standalone Python products: Docker, Neo4j, tens of thousands of lines. The lighter skill suites stay in Claude Code — but don’t run experiments or draw figures. This is the only pure Claude Code plugin that runs the entire arc, and the only tool of any kind with an editable-vector figure engine.
Capability comparison across AI-research systems
| System | End-to-end | Runs exper. | Draws figures | Editable vectors | No standing infra. |
|---|---|---|---|---|---|
| Spark-to-Paper (ours) | ✓ | ✓ | ✓ | ✓ | ✓ |
| AI Scientist / v2 | ✓ | ✓ | ✓ | – | – |
| AutoResearchClaw | ✓ | ✓ | ✓ | – | – |
| Kosmos / Robin | ● | ✓ | ● | – | – |
| Idea2Story | – | – | ● | – | ✓ |
| ARS | ● | – | – | – | ✓ |
| CycleResearcher | ● | ✓ | – | – | – |
✓ full · ● partial · – not offered or not documented. Reproduced from Table 1 of the paper, based on each system’s own public documentation.
BibTeX
Cite this work.
If Spark-to-Paper helps your research, please cite the paper (arXiv:2608.11924).
@article{qian2026sparktopaper,
author = {Qian, Zhuoyang and Wu, Biao and Wang, Yiran and Yan, Chris D and
Dai, Desan and Zheng, Liangwei and Jiang, Jin and
Zhang, Junsheng and Wang, Wenhao},
title = {Spark-to-Paper: End-to-End Research Paper Generation
as a Composable Skill},
journal = {arXiv preprint arXiv:2608.11924},
year = {2026},
url = {https://arxiv.org/abs/2608.11924}
}