Engineering Paper Skills

Quality Gates

This repository does not require GitHub releases. External reviews should cite the current commit hash plus recorded model-run and eval artifacts.

Normal skill users do not need to run this page's scripts. Install only skills/_shared and skills/engineering-* for ordinary Codex use. The commands below are maintainer/reviewer checks; see scripts/README.md for the script inventory.

Required Repository Checks

python scripts/validate_repo.py
python scripts/check_expected_behavior.py --spec-dir tests/expected
python scripts/check_expected_behavior.py --spec-dir tests/expected --outputs-dir tests/outputs/golden
python scripts/check_quality_assets.py
python scripts/check_response_diff.py
python scripts/check_venue_profiles.py
python scripts/check_metadata_files.py
python scripts/write_behavior_eval_stub.py --help

Behavior Evidence

Behavior evidence requires a real model command template. Without it, prompt regression is NOT_RUN.

python scripts/run_prompt_regression.py \
  --require-command \
  --case full_paper_realistic_audit \
  --output-dir /tmp/engineering-paper-skill-regression \
  --command-template "$PROMPT_REGRESSION_COMMAND_TEMPLATE"

A successful manual behavior run writes behavior_eval_stub.jsonl with the commit hash, output hash, artifact name, and ci_single_run evidence level. It remains unscored until human or semantic review is added.

Failed local behavior attempts are kept when they expose a real contract gap. The 6bec865 and 80d9c23 structured-audit attempts are recorded as local_single_run_failed, not as passing model runs.

Example Provenance

README and Pages examples are split into illustrative examples and recorded local Codex outputs. Illustrative examples are hand-written and must be labeled as not recorded model-run outputs. Recorded examples must link to prompt fixtures, raw output artifacts, and a manifest with hashes.

Current recorded demo outputs are tracked in tests/outputs/model_runs/demo/manifest.json, evals/results/681d305_demo_outputs_model_eval.jsonl, and evals/results/2dc9571_hero_demo_model_eval.jsonl.

Evidence Boundary

Golden outputs, static checks, and local model-run artifacts are review evidence. They are not proof of stable top-tier behavior on arbitrary unseen manuscripts.

No-release Versioning

Use KNOWN_GOOD.md as the current commit-based beta anchor and CHANGELOG.md for quality-gate history.