Commit b689dc2a authored by Matthias Simon's avatar Matthias Simon
Browse files

Keep long ETSI reviews inside one context per subagent.

Shard terminology and consistency checks, merge the report in two levels, and scan editorial issues once so a multi-thousand-line Markdown draft does not load every finding into a single agent.

Co-authored-by: default avatarCursor <cursoragent@cursor.com>
parent f3724530
Loading
Loading
Loading
Loading
Loading
+13 −7
Changes for .agents/skills/reviewing-etsi-standards/SKILL.md: 13 added lines, 7 removed lines.
Original line number Diff line number Diff line
@@ -14,6 +14,9 @@ Files in this skill:
  language, conditions, options, notations, references, maintenance).
- [delegation.md](delegation.md): how to split a long review across subagents.
- [scripts/scan.sh](scripts/scan.sh): the mechanical search.
- [scripts/shard-ledgers.sh](scripts/shard-ledgers.sh) and
  [scripts/assemble.sh](scripts/assemble.sh): split the cross-section checks and assemble
  the report of a long review (see delegation.md).

Sources: `assets/AGuideToWritingWorldClassStandards.pdf`, `assets/ETSI-Drafting-Rules-v53.pdf`,
`assets/ETSI-Use-of-English.pdf`, `assets/ETSI-Spelling-List.pdf`.
@@ -40,7 +43,9 @@ extract the text with `pdftotext -layout` and count pages.
- Longer: delegate. Read [delegation.md](delegation.md) now and follow it. The parent
  orchestrates and does not read the text; subagents build the term inventory, review the
  sections in parallel, check terminology and consistency, verify the Major findings, and
  write the report, all through files in `reviews/<name>/`.
  write the report, all through files in `reviews/<name>/`. Section size, the parts to
  exclude (front matter, grammar annexes) and when to shard depend on the length; see the
  table in delegation.md.
- The summary states coverage: `Reviewed: … Not reviewed: …` (or `Not reviewed: none`).

**1. Identify the document.** Determine the deliverable type (TS, ES, EN, TR, EG, SR, GS, GR),
@@ -61,7 +66,7 @@ definition and location. In this repository the terms clause is
term usage against the same list.

**3. Mechanical scan.** Run `scripts/scan.sh <file>...` (`-c` for counts only, `-r FROM:TO`
for one section). It searches
for one section, `-k`/`-x` to run or skip comma-separated categories). It searches
prose only (Markdown code is blanked; line numbers match the file) for wrong provision verbs,
vague words, open lists, `and`/`or` and condition words, quantifiers, unclear antecedents,
double negation, statements of belief, gap markers, personal pronouns, contractions, and
@@ -126,16 +131,17 @@ wherever the intent is evident. Do not edit the document unless the user asks fo
  than about ten times, give the count, the pattern, and three examples.
- More than about 20 findings, or a long text: write the full report to
  `reviews/<name>/report.md` (`reviews/<name>.md` for a single short text) and put the
  summary in the chat.
  summary in the chat. For a long text the report is merged in two levels and assembled by
  `scripts/assemble.sh` (delegation.md, phase 6).

## Long texts

Consistency, terminology and gaps span sections, but no single context window holds a long
text with all its findings. [delegation.md](delegation.md) assigns the steps above to
subagents (term inventory; section reviewers in parallel; terminology checker; consistency
checker; verifier; report writer), defines the files they exchange in `reviews/<name>/`,
and gives a self-contained brief for each. Each subagent returns a short summary; the
findings stay in files.
subagents (term inventory; editorial scanner; section reviewers in parallel; terminology and
consistency checkers, sharded for long texts; verifiers; group mergers; report writer),
defines the files they exchange in `reviews/<name>/`, and gives a self-contained brief for
each. Each subagent returns a short summary; the findings stay in files.

## Report template

+139 −41
Changes for .agents/skills/reviewing-etsi-standards/delegation.md: 139 added lines, 41 removed lines.
Original line number Diff line number Diff line
@@ -3,7 +3,8 @@
A long review does not fit in one context window: the text, the scan hits, the lookups and
the findings together crowd out the attention needed for each sentence. The parent agent
therefore orchestrates and does not review. Subagents do the reading, exchange results
through files in `reviews/<name>/`, and return short summaries.
through files in `reviews/<name>/`, and return short summaries. Steps that would need the
whole text or all findings in one context are split, or done by a script.

## Contents
- When to delegate
@@ -17,14 +18,22 @@ through files in `reviews/<name>/`, and return short summaries.
| Text | Plan |
|------|------|
| Up to about 300 lines | No subagents. Review inline. Use a lookup subagent only for a lookup that needs more than two or three searches. |
| About 300 to 1 500 lines | Delegate every role below. Section reviewers run in parallel. |
| More than 1 500 lines, or several files | As above, in batches of at most 4 section reviewers at a time. Split the term inventory by file if the terms clauses are long. |
| About 300 to 1 500 lines | Delegate every role below. Sections of about 300 lines. Section reviewers run in parallel. One terminology checker, one consistency checker, one report writer. |
| More than 1 500 lines, or several files | Sections of about 800 lines, in batches of at most 8 section reviewers at a time (fewer if the environment limits parallel subagents). Split the term inventory by file if the terms clauses are long. Shard the cross-section checks and merge the report in two levels (phases 4 and 6). |

Do not give a section reviewer front matter or notation listings. List these ranges in
`plan.md` as excluded, and under `Not reviewed` in the report unless the user asks for them:
- title page, table of contents, IPR notice, foreword, modal verbs terminology, history;
- normative notation annexes (BNF grammar, ASN.1, XSD, long code listings). Reviewers and
  checkers still read them for bounded lookups when prose must agree with them. If the user
  asks for them, review them as a consistency check against the prose, not with the
  sentence-level tests.

## Rules for the parent

- Read only: the heading list, line counts, the summaries the subagents return, and the
  Summary section of `report.md`. Do not read the reviewed text, the scan output, the
  findings files or the ledger files. If a summary raises a question, ask the subagent
- Read only: the heading list, line counts, the summaries the subagents and scripts return,
  and the Summary section of `report.md`. Do not read the reviewed text, the scan output,
  the findings files or the ledger files. If a summary raises a question, ask the subagent
  again (resume it) or start a lookup subagent.
- Each brief is self-contained. Subagents do not see the conversation: give the document id
  and type, the exact files and line ranges, the skill path, the output files, and the
@@ -40,31 +49,56 @@ through files in `reviews/<name>/`, and return short summaries.

## Working files

All in `reviews/<name>/`. Each file has one writer, so parallel subagents never write to the
same file.
All in `reviews/<name>/`. Each file has one writer at a time, so parallel subagents never
write to the same file.

| File | Writer | Content |
|------|--------|---------|
| `plan.md` | parent | document id and type, sections with file and line ranges, status of each role |
| `plan.md` | parent | document id and type, sections with file and line ranges, excluded ranges, groups for the report merge, status of each role |
| `terms.md` | term inventory | term, definition, location; one line per term |
| `findings-<section>.md` | section reviewer | findings in the report format, written as they are found |
| `ledger-<section>.md` | section reviewer | `concept — claim — file:line`, one line per statement that defines or constrains a concept; then `new term — file:line` for terms not in `terms.md` |
| `findings-terminology.md` | terminology checker | definition and term-use findings across sections |
| `findings-consistency.md` | consistency checker | contradictions and unexplained differences across sections |
| `verification.md` | verifier | per Major finding: `confirmed`, `changed` (with the correction) or `rejected` (with the reason) |
| `report.md` | report writer | the merged report |
| `siblings.md` | term inventory | analogous concepts, one group per line, separated by commas (for example `constant, variable, module parameter, template variable, formal parameter`) |
| `findings-editorial.md` | editorial scanner | language and spelling findings for the whole text |
| `findings-<section>.md` | section reviewer | findings in the report format, numbered, written as they are found |
| `ledger-<section>.md` | section reviewer | `concept — claim — file:line`, one line per statement that defines or constrains a concept |
| `concepts-<k>.md` | `scripts/shard-ledgers.sh` | the ledger lines of one shard, grouped by concept |
| `findings-terminology[-<k>].md` | terminology checker | definition and term-use findings across sections |
| `findings-consistency[-<k>].md` | consistency checker | contradictions and unexplained differences across sections |
| `verification-<findings>.md` | verifier | per Major finding of one or more findings files: `confirmed`, `changed` (with the correction) or `rejected` (with the reason) |
| `merged-<nn>-<group>.md` | group merger, then report writer | verified and deduplicated findings of one group, in the report format |
| `summary.md` | report writer | title and Summary section of the report |
| `report.md` | report writer, or `scripts/assemble.sh` | the merged report |

In the ledgers, `concept` is the term exactly as written in `terms.md`. A concept that is not
in `terms.md` is written as `+<term>`, which also marks it for the terminology checker.
Scripts group concepts case-insensitively.

## Roles and order

1. **Parent**: steps 0 and 1 of SKILL.md. Choose sections (a top-level clause, or about 300
   lines, cut at headings). Write `plan.md`.
2. **Term inventory** (one subagent): step 2. Must finish before phase 3.
3. **Section reviewers** (one per section, parallel): steps 3, 4 and 5 for their section.
4. **Terminology checker** and **consistency checker** (parallel): step 6, from `terms.md`
1. **Parent**: steps 0 and 1 of SKILL.md. Choose sections at headings (a top-level clause,
   or the section size from the table above) and the excluded ranges. Write `plan.md`.
2. **Term inventory** and **editorial scanner** (parallel): step 2, and the language checks
   of step 4 for the whole text. The term inventory must finish before phase 3.
3. **Section reviewers** (one per section, parallel in batches): steps 3, 4 and 5 for their
   section. On long texts they skip the editorial scan categories.
4. **Terminology checkers** and **consistency checkers** (parallel): step 6, from `terms.md`
   and the ledgers, with bounded lookups into the text.
5. **Verifier** (one subagent, or one per findings file for many Major findings): rereads the
   cited text of every Major finding and confirms it independently.
6. **Report writer** (one subagent): step 8, from all findings files and `verification.md`.
   - Up to about 1 000 ledger lines (`cat reviews/<name>/ledger-*.md | wc -l`): one
     consistency checker reads all ledgers.
   - More: run `scripts/shard-ledgers.sh reviews/<name> <k>` with `<k>` = ledger lines / 1 000,
     rounded up. Start one consistency checker per `concepts-<k>.md`.
   - Up to about 150 terms: one terminology checker. More: split `terms.md` into line ranges
     of about 150 terms, one checker each; the first one also checks the confusable pairs.
5. **Verifiers** (one per findings file, or one per few findings files with few Major
   findings): reread the cited text of every Major finding and confirm it independently.
6. **Report**:
   - Up to about 6 findings files: one report writer does step 8 from all findings files
     and verification files and writes `report.md`.
   - More: the parent groups the findings files in `plan.md`: one group per top-level clause
     or about 6 files, plus one group for `findings-editorial.md`, `findings-terminology*.md`
     and `findings-consistency*.md`. One **group merger** per group (parallel) writes
     `merged-<nn>-<group>.md`, with `<nn>` giving the order of the report (the cross-section
     group first). Then one **report writer** removes duplicates across groups, writes
     `summary.md` and runs `scripts/assemble.sh`. No subagent reads all findings at once.
7. **Parent**: step 7 if the text is a revision or CR (delegate it like a section review if
   the affected clauses are long). Read the report Summary, check coverage, and give the
   user the summary and the report path.
@@ -76,7 +110,7 @@ Common header for every brief:
```text
You are part of a review of <document id> (<deliverable type>, normative|informative)
against the ETSI quality rules. Skill: .agents/skills/reviewing-etsi-standards
(SKILL.md, quality-checks.md, reference.md, scripts/scan.sh). Working directory:
(SKILL.md, quality-checks.md, reference.md, scripts/). Working directory:
reviews/<name>/. Do not edit the reviewed document. Quote text exactly and cite file:line.
Return at most 15 lines; never return the findings themselves.
```
@@ -86,10 +120,26 @@ Return at most 15 lines; never return the findings themselves.
```text
<common header>
Task: step 2 of SKILL.md ("Terminology check", step 1, in quality-checks.md).
Sources: <terms clause file>, plus inline definitions in <files and line ranges>.
Sources: <terms clause file and lines>, plus inline definitions in <files and line ranges>.
Write reviews/<name>/terms.md: one line per term: `term — definition — file:line`.
Mark empty definitions and definitions that contain requirements with `(!)`.
Return: number of terms, number marked (!), and the file path.
Write reviews/<name>/siblings.md: groups of analogous concepts (constructs whose rules
should match unless the text says why they differ), one group per line, terms exactly as in
terms.md, separated by commas.
Return: number of terms, number marked (!), number of sibling groups, and the file paths.
```

### Editorial scanner

```text
<common header>
Task: the language checks of step 4 of SKILL.md for the whole text (<files>, excluding
<excluded ranges>). Run `scripts/scan.sh -k person,contraction,spelling,ise-spelling <files>`
and read each hit in its sentence; skip hits in code, quotations and references.
Write reviews/<name>/findings-editorial.md in the report format: a `### Editorial` table,
and a `### Minor` finding for a pattern that changes meaning. A pattern with more than
about ten hits is one row with the count and three locations.
Return: hits read, findings by severity, and the file path.
```

### Section reviewer
@@ -97,13 +147,20 @@ Return: number of terms, number marked (!), and the file path.
```text
<common header>
Task: steps 3, 4 and 5 of SKILL.md for <file> lines <from>-<to> (<clause title>) only.
Read quality-checks.md in full and the parts of reference.md that apply.
Run `scripts/scan.sh -r <from>:<to> <file>` for the scan hits of this section.
Read in SKILL.md: steps 3 to 5, "Report template", "Severity levels" and "Example findings".
Read quality-checks.md in full. Read in reference.md only: "Provision verbs", "Context and
conditions", "Combining requirements", "Optional requirements and profiles".
Run `scripts/scan.sh -r <from>:<to> <-x person,contraction,spelling,ise-spelling> <file>`
for the scan hits of this section. Language and spelling are checked elsewhere; report them
only where they change the meaning of a requirement.
Judge term usage against reviews/<name>/terms.md; do not edit it.
Read other text only for bounded lookups (a definition, a cited clause, a sibling rule).
Write findings to reviews/<name>/findings-<section>.md as you find them.
Write reviews/<name>/ledger-<section>.md: every statement in the section that defines or
constrains a concept, then the technical terms you met that are not in terms.md.
Write findings to reviews/<name>/findings-<section>.md as you find them, under the headings
`### Major`, `### Minor`, `### Editorial` and `### Open questions` of the report template,
numbered from 1 within each severity.
Write reviews/<name>/ledger-<section>.md: one line `concept — claim — file:line` for every
statement in the section that defines or constrains a concept. `concept` is the term exactly
as in terms.md, or `+<term>` for a technical term that is not in terms.md.
Return: findings by severity, lines reviewed and not reviewed, open questions (one line each).
```

@@ -112,11 +169,14 @@ Return: findings by severity, lines reviewed and not reviewed, open questions (o
```text
<common header>
Task: step 6 of SKILL.md, terminology only ("Terminology check", steps 2, 4 and 5, in
quality-checks.md). Inputs: reviews/<name>/terms.md and the "new term" lines of
reviews/<name>/ledger-*.md. Search the reviewed text (<files>) for each term and its
confusable pairs; read only the matching lines and their paragraph.
quality-checks.md). Inputs: reviews/<name>/terms.md <lines <from>-<to> | in full> and the
lines of reviews/<name>/ledger-*.md whose concept starts with `+` <that belong to these
terms | all>. <Also check the confusable pairs. | Do not check the confusable pairs.>
Search the reviewed text (<files>) for each term; read only the matching lines and their
paragraph.
Skip problems already reported in reviews/<name>/findings-*.md (search them by term).
Write reviews/<name>/findings-terminology.md.
Write reviews/<name>/findings-terminology<-k>.md, using the section headings and
numbering of the section reviewer brief.
Return: findings by severity and the file path.
```

@@ -125,10 +185,12 @@ Return: findings by severity and the file path.
```text
<common header>
Task: step 6 of SKILL.md, consistency only (section "Consistent" in quality-checks.md).
Inputs: reviews/<name>/ledger-*.md. Group the lines by concept and compare the claims.
Compare analogous constructs (siblings) across sections. Read the cited lines to confirm a
Input: <reviews/<name>/concepts-<k>.md | reviews/<name>/ledger-*.md>, and
reviews/<name>/siblings.md. Group the lines by concept and compare the claims. Compare the
analogous concepts of each sibling group with each other. Read the cited lines to confirm a
contradiction before reporting it.
Write reviews/<name>/findings-consistency.md.
Write reviews/<name>/findings-consistency<-k>.md, using the section headings and numbering
of the section reviewer brief.
Return: findings by severity and the file path.
```

@@ -136,20 +198,38 @@ Return: findings by severity and the file path.

```text
<common header>
Task: verify every Major finding in reviews/<name>/findings-*.md without trusting it.
Task: verify every Major finding in <reviews/<name>/findings-<x>.md, ...> without trusting it.
For each one: check that the quote is exact and at the cited location; reread the
paragraph and any cited definition; decide whether the problem is real, whether the
severity is right, and whether the proposal is correct and does not break other text.
Write reviews/<name>/verification.md: `<file>#<number> — confirmed|changed|rejected — reason`.
Write reviews/<name>/verification-<x>.md: `<findings file>#<number> — confirmed|changed|rejected — reason`.
Return: counts of confirmed, changed and rejected.
```

### Group merger

```text
<common header>
Task: part of step 8 of SKILL.md for one group. Inputs: <findings files of the group> and
the verification files that cover them (reviews/<name>/verification-*.md).
Drop rejected findings and apply the changes. Merge findings that describe the same problem
into one with all locations. Order Major findings by impact.
<For the cross-section group: also drop findings already reported in a section findings file
(search reviews/<name>/findings-*.md by location and term).>
Write reviews/<name>/merged-<nn>-<group>.md with the headings `### Major`, `### Minor`,
`### Editorial` (table) and `### Open questions` of the report template, findings numbered
from 1 within each severity, each starting `N. **<Quality>: <defect>**`.
Return: findings by severity after merging, and the file path.
```

### Report writer

Up to about 6 findings files:

```text
<common header>
Task: step 8 of SKILL.md. Inputs: reviews/<name>/plan.md, reviews/<name>/findings-*.md,
reviews/<name>/verification.md. Drop rejected findings and apply the changes.
reviews/<name>/verification-*.md. Drop rejected findings and apply the changes.
Merge findings that describe the same problem into one with all locations. Order Major
findings by impact. Fill in the quality table and the coverage line from plan.md and the
section returns: <coverage per section>.
@@ -157,6 +237,24 @@ Write reviews/<name>/report.md using the report template in SKILL.md.
Return: the Summary section of the report, and the file path.
```

More findings files (after the group mergers):

```text
<common header>
Task: step 8 of SKILL.md, final assembly. Inputs: reviews/<name>/plan.md and
reviews/<name>/merged-*.md. Do not read the merged files in full.
1. Run `scripts/assemble.sh -c reviews/<name>`: the quality table, totals, and the first line
   of every Major finding with its file.
2. Find Major findings in different files that describe the same problem. For each, keep
   one: add the other locations to it and delete the others from their merged files.
3. Write reviews/<name>/summary.md: the report title and `## Summary` with the coverage line
   (<coverage per section and excluded ranges>) and 2–4 sentences naming the most
   important Major findings by defect and location; assembly renumbers the findings.
4. Run `scripts/assemble.sh reviews/<name>`. It writes report.md: the summary, the quality
   table, and all findings renumbered in group order.
Return: the Summary section of the report, and the file path.
```

### Lookup

```text
+104 −0
Changes for .agents/skills/reviewing-etsi-standards/scripts/assemble.sh: 104 added lines, 0 removed lines.
Original line number Diff line number Diff line
#!/usr/bin/env bash
# Assembles the report of a long review from the merged findings files.
#
# Usage: assemble.sh [-c] DIR
#   -c  print only the quality table, the totals and the first line of each Major finding
#
# Reads DIR/merged-*.md in name order. Each file has the sections `### Major`, `### Minor`,
# `### Editorial` (table rows) and `### Open questions`, as in the report template; Major and
# Minor findings start with `N. **Quality: defect**`. Writes DIR/report.md: DIR/summary.md
# (title and Summary section), the quality table, then all findings renumbered in file order.
set -euo pipefail

count_only=0
if [[ "${1:-}" == -c ]]; then
    count_only=1
    shift
fi
[[ $# -eq 1 ]] || {
    echo "usage: $0 [-c] DIR" >&2
    exit 2
}
dir=$1
shopt -s nullglob
merged=("$dir"/merged-*.md)
((${#merged[@]})) || {
    echo "$0: no merged-*.md files in $dir" >&2
    exit 1
}
if ((!count_only)) && [[ ! -f "$dir/summary.md" ]]; then
    echo "$0: $dir/summary.md is missing" >&2
    exit 1
fi

assemble() {
    awk -v count_only="$count_only" '
        FNR == 1 { sec = ""; item = 0 }
        /^### / {
            s = tolower($0)
            sec = s ~ /major/ ? "major" : s ~ /minor/ ? "minor" : s ~ /editorial/ ? "editorial" : s ~ /open question/ ? "open" : ""
            item = 0
            next
        }
        /^#{1,2} / { sec = ""; item = 0; next }
        sec == "major" || sec == "minor" {
            if (match($0, /^[0-9]+\. /)) {
                k = ++num[sec]
                rest = substr($0, RLENGTH + 1)
                q = "Unlabelled"
                if (match(rest, /^\*\*[^:*]+:/)) q = substr(rest, 3, RLENGTH - 3)
                if (!(q in seen)) { seen[q]; extra[++nq] = q }
                count[q, sec]++
                body[sec] = body[sec] k ". " rest "\n"
                if (sec == "major") heads = heads FILENAME ": " rest "\n"
                item = 1
            } else if (item) {
                body[sec] = body[sec] $0 "\n"
            }
            next
        }
        sec == "editorial" && /^\|/ {
            if ($0 ~ /^\|[-:| ]+\|[[:space:]]*$/ || $0 ~ /^\|[[:space:]]*Location[[:space:]]*\|/) next
            editorial = editorial $0 "\n"
            ned++
            next
        }
        sec == "open" && NF { open = open $0 "\n"; if ($0 ~ /^- /) nopen++ }
        END {
            split("Necessary Unambiguous Complete Precise Consistent Testable", six, " ")
            label["Complete"] = "Complete (including semantic gaps)"
            label["Precise"] = "Precise (including terminology)"
            print "| Quality | Major | Minor |"
            print "|---------|-------|-------|"
            for (i = 1; i <= 6; i++) {
                q = six[i]
                printf "| %s | %d | %d |\n", (q in label ? label[q] : q), count[q, "major"], count[q, "minor"]
                fixed[q]
            }
            for (i = 1; i <= nq; i++) {
                q = extra[i]
                if (!(q in fixed)) printf "| %s | %d | %d |\n", q, count[q, "major"], count[q, "minor"]
            }
            print ""
            if (count_only) {
                printf "Totals: %d Major, %d Minor, %d Editorial rows, %d open questions\n\n", num["major"], num["minor"], ned, nopen
                printf "%s", heads
                exit
            }
            print "## Findings\n"
            printf "### Major\n%s\n", num["major"] ? body["major"] : "None.\n"
            printf "### Minor\n%s\n", num["minor"] ? body["minor"] : "None.\n"
            print "### Editorial"
            if (ned) printf "| Location | Text | Proposal |\n|----------|------|----------|\n%s\n", editorial
            else print "None.\n"
            printf "## Open questions for the committee\n%s", nopen ? open : "None.\n"
        }
    ' "${merged[@]}"
}

if ((count_only)); then
    assemble
else
    { cat "$dir/summary.md"; echo; assemble; } >"$dir/report.md"
    echo "$dir/report.md"
fi
+20 −9

File changed.

Preview size limit exceeded, changes collapsed.

+75 −0

File added.

Preview size limit exceeded, changes collapsed.