test: discussion mode #103

New Issue

2026-07-07T13:35:19+02:00

ffaerber commented

2026-07-07 13:35:19 +02:00

scaffolding for the discussion-mode test

🚀 1

ffaerber commented

2026-07-07 13:35:22 +02:00

@lead in your view, what is the single riskiest part of publish.sh right now? Just your opinion — no changes.

🚀 1

pm commented

2026-07-07 13:35:56 +02:00

@ffaerber — issue #103 is too sparse for me to plan. A few things I need before I can pick the right dev:

What is "discussion mode"? Is it a new agent-workflow phase (planning/discussion step before dev), a flag in agent.yml, a feature of the test harness itself, or something else? A link to the design doc / the issue that introduced the concept would help.
Where does the scaffolding live? .gitea/workflows/, tests/discussion-mode/, a new subdir, or alongside publish.sh / route.sh? Is there an existing test pattern in the repo I should mirror?
What framework? Bash + assertions (like the existing *.sh self-tests), pytest, bats, or a custom runner? If a new dep is needed I'd want to know now.
Scope of this PR — just the directory structure + placeholder files + a stub test that currently skips/fails, or also the first real assertion of what "discussion mode" should do?

Once I know these I can tell whether this is a @junior (drop in empty files + TODO comments) or a @senior (design the test harness shape first) job. Holding off on a plan and a DELEGATE until you fill these in.

details

🔧 0 tool calls · in 9780 · out 1752 tokens · subscription · model ollama-cloud/minimax-m3:cloud

(no tool calls — text-only reply)

_{tokens — input 9780 (fresh 9780 · cache 0w / 0r) · output 1752 · reasoning 0 · subscription}

@ffaerber — issue #103 is too sparse for me to plan. A few things I need before I can pick the right dev: 1. **What is "discussion mode"?** Is it a new agent-workflow phase (planning/discussion step before dev), a flag in `agent.yml`, a feature of the test harness itself, or something else? A link to the design doc / the issue that introduced the concept would help. 2. **Where does the scaffolding live?** `.gitea/workflows/`, `tests/discussion-mode/`, a new subdir, or alongside `publish.sh` / `route.sh`? Is there an existing test pattern in the repo I should mirror? 3. **What framework?** Bash + assertions (like the existing `*.sh` self-tests), `pytest`, `bats`, or a custom runner? If a new dep is needed I'd want to know now. 4. **Scope of this PR** — just the directory structure + placeholder files + a stub test that currently skips/fails, or also the first real assertion of what "discussion mode" should do? Once I know these I can tell whether this is a `@junior` (drop in empty files + TODO comments) or a `@senior` (design the test harness shape first) job. Holding off on a plan and a DELEGATE until you fill these in. <details> <summary>details</summary> 🔧 0 tool calls · in 9780 · out 1752 tokens · subscription · model ollama-cloud/minimax-m3:cloud _(no tool calls — text-only reply)_ <sub>tokens — input 9780 (fresh 9780 · cache 0w / 0r) · output 1752 · reasoning 0 · **subscription**</sub> </details>

lead commented

2026-07-07 13:36:49 +02:00

I'll read publish.sh to give an informed opinion.
Now I have a full read of publish.sh. My assessment:

The single riskiest part is the @qa bounce-counter regex/template pair (lines 134 & 142).

Here's my reasoning, and why it beats the other candidates:

Why it's the riskiest

It's the one place in the file where correctness depends on two independent strings staying byte-for-byte in sync — a jq ... test(...) regex at line 134 and the human-readable trigger template at line 142. There is no mechanism enforcing that; only a code comment (lines 140–141) begs the next editor to keep them aligned.
If they drift, the failure is silent and directional: the counter resets to 0, the 3-attempt cap never trips, and the qa↔dev loop can churn indefinitely — burning tokens/CI on every repo that calls this workflow, with no error surfaced.
The regex is brittle by construction: it anchors on exact prose including PR #[0-9?]+ and (fix attempt [0-9]+/3). Any reword of the ping — even punctuation — breaks the count. Our own LEARNINGS already record two real incidents in this exact area (the 1/3 → 3/3 inflation from quoting the template verbatim, and the fix that pinned template+regex together). That's a track record of recurrence, which is the best predictor of future breakage.
It's also load-bearing for autopilot safety: the cap is what returns a stuck issue to a human. A quietly-broken cap defeats a guardrail, which is worse than a visible crash.

Runner-up (honorable mention)

The MARK/🤖 trigger-gate convention (lines 32–43): the whole loop-prevention scheme hinges on marked vs. unmarked comments, and a comment posted through the wrong helper either fails to fire a run or fires an infinite one. It's architecturally central, but it's used consistently and has clearer, louder failure modes than the counter, so I rank it second.

What would de-risk the counter (opinion only, no change requested)

Replace the prose-matching count with a structured, machine-stable token in the trigger body (e.g. an invisible  marker) that the regex reads, decoupling the count from the human wording — so rewording the ping can never reset the cap.

For the actual issue #103 scaffolding work: this is a discussion, so no build from me. When @pm delegates, note that a bash/bats-style harness targeting exactly this counter (assert N/3 increments and caps at 3) would be the highest-value first test to write — and I'm happy to own the harness design if @pm wants the hardest part; the placeholder scaffolding itself is fine for @junior.

details

🔧 4 tool calls · in 112622 · out 1186 tokens · $0.2756 · model anthropic/claude-opus-4-8

🔧 read: {"filePath":"/workspace/gitea/agents/publish.sh"}
🔧 glob: ``
🔧 bash: ls -la && git ls-files | head -50
🔧 read: .gitea/workflows/scripts/publish.sh

_{tokens — input 112622 (fresh 10 · cache 32970w / 79642r) · output 1186 · reasoning 0 · $0.2756}

I'll read publish.sh to give an informed opinion. Now I have a full read of publish.sh. My assessment: The single riskiest part is the **`@qa` bounce-counter regex/template pair (lines 134 & 142)**. Here's my reasoning, and why it beats the other candidates: **Why it's the riskiest** - It's the one place in the file where **correctness depends on two independent strings staying byte-for-byte in sync** — a `jq ... test(...)` regex at line 134 and the human-readable trigger template at line 142. There is no mechanism enforcing that; only a code comment (lines 140–141) begs the next editor to keep them aligned. - If they drift, the failure is **silent and directional**: the counter resets to 0, the 3-attempt cap never trips, and the qa↔dev loop can churn indefinitely — burning tokens/CI on every repo that calls this workflow, with no error surfaced. - The regex is **brittle by construction**: it anchors on exact prose including `PR #[0-9?]+` and `(fix attempt [0-9]+/3)`. Any reword of the ping — even punctuation — breaks the count. Our own LEARNINGS already record two real incidents in this exact area (the `1/3 → 3/3` inflation from quoting the template verbatim, and the fix that pinned template+regex together). That's a track record of recurrence, which is the best predictor of future breakage. - It's also **load-bearing for autopilot safety**: the cap is what returns a stuck issue to a human. A quietly-broken cap defeats a guardrail, which is worse than a visible crash. **Runner-up (honorable mention)** - The `MARK`/`🤖` trigger-gate convention (lines 32–43): the whole loop-prevention scheme hinges on marked vs. unmarked comments, and a comment posted through the wrong helper either fails to fire a run or fires an infinite one. It's architecturally central, but it's used consistently and has clearer, louder failure modes than the counter, so I rank it second. **What would de-risk the counter (opinion only, no change requested)** - Replace the prose-matching count with a **structured, machine-stable token** in the trigger body (e.g. an invisible `` marker) that the regex reads, decoupling the count from the human wording — so rewording the ping can never reset the cap. For the actual issue #103 scaffolding work: this is a discussion, so no build from me. When @pm delegates, note that a bash/`bats`-style harness targeting exactly this counter (assert N/3 increments and caps at 3) would be the highest-value first test to write — and I'm happy to own the harness design if @pm wants the hardest part; the placeholder scaffolding itself is fine for @junior. <details> <summary>details</summary> 🔧 4 tool calls · in 112622 · out 1186 tokens · $0.2756 · model anthropic/claude-opus-4-8 🔧 **read**: `{"filePath":"/workspace/gitea/agents/publish.sh"}` 🔧 **glob**: `` 🔧 **bash**: `ls -la && git ls-files | head -50` 🔧 **read**: `.gitea/workflows/scripts/publish.sh` <sub>tokens — input 112622 (fresh 10 · cache 32970w / 79642r) · output 1186 · reasoning 0 · **$0.2756**</sub> </details>

ffaerber closed this issue

2026-07-07 13:37:30 +02:00

Sign in to join this conversation.

3 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: gitea/agents#103