agents: report tokens + $ cost on every agent comment

Each run now appends a report to the agent's reply: the tool calls it made plus
input/output token totals and the dollar cost. opencode's --format json emits
per-step `tokens` and `cost` (USD, priced from the model) on step_finish events;
build-activity-log.sh sums them across the run.

- build-activity-log.sh: compute for EVERY agent (not just devs — @pm/@qa also
  call tools and cost money); output a collapsed <details> report (summary line
  shows "N tool calls · in X · out Y · $Z"; body lists the tools + a token/cost
  breakdown). Zero-tool runs get a one-line "$Z · in X · out Y" footer.
- publish.sh: build $activity once (near the top) and append it to every agent's
  reply — @pm plan/finalize, @qa verdict/recommendations, and dev PR comments.
- agent.yml: rename the step accordingly.

Models without pricing (self-hosted ollama) report cost $0.0000.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Felix Faerber
2026-07-06 09:22:00 +03:00
co-authored by Claude Opus 4.8
parent af38a44d86
commit 6334ebe8c6
3 changed files with 52 additions and 30 deletions
+41 -14
View File
@@ -1,24 +1,51 @@
#!/usr/bin/env bash
# Build the activity log — the list of TOOL CALLS the agent made — into /tmp/activity_log.md.
# Only dev agents (mode=pr) get an activity-log comment — comment-only roles (pm/qa) do no tool calls.
# NOTE: we deliberately DO NOT include the agent's prose text parts. That final "here's what I did"
# text is just a restatement of the PR description (already published as the PR body), not a tool
# call — so it was noise in a section titled "tool calls". The log is the record of ACTIONS taken.
# Build the RUN REPORT appended to the agent's reply comment: the TOOL CALLS the agent made plus a
# usage line (input / output tokens + $ cost). Written to /tmp/activity_log.md; the Publish step
# appends it to the agent's reply. Applies to EVERY agent — @pm/@qa also call tools and cost money.
#
# Required env (provided by the workflow step): MODE
# opencode --format json emits one JSON event per line. `step_finish` events carry, per LLM step,
# .part.tokens {input, output, reasoning, cache:{read, write}} and .part.cost (USD, already computed
# by opencode from the model's pricing). We sum them across all steps of the run. Models without
# pricing (e.g. self-hosted ollama) report cost 0 — shown as $0.0000.
#
# Required env (provided by the workflow step): (none needed; reads /tmp/events.jsonl)
set -u
E=/tmp/events.jsonl
: > /tmp/activity_log.md
[ -s "$E" ] || { echo "no events — empty report"; exit 0; }
if [ "$MODE" != "pr" ]; then
echo "skipping activity log for comment-mode agent"; : > /tmp/activity_log.md; exit 0
fi
# Tool calls = the ACTIONS taken (not the agent's prose text parts).
jq -r '
def trunc(n): if length > n then (.[0:n] + "…") else . end;
select(.type=="tool_use") |
(.part.tool // "?") as $t |
((.part.state.title // (.part.state.input | tojson | trunc(160)) // "")) as $title |
"🔧 **" + $t + "**: `" + ($title | trunc(240)) + "`"
' /tmp/events.jsonl > /tmp/activity_log.md 2>/dev/null || true
n=$(wc -l < /tmp/activity_log.md 2>/dev/null || echo 0)
echo "activity log: $n tool calls"
[ "$n" -eq 0 ] && : > /tmp/activity_log.md
head -3 /tmp/activity_log.md
' "$E" > /tmp/tools.md 2>/dev/null || true
n=$(wc -l < /tmp/tools.md 2>/dev/null || echo 0); n=${n:-0}
# Usage: sum per-step tokens + cost across every step_finish event (tab-separated for `read`).
read -r COST INP OUT CR CW RE < <(jq -rs '
[ .[] | select(.type=="step_finish") | .part ] as $s
| [ ([$s[].cost // 0]|add // 0),
([$s[].tokens.input // 0]|add // 0),
([$s[].tokens.output // 0]|add // 0),
([$s[].tokens.cache.read // 0]|add // 0),
([$s[].tokens.cache.write // 0]|add // 0),
([$s[].tokens.reasoning // 0]|add // 0) ]
| @tsv' "$E" 2>/dev/null)
COST=${COST:-0}; INP=${INP:-0}; OUT=${OUT:-0}; CR=${CR:-0}; CW=${CW:-0}; RE=${RE:-0}
IN_TOTAL=$(( INP + CR + CW )) # total input context processed
COSTF=$(awk -v c="$COST" 'BEGIN{printf "$%.4f", c+0}')
echo "usage: in=$IN_TOTAL out=$OUT cost=$COSTF (fresh=$INP cache_r=$CR cache_w=$CW reasoning=$RE); tools=$n"
{
if [ "$n" -gt 0 ]; then
printf '\n\n<details>\n<summary>🔧 %s tool calls · in %s · out %s · %s</summary>\n\n' "$n" "$IN_TOTAL" "$OUT" "$COSTF"
cat /tmp/tools.md
printf '\n\n<sub>tokens — input %s (fresh %s · cache %sw / %sr) · output %s · reasoning %s · **cost %s**</sub>\n</details>' \
"$IN_TOTAL" "$INP" "$CW" "$CR" "$OUT" "$RE" "$COSTF"
else
printf '\n\n<sub>💰 **%s** · in %s · out %s tokens (cache %sw / %sr)</sub>' "$COSTF" "$IN_TOTAL" "$OUT" "$CW" "$CR"
fi
} > /tmp/activity_log.md