agents: report tokens + $ cost on every agent comment
Each run now appends a report to the agent's reply: the tool calls it made plus input/output token totals and the dollar cost. opencode's --format json emits per-step `tokens` and `cost` (USD, priced from the model) on step_finish events; build-activity-log.sh sums them across the run. - build-activity-log.sh: compute for EVERY agent (not just devs — @pm/@qa also call tools and cost money); output a collapsed <details> report (summary line shows "N tool calls · in X · out Y · $Z"; body lists the tools + a token/cost breakdown). Zero-tool runs get a one-line "$Z · in X · out Y" footer. - publish.sh: build $activity once (near the top) and append it to every agent's reply — @pm plan/finalize, @qa verdict/recommendations, and dev PR comments. - agent.yml: rename the step accordingly. Models without pricing (self-hosted ollama) report cost $0.0000. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
af38a44d86
commit
6334ebe8c6
@@ -1,24 +1,51 @@
|
||||
#!/usr/bin/env bash
|
||||
# Build the activity log — the list of TOOL CALLS the agent made — into /tmp/activity_log.md.
|
||||
# Only dev agents (mode=pr) get an activity-log comment — comment-only roles (pm/qa) do no tool calls.
|
||||
# NOTE: we deliberately DO NOT include the agent's prose text parts. That final "here's what I did"
|
||||
# text is just a restatement of the PR description (already published as the PR body), not a tool
|
||||
# call — so it was noise in a section titled "tool calls". The log is the record of ACTIONS taken.
|
||||
# Build the RUN REPORT appended to the agent's reply comment: the TOOL CALLS the agent made plus a
|
||||
# usage line (input / output tokens + $ cost). Written to /tmp/activity_log.md; the Publish step
|
||||
# appends it to the agent's reply. Applies to EVERY agent — @pm/@qa also call tools and cost money.
|
||||
#
|
||||
# Required env (provided by the workflow step): MODE
|
||||
# opencode --format json emits one JSON event per line. `step_finish` events carry, per LLM step,
|
||||
# .part.tokens {input, output, reasoning, cache:{read, write}} and .part.cost (USD, already computed
|
||||
# by opencode from the model's pricing). We sum them across all steps of the run. Models without
|
||||
# pricing (e.g. self-hosted ollama) report cost 0 — shown as $0.0000.
|
||||
#
|
||||
# Required env (provided by the workflow step): (none needed; reads /tmp/events.jsonl)
|
||||
set -u
|
||||
E=/tmp/events.jsonl
|
||||
: > /tmp/activity_log.md
|
||||
[ -s "$E" ] || { echo "no events — empty report"; exit 0; }
|
||||
|
||||
if [ "$MODE" != "pr" ]; then
|
||||
echo "skipping activity log for comment-mode agent"; : > /tmp/activity_log.md; exit 0
|
||||
fi
|
||||
# Tool calls = the ACTIONS taken (not the agent's prose text parts).
|
||||
jq -r '
|
||||
def trunc(n): if length > n then (.[0:n] + "…") else . end;
|
||||
select(.type=="tool_use") |
|
||||
(.part.tool // "?") as $t |
|
||||
((.part.state.title // (.part.state.input | tojson | trunc(160)) // "")) as $title |
|
||||
"🔧 **" + $t + "**: `" + ($title | trunc(240)) + "`"
|
||||
' /tmp/events.jsonl > /tmp/activity_log.md 2>/dev/null || true
|
||||
n=$(wc -l < /tmp/activity_log.md 2>/dev/null || echo 0)
|
||||
echo "activity log: $n tool calls"
|
||||
[ "$n" -eq 0 ] && : > /tmp/activity_log.md
|
||||
head -3 /tmp/activity_log.md
|
||||
' "$E" > /tmp/tools.md 2>/dev/null || true
|
||||
n=$(wc -l < /tmp/tools.md 2>/dev/null || echo 0); n=${n:-0}
|
||||
|
||||
# Usage: sum per-step tokens + cost across every step_finish event (tab-separated for `read`).
|
||||
read -r COST INP OUT CR CW RE < <(jq -rs '
|
||||
[ .[] | select(.type=="step_finish") | .part ] as $s
|
||||
| [ ([$s[].cost // 0]|add // 0),
|
||||
([$s[].tokens.input // 0]|add // 0),
|
||||
([$s[].tokens.output // 0]|add // 0),
|
||||
([$s[].tokens.cache.read // 0]|add // 0),
|
||||
([$s[].tokens.cache.write // 0]|add // 0),
|
||||
([$s[].tokens.reasoning // 0]|add // 0) ]
|
||||
| @tsv' "$E" 2>/dev/null)
|
||||
COST=${COST:-0}; INP=${INP:-0}; OUT=${OUT:-0}; CR=${CR:-0}; CW=${CW:-0}; RE=${RE:-0}
|
||||
IN_TOTAL=$(( INP + CR + CW )) # total input context processed
|
||||
COSTF=$(awk -v c="$COST" 'BEGIN{printf "$%.4f", c+0}')
|
||||
echo "usage: in=$IN_TOTAL out=$OUT cost=$COSTF (fresh=$INP cache_r=$CR cache_w=$CW reasoning=$RE); tools=$n"
|
||||
|
||||
{
|
||||
if [ "$n" -gt 0 ]; then
|
||||
printf '\n\n<details>\n<summary>🔧 %s tool calls · in %s · out %s · %s</summary>\n\n' "$n" "$IN_TOTAL" "$OUT" "$COSTF"
|
||||
cat /tmp/tools.md
|
||||
printf '\n\n<sub>tokens — input %s (fresh %s · cache %sw / %sr) · output %s · reasoning %s · **cost %s**</sub>\n</details>' \
|
||||
"$IN_TOTAL" "$INP" "$CW" "$CR" "$OUT" "$RE" "$COSTF"
|
||||
else
|
||||
printf '\n\n<sub>💰 **%s** · in %s · out %s tokens (cache %sw / %sr)</sub>' "$COSTF" "$IN_TOTAL" "$OUT" "$CW" "$CR"
|
||||
fi
|
||||
} > /tmp/activity_log.md
|
||||
|
||||
Reference in New Issue
Block a user