gateway: persistent PSS key + retry resolveSelfOverlay

Two bugs that together blackholed every reverse-PSS message from the
provider when the api ran more than one replica:

1) `pssKeys = generatePssKeypair()` ran per-container, so each replica
   had a different ECIES identity. Provider encrypts its reply to ONE
   of those keys, only that replica can decrypt — so two replicas =
   ~50% of chats hang forever.

2) `resolveSelfOverlay` swallowed errors and returned 0x00…00 when
   Bee wasn't ready yet at boot. That zeroed overlay landed in the
   gateway context AND in the job_notify envelope sent to the provider
   — provider then sent PSS replies into the void.

Fix: load PSS key from PSS_KEY_PATH (shared volume across replicas);
retry getNodeAddresses up to 5 minutes and throw if still zero.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Felix FaerberandClaude Opus 4.7 committed 2026-05-28 11:49:54 +03:00
1 parent 9503b19378
commit fcf184b016
2 files changed
+26 -8

No files matched your search

+20 -8
View File
@@ -14,7 +14,7 @@ import { discoverUsableBatchId, downloadChunk, PssTransport } from '../t4t/lib/s
import { ensureManagedStamp, topUpIfBelow, type ManagedStamp } from '../t4t/lib/stamps';
import { clientTopic } from '../t4t/lib/envelope';
import { EciesCipher, jsonDecrypt } from '../t4t/lib/crypto';
import { generatePssKeypair } from '../t4t/lib/keys';
import { loadOrCreatePssKey } from '../t4t/lib/keys';
import { ModelDiscovery } from '../t4t/modes/gateway/models';
import type {
JobAckBody,
@@ -89,12 +89,16 @@ async function bootGatewayContext(): Promise<GatewayContext> {
xbzz: env.XBZZ_ADDRESS as Hex,
});
// Operator PSS keypair — non-persistent for now; tenants reuse this single
// operator key (providers see all jobs as coming from the same client PSS
// identity, which matches the on-chain wallet model).
const pssKeys = generatePssKeypair();
// Operator PSS keypair — persisted to disk so every api replica shares the
// same identity. Without this, each replica generates a fresh keypair at
// boot and providers encrypt their PSS replies to a key only one replica
// can decrypt, so most chat completions hang.
const pssKeys = loadOrCreatePssKey(env.PSS_KEY_PATH);
const cipher = new EciesCipher(pssKeys.privateKey);
// Block until Bee returns a non-zero overlay; without this, a replica that
// races Bee's boot persists `selfOverlay = 0x00…00` in its gateway context
// and providers send PSS replies into the void.
const selfOverlay = await resolveSelfOverlay(bee);
const pss = new PssTransport({ bee, postageBatchId, logger, selfAddress: chain.address });
@@ -297,9 +301,17 @@ async function resolvePostageBatch(bee: Bee): Promise<string> {
}
async function resolveSelfOverlay(bee: Bee): Promise<Hex> {
const info = await bee.getNodeAddresses().catch(() => null);
const overlay = info?.overlay?.toString() ?? '00'.repeat(32);
return ('0x' + overlay.replace(/^0x/, '')) as Hex;
const zero = '0x' + '00'.repeat(32);
for (let attempt = 0; attempt < 60; attempt++) {
const info = await bee.getNodeAddresses().catch(() => null);
const overlay = info?.overlay?.toString();
if (overlay && overlay.replace(/^0x/, '') !== '00'.repeat(32)) {
return ('0x' + overlay.replace(/^0x/, '')) as Hex;
}
logger.warn({ attempt }, 'bee.getNodeAddresses returned no overlay yet — retrying in 5s');
await new Promise((r) => setTimeout(r, 5000));
}
throw new Error('bee did not return a non-zero overlay after 60 attempts — refusing to boot with a zeroed selfOverlay');
}
/** Update a GatewayJob row. Direct prisma write (not model.gatewayJob.update)