「820」以程式判定、由 Agent 解説:Luke Shin 記錄用 Kiro 開發付款檔案核對工作台的兩日經過
Verdict by code, narration by agent: two days with Kiro on a dead payment format
Luke Shin 在 Kiroween 2026 以 Kiro 製作「820」付款檔案核對工作台,讓程式負責判定、Agent 只解説已計算結果。工作台依 COBOL copybook 的位元組偏移解析固定寬度檔案,再核對貨幣總額、記錄數、參考資料和結算日期;測試套件共有 311 項測試,其中 31 項為屬性測試。
The line I keep coming back to from this build is not a feature. It is one row of a log file:
2026-11-21T09:32:22.134Z trigger=PostFileSave unit+pbt tests ran
Nobody asked for that test run. I saved a TypeScript file in a Kiro CLI session and a hook ran the unit and property suites before the agent's next turn. The row is committed in out/hook-log.txt, so anyone can read it without taking my word for it.
This post is written as part of the Kiroween 2026 hackathon, where I entered a project called 「820」 in the Resurrection category. It is a build log more than a tutorial: what I asked Kiro to do, what it did, and the places where I was confident and wrong.
The problem: the files survived, the readers did not
Corporate payment files never died. EDI 820 remittance orders, 823 confirmations and BAI2 bank statements still move between banks and ERP systems every day as fixed-width text. What each byte means is written in a COBOL copybook, and the copybook is a separate file that a modern stack has no idea what to do with.
A back office that cannot read the copybook retypes the file into a spreadsheet. That is how a payment file becomes a person's memory of a payment file.
So the tool reads the file plus its copybook and produces a verdict, with the bytes behind every finding. Four layers, and only one of them is allowed to be a model:
- Parser: types every record at the copybook's byte offsets. Trailing overpunch, leading separate sign, COMP-3 packed decimal with no line breaks at all, implied decimals. A copybook that declares
REDEFINES(one hundred bytes that are the payment in one layout and the trailer in another) is read as a region, and a record whose discriminator matches no layout is reported instead of guessed. - Reconciler: control totals per currency, record counts, references, value dates against a settlement window. Coded findings with signed
bigintdeltas. - Explainer: an optional agent layer that narrates a finding and may cite only numbers the first two layers produced. Anything else becomes
INSUFFICIENT_EVIDENCE. - Register: a human decision per finding, append only and hash chained, with an owner, a reason, a compensating control and an expiry. A waiver never turns a verdict green.
The phrase I used while building was "verdict by code, narration by agent". It is not a slogan about distrust of AI. It is a rule about what a reviewer is allowed to check: if the model produces the verdict, the only way to check the answer is to trust the model again.
Links, if you want to poke at it before reading the rest:
- Repository (public, Apache 2.0,
.kiroat the root): https://github.com/luke-shin-ai/edi820-workbench - No-install demo, the same engine compiled for the browser: https://luke-shin-ai.github.io/edi820-workbench/demo/
- Two minute 55 second demo video: https://youtu.be/rncG1wwqTAM
- Devpost entry: https://devpost.com/software/820-a-reconciliation-workbench-for-dead-payment-files
The spec came first, and the copybook was not the spec
.kiro/specs/edi820-importer/ holds 33 requirements in EARS form, in seven groups, plus an explicit out of scope section. Writing them was the slowest part of the build and the part Kiro could not do for me.
Here is one requirement the copybook cannot express, quoted from the spec (R2.2):
WHEN a record's byte length differs from the copybook record length,
THE SYSTEM SHALL emit RECORD_LENGTH_MISMATCH with record index and both lengths,
and SHALL continue parsing at the declared record boundary.
The copybook says where a field starts and how wide it is. It does not say whether a 99-byte record should be skipped, re-synchronised or reported. The easy implementation re-synchronises silently, because that makes the demo pass. Writing "SHALL continue parsing at the declared record boundary" as a requirement is what turned that convenience into a bug when the code did it anyway.
I also wrote a design.md with a delta log for every decision that changed while I was building. That log is the most useful document in the repository now, because it shows where the first idea lost: typing money as number, fetching a matched count from a human readable sentence, treating a REDEFINES region as two independent fields.
The honest boundary: I did not run the Kiro IDE spec flow from requirements to tasks to implementation inside one session. I authored the spec in the workspace from the copybooks, and the implementation ran with the spec as the contract. Kiro implemented one layer, the exception register, from spec task T7.1 in a real session, and the repository marks which evidence rows came from which session rather than implying the whole thing was generated.
Steering: the one rule that changed the code
Five steering files, and one of them visibly overruled the design. tech.md says money is never floating point. The design draft had typed numeric amounts as number, which in JavaScript is a float. The implementation uses bigint minor units with an ISO 4217 currency, and the change is recorded as implementation delta 1 in design.md section 10. A reader can watch the steering file win against the design draft, which is a more useful artifact than a sentence that says steering is helpful.
The steering strategy that made the biggest difference was writing prohibitions as rules instead of preferences: no floats, no silent re-sync, no model in the verdict path, no exception without a compensating control. An agent will happily violate a preference to make a test go green. A prohibition with a name is something a reviewer can grep for.
The reviewer agent, and the review it made up
Two custom agents live in .kiro/agents/: an implementer and an adversarial reviewer. The reviewer is not allowed to rewrite code; it reads the spec, calls the project's MCP server, and tries to break the claims.
The first version of it had no tools array in its config. It produced a fluent, confident review with file names and line references. Three of its four claims were false. It said no decoder existed for zoned or packed decimal (both are in src/parser/decode.ts). It said CURRENCY_SCALE_MISMATCH was emitted nowhere (it is emitted in src/parser/parser.ts). It said an invalid overpunch returned zero silently (it throws, and a test asserts that).
With tools: [read, grep, glob, thinking] and a prompt that says read a file before you cite it, the same agent found two real defects:
- A record with a blank reference was skipped entirely, so it never showed up as
REF_UNMATCHED. That is exactly the silent drop the tool claims it does not do. - The matched reference count was derived by string matching a sentence in a finding's detail text, instead of being counted where the matching happens.
Both are fixed with tests. The first one is now the reason the reconcile test suite has a case for a blank reference with an expected finding, not an expected skip.
That review changed the product. The explainer layer is built on the same lesson: a citation that comes from prose instead of from a computed value is refused, and the refusal is a code path (INSUFFICIENT_EVIDENCE), not a strongly worded prompt.
Hooks: three triggers, two of them named wrong
.kiro/hooks/parser-guardrails.json declares three hooks, and all three now fire in live sessions:
-
PostFileSavematching\.(ts|mjs)$andcopybooks/.*\.cpy$, which runs the unit and property suites while an agent edits. -
Stop, which finds the newest report inout/that holds a verdict and re-derives it with the real CLI entry, so a fingerprint drift or a missing run is printed into the session instead of being swallowed. -
PostTaskExec, which appends one row toEVIDENCE.mdwhen a spec task completes and then logs its own firing.
Getting there took longer than writing them, because a broken hook fails silently.
The stop hook was declared with "trigger": "AgentStop". That is the name the docs display. On Kiro CLI 3.5.0 the event is Stop, and AgentStop is accepted into the file and never matched. I found it the only way such things can be found, with a two-line probe hook that writes a file: on AgentStop nothing appeared, on Stop it fired on the next session.
Worse, the stop hook's command pointed at src/cli/index.ts, a module that exports a function and has no entry point, and at out/ while reports live in out/<timestamp>/. So it printed nothing and exited 0. A hook that does nothing and reports success is more dangerous than one that crashes, because the log looks clean. It now calls a script that resolves the newest report and hands it to the real entry point, and every firing is appended to out/hook-log.txt.
The task hook had the same kind of defect twice over. Its trigger was PostTaskExecution instead of PostTaskExec, and no session had the spec tools at all, because they only exist under --mode spec. So the hook could not have fired whatever it was called. On 2026-11-22 I ran a session with --mode spec, set spec task T9.5 to in progress and then to completed through Kiro's own task_update tool, and the hook wrote its row:
2026-11-22T01:45:25.455Z trigger=PostTaskExec evidence row appended
2026-11-22T01:45:27.543Z trigger=Stop run=final-check ... exit=0
Three earlier rows in the same log say PostTaskExecution, because the hook command was still passing the old label when it wrote them. I left them in the file. An append only log that gets rewritten when the config improves is not a log.
The lesson generalizes past Kiro: an automation you have not watched fire is a comment.
MCP: letting the reviewer ask the engine
src/mcp/ is a JSON-RPC server over stdio with five tools: parse_file, reconcile, explain_findings, verify, exceptions. Each returns the same JSON the CLI writes, because all three surfaces call src/core/ops.ts.
It is wired into the reviewer agent through .kiro/settings/mcp.json and allowedTools. In the review session the agent called @820/reconcile, @820/exceptions and @820/verify, and got back the verdict FAIL with 4 findings and the fingerprint 7af64454.... Running node scripts/mcp-call.mjs outside any agent prints the same values.
That equality is the whole point. The agent is not describing what the code probably does. It is quoting the number the code produced, and the reviewer and the command line cannot disagree, because there is only one implementation.
I wrote the server against the protocol directly, newline delimited JSON-RPC with initialize, tools/list, tools/call, -32601 for an unknown method and -32700 for a malformed line, because the MCP SDK would not install in this environment. The test opens a real stdio session against the process and sends a malformed line that must not kill the server.
Property based testing: 31 properties, 311 tests
The suite is 311 tests in 23 files, 31 of them properties over generated inputs. Four properties did most of the work:
- Amounts re-encode to the same bytes across overpunch, separate sign and packed decimal. This one found a real encoder bug: values were sized by integer digits only, so implied decimals were dropped.
- Parse, serialize, compare bytes. Identical, including CRLF, a missing final newline, and a binary COMP-3 stream with no newlines at all.
- 500 generated copybooks compile to contiguous field starts from 1, with field widths summing to the record length.
- For generated two-layout copybooks, both layouts cover the same byte range, a record written through one layout parses back with that layout's fields and none of the other's, and a discriminator that selects no layout is reported.
My overpunch table was also wrong in the first draft, in a way a unit test would have matched: it was built from 'A' + 0, so A mapped to 0 instead of +1. The property that caught it is kept as test/regressions/overpunch-and-width.test.ts, next to the input that failed.
The regressions I care about most are the ones where the tool was confidently wrong. A truncated record used to reach PASS with zero findings; it now produces INCOMPLETE and exit code 2. A REDEFINES record with a damaged type byte used to be decoded anyway; its bytes now stay undecoded and the record is reported.
What I would not claim
The temptation in a hackathon write-up is to describe the tool you meant to build. The repository states its boundaries instead of leaving them to be found: the IDE spec flow was not run, Kiro powers were not used, and the exception register is the one layer that a Kiro session implemented from the spec.
There is also a boundary that belongs in the product itself. docs/guarantees.md lists six things the tool guarantees mechanically (byte exact slicing, one layout per record, bigint arithmetic, no silently dropped reference, reproducible fingerprints, grounded explanations) and four it does not (whether a payment is legally adequate, whether an exception is acceptable, coverage of unmodelled copybook clauses, whether the bank sent the right file). It is printed by the CLI and shown in the screen, so the limits travel with the answer.
In a regulated back office, a tool that says "here is exactly where my guarantee stops and here is who decides past that line" is usable. One that says it is always right is a liability.
What changed in how I build
Before this, my loop with an agent was: describe, accept, run tests at the end. The loop now has three more steps in it, and they are all about making the agent's claims checkable.
The spec became the place where I decide, so the agent does not get to decide by writing code that passes. Steering became the list of things I refuse, written down where the agent reads them every turn. And the hooks became the reviewer that never gets tired: the tests run on save, the verdict gets re-derived when the session stops, and the evidence row is written by the workflow instead of by me at the end.
The part I did not expect is how much of the value came from the failures. The reviewer that invented three claims, the stop hook that exited 0 without doing anything, the trigger names that were accepted and never matched: each one is now a line in the repository, and each one is the reason a specific check exists.
「820」 is at https://github.com/luke-shin-ai/edi820-workbench if you want to read the spec, the delta log, the hook log and the tests. The demo runs in a browser tab with no server: https://luke-shin-ai.github.io/edi820-workbench/demo/
來源:DEV Community · dev.to