Case Study

Decisions, not process.

Latentmachine is a browser-based engine for verifying AI-transformed data and inferring deterministic rules from examples. It accepts JSON, XML, CSV, TOML, YAML, .env, and SQL INSERT as input, exports reusable code and output formats, and refuses when its evidence is insufficient. This page is not about what the tool does — you can try that yourself. It is about the decisions behind it: what was chosen, what was rejected, and why.

The project is solo work, built over several months with heavy AI-assisted development. The architectural direction, the product thinking, and the design choices are mine. The implementation was a collaboration between my judgment and AI coding agents. That distinction matters, so I am stating it upfront.

Why Verify leads

Latentmachine did not start as a verification tool. It started as Infer — transform structured data by demonstrating a before-and-after example. Verify was the last tool I built. Today it is first in the navigation, the hero of the landing page, and the first thing a new visitor is asked to try. That inversion is a positioning decision, and it is the most consequential one in the project.

The reasoning: casual data transformation is being commoditized by the same technology this tool quietly competes with. If you need thirty records reshaped once, pasting them into a chat model is a fine answer, and it gets better every quarter. Betting the product on that use case means betting against the tide. But every batch an LLM transforms creates an artifact that did not exist before: output nobody has checked. Generation and verification sit on opposite sides of the same trade — as one gets cheaper, the other becomes more valuable. Verify is positioned on the side that appreciates.

What made the decision cheap is that nothing had to be rebuilt. The engine already inferred rules from evidence; Verify points that ability at a batch and asks whether one rule explains every row. The repositioning was navigation order, hierarchy, and copy — which door the site opens first, and what you read when you arrive. The resulting workflow is the honest one: the AI does the work, Verify checks it, Infer replaces the step that broke. A deterministic engine will not out-generate a language model. It can out-verify one, permanently — "no LLM" is not a limitation here, it is the qualification for the job. You do not audit a system with a second instance of the same system.

The rule is the product, not the output

Most transformation tools optimize for producing output as fast as possible. Paste data in, get data out. The output is the product. Latentmachine inverts this. The primary result is the symbolic rule — a list of named operations, each one inspectable, each one traceable to the examples that support it.

The alternative was obvious: just show the transformed output and let the user copy it. That is what most users expect. But the output is only useful once. The rule is useful every time the source format sends new data. And the rule is the only thing the user can actually verify — you cannot look at a transformed JSON object and know whether field X was copied correctly without seeing the rule that produced it.

This decision shaped everything downstream. The diagnosis panel exists because the rule is visible and needs annotation. The confidence system exists because the rule needs a trust label. The export system produces code that embeds the rule as a comment header, so the rule survives even after it leaves the tool. If the output were the product, none of those features would exist.

Confidence is a checklist, not a number

The engine's confidence label is not a percentage. It is a named state — proven, supported, needs-proof, unsafe, blocked — derived from a checklist of five evidence checks: exact fit, no blocking guardrails, all paths explained, no meaningful ambiguity, and at least two examples.

The alternative was a numeric score between 0 and 1. It would be easier to compute, easier to display, and easier for users to interpret at a glance. The problem is that a number like "0.78" communicates false precision. It invites the question "is 0.78 high enough?" which has no meaningful answer because the number is not a probability. It is an evidence assessment.

A checklist makes the evidence legible. The user can see which checks passed and which did not. "4 of 5 checks passed, the missing check is: at least two examples" is actionable. "Confidence: 0.78" is not. The tradeoff is that a checklist requires more screen space and more cognitive load. It is a harder interface to glance at. I chose it anyway because glanceability is the wrong optimization for a trust decision.

Refusal as a feature

When the engine cannot distinguish between two plausible rules, it does not pick one. It reports both, classifies the ambiguity, and suggests what kind of example would resolve it. When a new input contains a value the engine has never seen, it does not guess the output. It flags the field as unresolved.

The alternative — and the default in virtually every other tool — is to always produce output. Pick the simpler rule. Use a default for unseen values. Keep the user moving. This is a reasonable design choice for tools where the cost of a wrong output is low. For data transformations that feed into databases, pipelines, and automated workflows, the cost of a wrong output is not low. It is a silent corruption that compounds.

The refusal system is enforced by a product contract that is tested by the benchmark suite. The engine cannot ship a build where it silently guesses on ambiguous input — the adversarial benchmarks catch it. If someone removes or weakens an ambiguity-detection test, the build fails before it reaches the deploy step. This is not a policy decision. It is an architectural constraint.

The diagnosis as a conversation

When the engine reports a problem, it also reports a next step. "Ambiguous" comes with "add an example where $.status has a different value." "Contradictory" comes with the specific examples that disagree. "Insufficient" comes with "add a second example with different values." The diagnosis is not a wall of errors. It is one turn in a back-and-forth.

This changes the interaction pattern. Instead of paste-and-pray, the user teaches the engine iteratively. Two examples, read the diagnosis, add a third if needed. The engine converges within two or three rounds on most real-world payloads. The design decision here was to invest in the suggestion system (which requires the engine to reason about what it does not know) rather than in a more forgiving inference that guesses more aggressively. Both paths lead to faster results. Only one leads to trustworthy results.

Why vanilla JS with no framework

The UI is 1,857 lines of vanilla JavaScript with string-template rendering and manual DOM event delegation. No React, no Vue, no build-time framework. The rendering layer is extracted into a separate module (render-helpers.js, 1,596 lines) that contains pure functions producing HTML strings.

The alternative was React or Preact. Either would have given me component composition, reactive state, and virtual DOM diffing. The tradeoff is a build step, a runtime dependency, and a framework idiom that would obscure the application logic behind lifecycle hooks and component boundaries. For a tool where the UI is thin and the engine is thick, the framework would be solving a problem I do not have (complex component state) while adding a problem I do not want (a dependency I cannot audit in the browser).

The cost is real: no diffing means full re-renders on every state change, which means more manual work in the event handling layer. The benefit is also real: the entire application loads in one request, runs without compilation, and can be read in the browser's source view. For a tool whose trust story depends on transparency, that matters.

The cost model

The engine uses a Minimum Description Length (MDL) inspired cost model to choose between competing rules. Each operation type has a fixed cost prior — a simple field copy is cheap (1.05), a learned value lookup is expensive (2.25). When multiple rules fit the examples, the engine picks the cheapest one, on the principle that the simplest explanation is the most likely to generalize.

The cost priors are hand-tuned, not learned. This was deliberate. A learned cost model would optimize for the training distribution, which means it would encode assumptions about what "typical" data looks like. A hand-tuned model encodes structural complexity: a copy is simpler than a template, which is simpler than a lookup. This ordering is true regardless of the data distribution.

Evidence adjustments are allowed to break ties within a cost band but cannot override the structural ordering. This is enforced by a cap (COST_ADJUSTMENT_CAP) that bounds the total adjustment to less than the gap between adjacent cost priors. The engine can prefer one copy over another based on path similarity, but it cannot make a lookup look simpler than a copy. The constraint prevents the evidence from rewriting the architecture's assumptions.

One engine, four tools

Verify, Infer, Regex, and jq all share the same intelligence layer. The engine is cleanly separated from the UI: pure functions that take examples and return rule artifacts, with no DOM coupling. Each tool composes the engine differently — Verify uses it to audit a batch, Infer teaches it with examples, Regex uses a different synthesis path for patterns, and jq turns selected or example-shaped JSON into verified query expressions — but the contracts, the benchmark system, and the cost model are shared.

The architectural benefit is consistency: every tool inherits the honesty model and the evidence system without reimplementing them. The portfolio benefit is different: having four tools that share one engine demonstrates that the architecture generalizes. It is easy to build one tool well. It is harder to build a platform that supports multiple tools without the abstraction leaking. The tool contract system — a shared lifecycle with explicit states, benchmark assertions, and versioned schemas — is what makes this work.

The engine leaves the browser

The website is the demonstration. The engine is the product. Because the intelligence layer is pure functions with zero dependencies and no DOM coupling, extraction cost structure, not rewriting: the same artifact now exists as a standalone npm package with its own test suite and publish pipeline, as a CLI that exits 0 or 1 so a CI pipeline can gate on batch consistency, and as an MCP server that lets AI agents call the engine as tools.

The MCP server forced the most representative decision in this section. The obvious implementation imports @modelcontextprotocol/sdk and zod — two dependencies in a project that has none. But the protocol underneath is JSON-RPC over HTTP POST, and for stateless, synchronous tools it fits comfortably in one serverless file. So the remote server at /api/mcp implements the protocol inline: no SDK, no schema library, no session state — around 400 lines you can read top to bottom, sitting next to an engine you can also read top to bottom.

MCP matters more than the other surfaces because verification is worth the most inside the loop where the AI works. A website requires someone to remember to check. An MCP tool lets the agent check itself: the model transforms a batch, calls Verify on its own output, and gets a deterministic verdict before the result moves on. The rejected alternative was a hosted API with keys and accounts — which would have meant a database, sessions, and a privacy story with asterisks. This way the architecture keeps the shape it has always had: stateless, inspectable, nothing retained.

Over 800 checks gate every deploy

The engine cannot ship without passing over 800 checks covering unit tests, composition tests, adversarial tests (ambiguity between competing rules), regression tests, golden tests (with assertion checks for method, confidence, explanation text, and duration), cross-format translation tests, tool acceptances, import checks, and deploy preflight. The build command runs the full suite before producing output. A failing check blocks the deploy.

This is not test coverage for its own sake. Each benchmark encodes a design decision. The adversarial benchmarks verify that the cost model picks the right rule when two operations compete. The golden benchmarks verify that the explanation text matches the operation. The regression benchmarks encode bugs that were found and fixed. The suite is the executable specification of the product contract. If I change the cost model and a benchmark fails, the failure tells me which product decision I just broke.

What I would do differently

The rendering layer should have been separated from app.js earlier. The monolith reached 3,287 lines before I extracted render-helpers.js, and some of that extraction was mechanical work that a cleaner initial separation would have avoided. The lesson is that "no framework" does not mean "no architecture." Vanilla JS needs the same separation of concerns as any framework project — it just does not enforce it.

Verify should have existed from day one. The first section explains why it leads now; the honest admission is that I only saw it after building everything else. The engine's most defensible ability — inferring the majority rule and flagging the rows that break it — was sitting inside Infer the whole time, unnamed. Verify is what makes the engine's value tangible for people who will never build a transformation but constantly need to audit one. If I were starting over, I would build the auditor before the builder.

Closing

There is a pattern under all of these decisions, and it is worth naming once. Every claim this project makes is designed to be checkable rather than believable. "Nothing leaves your browser" is not a policy — it is visible in the Network tab. "The engine refuses to guess" is not a value statement — adversarial benchmarks fail the build the moment it starts guessing. The product contract has a section titled Refusals, and the benchmark suite is its enforcement. The confidence label is a checklist you can audit, the rule is a program you can read, and the source ships unminified so that reading is actually possible. A verification tool that asked to be taken on faith would be a contradiction in terms.

That stance is also why the project exists. I design and build with AI every day, and this is the piece of software that refuses to take AI's word for it. The engine, the tools, the articles, and this page are one-person work — with AI agents doing the implementation, and over 800 deterministic checks deciding whether the result ships. If you want to see the decisions in action, Verify is the entry point. The Latentlog covers the technical reasoning behind the engine, the honesty model, and the building process in more detail.