The last post ended on the realization that the bugs weren't the problem — the pattern of them was. This is the part where I tried to do something about it, on both ends of the app at once, and learned what that costs.
The front end: shortening the path
The first thing I went after was the scan flow itself. If the OCR couldn't be perfect, maybe I could shorten the distance between user opens the app and user sees an answer so much that the imperfection mattered less.
So I flipped the scan from barcode-first to tag-first, because the tag is what the user is looking at anyway. Then I rebuilt the UI around getting to the deal strata — is this a good price — as fast as possible. Then I tried deal-first, price-confirm-after, which is still the plan, even if I couldn't quite get the confidence high enough to ship it the way I wanted. Then Android added a native barcode detector, and it was beautiful, and I wired it in. The barcode side became wonderful. The OCR side stayed stubbornly seventy-something.
Somewhere in the middle of all this I started actually understanding what my tools were doing under the hood — not just calling them and trusting the output. That's where the idea of vision_blocks came from, which I'll come back to in the next post.
The back end: the admin grind, then the LLMs
While I was rearranging the front, the back end was its own mountain. Bad tags came in. Misidentified products came in. The only way to fix any of it, at first, was for me — sitting at a laptop — to fix it by hand.
So I built tools to make the fixing faster. Golden tags. A product identification cleanup workflow. Each of those tools made me individually faster at correcting one wrong thing. None of them made the amount of wrong things smaller.
Then I started using LLMs. First to generate parser rules — patterns the parser could learn from rather than me hand-coding them store by store. Then to clean up product identities at scale. Gemma 4 dropped around then, and the hosting costs for a model that good came down enough that I could actually run it on every scan if I wanted to. The option space cracked open. I poured more into it.
The juggling
You should know what this actually looked like day to day. I'd start the morning planning to fix one specific OCR issue. Twenty minutes in I'd notice a stuck admin queue. I'd dive into that, get halfway through, realize the queue was stuck because of a parser rule that was misfiring, switch over to fixing the rule, get distracted by an LLM prompt I wanted to tune, lose an hour on that, come back to the OCR issue at 3pm having forgotten which file I was in.
It wasn't a strategy. It was an unmedicated ADHD person at a craft fair. Every booth interesting, every booth half-finished, the keys to the car somewhere in the bag.
The work was real and most of it shipped. But the pattern from the last post — fix one thing, two more wake up — was happening on both ends at once, and I was the only person noticing.
The bills
The other thing happening around then, quietly, was that the cost of running all of this had stopped being free.
The Anthropic credits I'd been spending to power the LLM tooling were a real line on a real card. The DigitalOcean invoices stopped being rounding errors. None of it was catastrophic. It was more than a few meals, less than rent. But it was real money, going out, every month, on a project that had not made one dollar.
I started this thinking the technology would be the hard part and the cost would take care of itself. The cost was not taking care of itself.
And the quiet around it
The part that took the longest to admit is that nobody around me seemed to care.
I'd tell a friend what I was building. They'd nod. They'd say something polite. And then they'd change the subject, because the underlying thing — grocery prices, grocery stores, the slow squeeze of being a regular person buying food — was something most of them had quietly given up on. The shrug was the same shrug every time. Yeah, it's bad. What are you going to do.
That was harder to sit with than the bills. The bills you can model. The resignation you can't.
Where this leaves me
So: working on both ends, juggling badly, burning real money, building something that the people I'd hoped would be excited about it had mostly already made peace with.
The thing that gave next wasn't the part I expected.
— Elmer