💡 Giving AI smaller questions

A project produces plenty of information that probably should not become permanent documentation. A passing suggestion. A build that is still running. A decision nobody has actually accepted yet.

It also produces things worth keeping: why a tool behaves a certain way, which approach was chosen, or a preference that would save another round of explanation.

I keep reusable notes in a hub repository that coordinates work across my projects. Deciding what belongs there involves a small judgment: does this observation deserve a record, and does the evidence support what the record would say?

That is one place I have started using Jev, TypeSafe’s model for focused judgments. Other experiments cover project names, finding shell commands, and choosing blog stories.

These are early tools and experiments. They share a division of work: code gathers evidence, Jev answers a bounded question, and the caller decides what happens next. A larger model handles work such as generating names or writing prose.

flowchart TD
  A[Tools gather evidence] --> B[Jev answers a focused question]
  B --> C[Code and reviewer inspect the answer]
  C --> D[Take the next step]
  C --> E[Review or gather more evidence]
  E --> A

01🎯Give the model something it can answer

TypeSafe describes Jev as a “System One” model: a model for focused judgments on supplied context. It returns typed answers and probabilities rather than explanations or prose.

“Typed” means the answer has a defined shape. If I supply four categories, the result can identify one of those categories. Program code can consume that directly without extracting a decision from a paragraph.

The supplied context is called state. In my integrations, Jev cannot follow a repository link or fetch missing information. The calling tool has to bring the relevant evidence.

There are three question types I use. Choice selects from known options. Score rates something along explicitly ordered levels. Noul, despite its unfamiliar name, serves a straightforward purpose: it gives a probability for a yes/no question.

Those distinctions matter. “Which candidate is best?” and “Is this candidate good enough?” are different questions. So are “How serious is this?” and “How likely is this to be serious?” A yes probability of 0.5 expresses uncertainty, not medium severity. The Noul documentation explains how to read that probability.

Independent questions can share the same state in a batch. They cannot see each other’s answers; application code combines the results.

A tidy output shape still cannot make the supplied facts true.

02🗒️Keeping a note requires more than a category

My knowledge-capture tool is a Rust command-line program. It accepts a candidate note, supporting evidence, and existing records, then asks Jev three questions: what kind of record is this, is it worth keeping, and does the evidence support it?

The categories include fact, decision, preference, and skip. Code combines the answers into capture, review, or skip.

Even “capture” does not write anything. The command returns a recommendation. The coding assistant still checks the actual evidence and, where relevant, whether I accepted the decision before writing a record. The public capture workflow describes that handoff.

The live integration has been exercised, but its thresholds remain provisional. In the initial sanitized examples, even the positive cases went to review. That illustrates cautious settings, not measured accuracy.

This is a useful boundary for the experiment: the model can help identify a note worth examining, while the evidence check remains a separate step.

03🏷️A favorite name can still be mediocre

Naming a project has a similar split.

My naming workflow uses Codex to generate candidates. A Rust tool checks relevant registries and APIs. Jev can then assess naming fit, whether the supplied evidence suggests a confusing product in the same category, and which candidate to recommend. The recommendation includes a “none” option.

Jev does not generate the names in this workflow. It also cannot turn a failed registry request into an availability result. Unknown stays unknown.

In a live comparison of three candidates, it preferred Taplume. Asking about absolute fit made that recommendation less flattering:

That is a useful result to keep intact. Winning a comparison does not mean clearing the bar. A shortlist can have a first place without having a name I should use.

Later web searches found historical listings for a similarly named Android app in third-party catalogs. Those were leads to investigate, without independently verified first-party evidence establishing a conflict. No name was accepted.

The naming tool helps me build a shortlist. It does not provide legal clearance.

04🐚Finding a command starts with a catalog

Quirl is my shell project. A shell interprets the commands you type into a terminal, including the flags that somehow become memorable only after you have looked them up again.

For a command-discovery experiment, I gave Jev descriptions of 62 known commands. It could choose a command or return NONE, CLARIFY, or COMPOSE when nothing fit, more information was needed, or the request called for multiple commands.

The experiment selected options. It did not assemble complete command lines or execute anything, and it is not a production integration.

One case asked for recursive directory sizes. The expected answer and Jev disagreed. Open the catalog evidence before deciding which one was wrong:

The catalog contained options that supported directory totals. The expected answer was corrected, with the original evidence retained alongside the correction. A mismatch had caught a problem in the test itself.

The benchmark report keeps the unresolved case visible alongside the validated results:

That is a reason to keep the evidence close to the recommendation. “The model disagreed” is the start of an investigation; the command descriptions are what let me resolve it.

05✍️Choosing a story does not finish the editorial work

The draft behind this article went through another experiment.

A new storytelling workflow gathers repository evidence and checks existing posts and drafts. It supplies Jev with summarized story angles and coverage, then asks for a recommendation and readiness assessments.

The first recommendation was useful: the benchmark disagreement survived into this article. It just needed a wider frame. Before asking a reader to care about Jev choosing a shell command, I needed to explain what Jev was doing in my projects in the first place.

06🔍Keep the question small enough to inspect

The use case I keep returning to is a small decision between gathering information and doing something with it. Is this note worth keeping? Does this name fit? Which command deserves a closer look? Which piece of work could become a story?

Those questions have something in common: I can put the evidence beside the answer and inspect the disagreement. The cutoff for trusting an answer still needs testing. A confidence value describes how concentrated the model’s answer is; it cannot tell me that the question was well chosen.

For the next integration, that is where I would start: one recurring judgment, the context it needs, and a way to leave it unresolved. A name can remain unchosen. A note can wait for review. And a perfectly reasonable story recommendation can turn into a different article.