Before you add AI, name the job.
Start with the outcome someone needs, then decide whether a rule, a model, or a combination of both is the right tool.
“Add an AI assistant” sounds like a product decision. Usually it is the start of a question.
What should the assistant help someone finish?
Suppose the idea is to help people organize a collection of saved articles. There are several different jobs hiding inside that sentence. Finding an exact title, suggesting a topic, summarizing a long piece, and choosing what to read next are not interchangeable problems.
Putting a conversation box in front of all four does not make the distinction disappear.
Describe the useful result
Start with a sentence someone could recognize from their own day: I saved something about this subject, and I want to find it without opening twenty tabs.
Now there is a result to design for. A title search might handle it. A filter might handle it. Searching by meaning could help when the person remembers the idea but not the wording.
The decision becomes more interesting once the task is specific. You can ask what information is available, where interpretation is needed, and how someone will know the result is right.
A tool that returns the right article with a clear link may be more useful than an assistant that writes a confident paragraph about the wrong one.
Separate interpretation from execution
For this hypothetical reading tool, a model might propose a few labels for an article. Ordinary application code can check whether those labels already exist, save the accepted choice, and preserve the original text.
The person should be able to see and correct the suggestion before it becomes part of their library. The interpretation is uncertain; the record of what was accepted should not be.
Anthropic’s engineering guide Building effective agents distinguishes fixed workflows from systems where a model directs the next steps. It also recommends beginning with the simplest approach that meets the need. That is a useful architectural distinction, not a reason to choose a particular vendor or framework.
A fixed workflow with one well-placed model call can still be a thoughtful AI feature.
Compare it with the simpler version
Build a small evaluation set from examples you have permission to use. Include an obvious match, a vague request, an ambiguous case, and a question the collection cannot answer.
Decide what counts as success before comparing implementations. For article search, a useful check might be whether the intended source appears and whether the interface admits when there is no good match. A fluent answer is not a substitute for either.
Look at the whole task as well. How much checking does the result require? Does the feature preserve the source? What happens when the model is unavailable? Would a straightforward filter have been quicker to understand?
Keep examples that expose mistakes. They give you something concrete to revisit when a prompt, model, or retrieval method changes.
Make the result the main attraction
A person should not need to admire the technology to benefit from the feature. They should be able to find the article, accept or reject a label, and continue with what they were doing.
That leaves plenty of room for ambitious AI work. It simply gives the ambition a destination.
And when the feature starts taking actions rather than making suggestions, the next design question is which permissions it actually needs.