This post introduces a new approach to creating LLM-free support bots. In addition to providing a great user experience, these bots are reliable, speedy, hallucination free, and overall cheaper, simpler, and safer to operate than bots based on LLMs. They are immune to prompt-injection attacks and the possibility of leaking personal information to third-party AI model providers. There are no prompts or prompt-tuning and these bots do not ship any data off to an AI model. Many products may be better served by a bot built with these techniques.

Before explaining how it all works, let’s look at a quick demo, this one a support bot for an online bookstore. A few interactions are shown here: checking order status, changing the shipping address on a recent order, checking gift card balance, looking for a nearby store, and initiating a return.

Interactions are kicked off with natural language, with rich autocomplete, but from there the bot is deterministic, with flows defined using regular code. The approach works well. Have a look:

This is a transactional support bot which empowers the user to directly take action within the conversation, using contextually appropriate UI controls. It is not a purely informational textual chatbot providing possibly hallucinated pointers to documentation or confusing instructions about how to accomplish things elsewhere in the app. (“Go to ‘Account’ > ‘Orders’, then in the left sidebar, click Inscrutable Icon 72, select ‘Associated Shipping Address’ from the popup menu…”)

Here’s a side-by-side comparison of our approach vs an LLM-based bot:

The AI-free bot is faster, more discoverable, and any actions its vocabulary are executed with 100% reliability, right then and there, in the chat UI. The LLM-based chatbot is slower and ends the interactions with confusing and possibly hallucinated or out of date instructions that the user must follow to carry their request out elsewhere in the application.

Computer, if you understand what the user is trying to do, don’t give them instructions, just do the thing.

How does it work?

The are two ideas at work here. First, for parsing natural language input to structured commands, Natural Language Disambiguators (NLDs) are used, an LLM-free natural language processing method that is robust to differences in phrasing. It runs in milliseconds on the front-end within an autocompleted text input control.¹ Here’s a closer look at this UI control, and see this post to learn more about NLDs:

A free-form text box with no feedback until hitting the submit button (followed by a long “thinking” pause) just isn’t great. It’s really too open-ended, encouraging users to think of the system as a general intelligence capable of responding to all inputs, when the system is often only capable of a very narrow class of computations.

After the NLD input parses the natural language request to one of the structured commands supported by the bot, the remainder of the flow is deterministic, described with regular code in a high-level domain-specific language. The DSL is based on computational conversations, a programming model for computations that can pause for human input with rich UI controls, then resume when given responses. This simple but powerful idea has many applications and it’s useful for creating experiences that feel conversational yet are still deterministic and reliable.

The computational conversations framework automatically handles persistence and resumption of flows, so the specification of these bots stays nice and high level, focused entirely on the business logic. Often with this approach, there is less code than what would be needed to prompt an LLM to (unreliably) implement the same behavior.²

That’s it for a high level overview of how it’s done (but see the computational conversations post and the NLDs post for details). Now let’s explore the strengths and limitations of this approach.

Strengths and limitations

An LLM-free support bot based on computational chat doesn’t purport to be capable of more than it is. Anything it can do, it does with 100% reliability, with a contextually appropriate UI assembled on the fly. Anything it can’t do, it escalates to a human support agent or doesn’t attempt. The bot never confabulates answers that seem plausible but incorrect, nor does it try things it is incapable of. It is a humble tool that users can trust.

Is it a limitation that this style of bot encodes all its automated capabilities with regular code? Most of the time, no, not really. To understand why, let’s look at a few different kinds of support requests.

Many support requests are obviously automatable tasks like “change the shipping address on my order” or “check my gift card balance” or “track my order”.³ They end up as support requests either because the application doesn’t surface the functionality, or it does but it’s buried somewhere undiscoverable. Because these cases don’t require human judgement, spending the time to convert these to regular code that can be triggered via natural language input in the bot will pay for itself many times over. Users are generally happier to accomplish these tasks in a few taps rather than waiting to talk to a human. Everybody wins.

Even if a type of support case sometimes has situations where human judgement is warranted (“should we make an exception to our return policy for this loyal customer, because of these extenuating circumstances?”), that’s perfectly fine; the bot can handle the easy cases automatically and escalate to a human for the exceptional ones.

Something working against LLMs as a substitute for any sort of “escalate to humans” flow is their lack of reliability and tendency to confabulate, especially for out-of-the-ordinary situations. Most users are only contacting support because they couldn’t figure out how to accomplish what they wanted. There’s a decent chance they’re a bit frustrated or annoyed. Even if a well-prompted LLM could do the right thing 60% of the time, that’s 40% of the time where users get a terrible support experience that destroys their trust in the bot and downgrades the company brand in their eyes.

But even getting it right 60% of the time is often out of reach. For truly exceptional circumstances, the organization doesn’t have enough foreknowledge to successfully prompt an LLM to respond appropriately. For example, suppose the company has just shipped a new feature and a user is confused about how it works or is encountering a bug. An automated system will always struggle to fully handle such a case. Likely the humans involved in producing the feature or who understand the bug will need to understand what the user is seeing and make a judgement about how to respond.

Even if an LLM were capable of responding to the situation given the right prompt and sufficient context, the people involved don’t even know enough to provide such a prompt in the first place, nor is there sufficient data to tune the agent to act reliably. It is only after the situation has occurred multiple times and is understood that such a prompt could be attempted and refined. But again, depending on the issue, the effort spent on prompting an LLM may be better spent on just writing regular code to handle that class of support case reliably.

When AI can add value

AI can add value for parts of a business process that are well-defined and well-understood but for whatever reason aren’t amenable to regular code. For instance, suppose as part of an order return or refund flow, if the item is over $500 and the customer says it’s damaged, the customer is asked to upload a picture of the damage. Yes, a human could look at the picture to verify the damage, but that image processing could also be done with (say) a multi-modal AI model. Even if not fully reliable on this task, that may be perfectly fine.

Another place where LLMs specifically can add value is converting natural language requests to more complicated structured queries or commands. Going back to our online bookstore example, consider the request “how much money have I spent on sci-fi novels this year?” This is more complicated than “order status” or “change shipping address on my order” in that we want to produce not just a single command, but a composite expression in some (safe) underlying query DSL. LLMs are good at this sort of thing—it’s adjacent to LLM usage in coding assistants, where they’ve been successful. But for a support bot, this is a somewhat niche usage, and it takes work to make reliable.

More usefully, LLMs can search for and dynamically summarize, explain, or answer questions about official documentation. Even when they make mistakes, the alternative might be something like keyword search with links to documentation pages, where the information the user cares about is buried or explained in a confusing or less contextually-appropriate way.

That said, support bots are not primarily about documentation search, and users don’t typically enjoy reading pages of documentation if they don’t have to. A good support bot should be capable of actually doing the thing, not just providing pointers to long-form documentation that says how the thing can be accomplished elsewhere.

Conclusions

Support is a crucial part of the customer experience. It is a place where businesses can build up user trust and goodwill, or erode it. Perhaps moreso than other domains where AI seems applicable, reliability matters and the cost of mistakes is high.

Often in computing, a combination of more specialized and straightforward methods can work far better than the hyped AI techniques of the day. All it takes is a clear understanding of the problem, a willingness to take a first-principles approach, and good old fashioned engineering. This post introduced a method for creating highly reliable and low-latency support bots that don’t use AI at all. Free-form textual input is done with Natural Language Disambiguators, providing robust parsing of structured commands from natural language input, with real-time autocomplete support for discoverability. And once commands are parsed, deterministic flows using computational conversations implement the actual business processes. Persistence and resumption of interactions is handled automatically by the framework, keeping the bot code delightfully straightforward. The overall UX is fast, reliable, easily understood, and builds trust, far better than hallucinatory chatbots which cannot even directly handle requests in-conversation.

We are currently living in a distorted world where hundreds of billions of dollars are being poured into doing something, anything, with LLMs and other forms of AI. There is fear at being left behind, and pressure on company leadership to ship AI features, often with minimal consideration of whether those features make financial or technical sense or improve the product. All this money and pressure has led to some very silly outcomes. In the realm of support bots, we regularly see companies implementing business logic unreliably with text prompts passed to an LLM, even when this logic is easily expressible as regular programs that run with 100% reliability at a fraction of the cost and latency. None of this makes sense. None of this benefits users. And it’s time to consider more reasonable alternatives.

If you’re working in this area or like the vision here and want one of these AI-free bots for your app, we’d love to hear from you. Send us an email at acid-burn example dotcom .