[Project] Building an Agentic Chatbot

Agentic Chatbot is a simple app I’ve built over the last month. In simple terms, it is a chatbot that allows you to build custom workflows – combinations of LLMs working in a structured, user-defined way – to produce better outputs than by interacting directly with a single LLM instance.

The app’s two core concepts are:

1. Agents: An instance of an LLM with a specific system instruction

2. Workflows: An orchestration of Agents, defining how they should work together

Say you want to write some code, so you paste into ChatGPT the codebase or relevant files that you’re working with, and you make a request. You either naively prompt to “do the following”, or you can try to build a more structured and specific prompt. The more structured version might include guidance to make a plan before writing code, to make certain trade-offs (e.g., simplicity over efficiency for a smaller app), or to check its work before finishing. In my experience, the latter works much better. However LLMs still (for now) struggle with adhering to many instructions, especially when longer contexts are involved. How would you do this with Agentic Chatbot? One simple option would be to:

• Create Agents for each major component involved in the request

  • Code Planner: instructed to build a detailed plan of how to implement the request/s in the codebase

  • Code Executor: instructed to write production code

  • Code Reviewer: instructed to review code changes against a plan and the original user request

• Build a workflow using the three agents above

  • First, feed the user input to Code Planner

  • Second, feed the user input and the output from Code Planner to Code Executor to implement the plan

  • Third, feed the user input and the output from Code Planner and the output from Code Executor to Code Reviewer to review all changes

In my experience, Workflows tend to produce better results through some combination of more compute per request and specialization.

In addition to being able to build Workflows, you get some other nice benefits from building your own chatbot. You can switch models within the same conversation, or switch from a single model (e.g., GPT-4o) to a Workflow (e.g., the code workflow described above) on a message-by-message basis. This allows you to somewhat tailor the compute used per request.

Building the app was reasonably challenging given I haven’t built an end-to-end piece of software before. It took a lot of time and wouldn’t have been possible (at least not in the same time frame) without LLMs.

Currently, the app is live in a very preliminary state (expect bugs!). It supports OpenAI, Anthropic, Gemini, Llama 3, and some Mistral models. If you do use and it you find any bugs, or want it to support something new, or have feedback, please email me.