Towards a Framework for AI Agent Design, Part 2

Written By Charles Brewer

In Part 1 of this series on a Framework for AI Agent Design, we looked at a life cycle model for your agentic projects. Looking at the design process chronologically or sequentially can indeed help you see the big picture so you can make the big moves.

But you are right to ask: "What exactly is IN an AI Agent?"

Successful AI agents are composed of tangible and intangible components

Now let's look at a different aspect or side to this framework. It has a different focus: Consider the functional components of AI agent design.

When we shop at Ikea, we bring back boxes filled with a huge number of pieces that need to be gathered, verified, and then put together. In the kitchen, we might follow a recipe. It gives a list of required ingredients and how they should be prepped and then cooked.

AI agent design deals with screens and content. But most of what goes into our process are the intangible but vital components of a software system. These components are processes, resources, factors and configurations, and strategies.

The AI agent design process is a series of lenses for examining the work

Using the seven lenses below, we make a comprehensive inspection of the big picture environment for the AI agent. The decisions we make for each form our AI product and design requirements. Deciding on each of the following AI components is Part 2 of our framework.

Users

The whole point of AI agents is for people to use them; agents do work for humans, under their review. So start with understanding the people who need or want the AI agent solution. Expand this to all the key stakeholders: Workers, users, managers, executives, other internal stakeholders, and the business's customers and audiences. Questions to answer include:

  • What is the work they do? What is their concept of their job or function?

  • What's currently working well? Not well?

  • What constitute good versus bad work outcomes?

  • What are their hopes for AI implementation? Their fears?

  • What's the ideal solution in their view? Why do they think that?

  • Pro tip: It's not too early to start thinking of AI agents as junior workers alongside human workers. What is the best user persona of the AI Agent itself?

Data

AI agents and the large language models that power them excel at finding, ingesting, analyzing, and summarizing large amounts of information. Most companies are sitting on huge quantities of data: document servers, email, slack channels, web and app analytics, CRM systems, sales and customer support systems. It may not be available or structured effectively, but the agents may need to access it. An audit of the data is called for:

  • What data exists? What data is missing?

  • How good is it: How timely? Is it structured for AI understanding?

  • How is data maintained? Deleted?

  • How do AI agents need to access it, transform it, store it?

Context

This is the big one. Context starts at a very general, global level and includes:

  • Background information on the business

  • The industry

  • Competitors

  • Economic and financial trends

  • This sort of context might show up as "system prompts" for AI Agents that all employees and agents start with.

From there, context becomes increasingly specific, granular, and local. Each level has its own context, which needs to be documented as references or prompts:

  • Department context

  • Job context

  • Task context

  • Goals and metrics exist at all of these levels as well. The strategy for context is for the company or an individual (or even an AI agent) to be able to generate repeatedly high quality, accurate, effective context and prompting for the AI agent.

Workflows

Workflows describe the actual work to be done, and how it is sequenced. Finding and documenting workflows is the main focus of the discovery or strategy phase of an agent project. There are multiple frameworks that offer similar methods for doing this: developing product sense, user research, mental models, listening tours, contextual inquiry. Aspects to be documented include:

  • Starting point of the workflow

  • Prior states and triggers

  • Inputs and outputs

  • Tasks, subtasks, sequences, and transformations

  • Errors & null conditions

  • Integrations with other systems

  • How many AI agents do you need to complete the workflow?

  • If multiple agents, what will their roles be and what are the sub-agents?

UX Design

AI agents may become autonomous, but humans and agents must still interact. If the agent UI is confusing, incomplete or hard to use, the expected performance gains will not be achieved. What is the optimal intended experience of using the AI Agent and completing the workflow successfully? While interaction modalities are still somewhat limited with AI Agents, this is evolving rapidly beyond the text query box. What can you do to raise the Agent experience beyond crude completion? Can you make it easy to show, sell, learn, use, maintain?

AI Models and Configurations

Choices need to be made at the AI model level regarding which model(s) to use. Also what associated internal and external computing services to use. A strategy for the AI infrastructure would develop answers to questions such as:

  • What AI language model or models should be used?

  • Which are allowed? Which are forbidden?

  • What are the capabilities and costs of the selected model?

  • What happens if a better model is released, or compute fees go up?

  • What APIs and tools does the AI agent need to connect to?

  • Do additional ones need approval, purchase, and integration?

  • Who or what will use and store these inputs and outputs?

  • What is the plan for when the underlying LLM must be changed?

  • Pro tip: What are the triggers that elevate the agent's work for human review? What are those scenarios?

Cybersecurity and Usage Policies

Agents are AI and software. Most of this software exists as cloud services. All of these can be hacked, hijacked, subjected to prompt insertion attacks, and much more. The cybersecurity concerns of AI Agents are just emerging, but mission critical data, systems, and workflows must be totally secure. The security of proprietary and confidential data must be maintained for competitive and legal reasons. Policies must be developed and protections put in place to constrain the access rights and scope of impact of the LLMs, the AI agents, and the human workers.

These are the seven functional components of a proposed framework for AI agent design. They add to the six life cycle stages outlined in Part 1.

Follow me for Part 3, where we look at how these two sides of AI agent design might work together.

What are your thoughts on how we can approach AI Agent design for maximum effectiveness and success?

Comments and criticisms welcome.

https://get.mindstudio.ai/t3p8e2ypz6tp

Charles Brewer

Making things.

Previous

Towards a Framework for AI Agent Design, Part 3
Next

Towards a Framework for AI Agent Design, Part 1