A practical guide to building agents

Getting started with agent architecture

Ever wondered why some bots feel like talking to a brick wall while others actually get stuff done? It’s usually because the "smart" ones aren't just running a script—they’re using an agentic loop to actually think through a problem.

The biggest mistake people make is thinking an ai agent is just a fancy python script. It’s not. A script follows a path from A to B every single time, but an agent uses a reasoning engine to decide its own path based on what happens in real-time.

• Reasoning vs. Rules: In healthcare, a script might just send a generic appointment reminder. An agent can look at a patient's history, see they missed three labs, and decide to ask if they need help with transportation instead.
• Planning and Memory: For retail marketing, an agent doesn't just blast emails. It remembers that a customer looked at red shoes twice and "plans" to wait until they're on sale before sending a ping.
• Autonomous Adjustment: If an api fails in a finance workflow, a simple script breaks. An agent sees the error, tries a different data source, or logs the issue while moving to the next task.

You don't always need a massive platform to start. honestly, sometimes a custom python script is better if you're just doing one specific thing, but for scaling, you'll want something like LangChain or Semantic Kernel to handle the heavy lifting of connecting to different services.

A 2024 report by Capgemini found that 82% of companies plan to integrate ai agents within the next year, mostly to bridge the gap between their messy internal data and customer-facing apps. (AI agents in Organizations 2024 | Capgemini Switzerland)

Next, we’re gonna look at how you actually pick the brain for these agents—the llm selection—and why bigger isn't always better.

LLM Selection: Picking the right brain

Choosing an llm is where most people get stuck because they think they just need the "smartest" one. But picking a model is more like hiring an employee—you don't need a rocket scientist to file papers.

• GPT-4o / Claude 3.5 Sonnet: These are your heavy hitters. If your agent needs to do complex reasoning, like analyzing legal docs or writing code, you go here. Claude is especially good at following long instructions without getting confused.
• Llama 3 / Mistral: These are great if you want to run things on your own servers for privacy. Llama 3 is surprisingly fast and works great for "routing" tasks where the agent just needs to decide which tool to use next.
• Small Models (Haiku / Flash): Use these for the simple stuff. If an agent is just summarizing a chat or checking if a lead is "hot" or "cold," using a massive model is just throwing money away.

The trick is to use a "router" approach. You use a cheap model to figure out what the user wants, and only call the expensive gpt-4 brain when things get complicated. This keeps your api bills from exploding.

Next, we’re diving into how to actually design these workflows for marketing and sales.

Designing workflows for marketing and sales

Marketing and sales is usually where people first try to integrate ai into their business, but most just end up with a glorified auto-responder. If you want something that actually moves the needle, you gotta build workflows that treat your crm like a living thing, not a dusty filing cabinet.

Most marketing teams spend way too much time copy-pasting data between tabs. An agentic workflow can handle document processing by actually "reading" a proposal or a lead's linkedin profile and pulling out the bits that matter. instead of just saving a name, it can tag a lead as "high priority" because it saw they just got a new round of funding.

• Lead Enrichment: Agents can browse the web to find what a company actually does before a sales rep even picks up the phone. This stops those awkward "so what do you guys do?" openers.
• Dynamic Content: In retail, agents can generate personalized product descriptions based on a user's past browsing history. If someone always buys organic, the ai highlights the "eco-friendly" bits of a new item automatically.
• CRM Syncing: You can set up agents to listen for specific triggers—like a lead opening an email three times—and then automatically draft a follow-up in your CRM.

The tech is getting easier to use, but the integration layer is the hard part. You need a solid middleware—whether you build it in-house or work with specialists like technokeens—to make sure the ai talks to your existing stack without breaking everything.

A 2023 report by Gartner found that 63% of marketing leaders were already planning to invest in generative ai to help with efficiency and scale.

You gotta be careful though. Nobody wants to feel like they're being stalked by a robot. I've seen companies over-automate and lose that "human touch" that actually closes deals. Always keep a human in the loop for the final send, especially for high-ticket b2b sales where trust is everything.

Next, we’re looking at security and iam to make sure your agents don't go rogue.

Security and IAM for ai agents

If you give an ai agent the keys to your database without a plan, you're basically asking for a digital break-in. It’s one thing to have a chatbot hallucinate a poem, but it’s a whole different disaster when an agent accidentally deletes a customer’s billing history because it had "admin" rights it didn't need.

We need to stop treating agents like users and start treating them like employees with specific job descriptions. That means giving every agent its own service account. If you use one giant api key for everything, you'll never know which agent did what when things go sideways.

I'm a big fan of combining RBAC (Role-Based Access Control) with ABAC (Attribute-Based Access Control). RBAC gives the agent a general role, but ABAC is where the real control happens. You pass "attributes"—like the user's department or geographic location—directly into the api gateway or the prompt context. For example, a retail agent might have the "role" of a customer service rep, but the "attribute" check at the api level ensures it can only fetch data for customers in the UK.

Don't forget about token rotation. Hardcoding secrets in your agent's config is a rookie move. Use a secret manager so the agent fetches a fresh token every time it starts a new task.

You can't trust an agent just because it’s "internal." A Zero Trust approach means every single request the ai makes must be verified. This is huge for healthcare or finance where GDPR and SOC2 aren't just suggestions—they're the law.

• Audit Trails: Every decision an agent makes needs a log. Not just the output, but the "reasoning" it used to get there.
• Data Masking: Before data even hits the llm, scrub out the PII (Personally Identifiable Information).
• Human Gatekeepers: For high-risk actions, like moving money or changing medical records, always require a human to click "approve."

Security is usually an afterthought until something breaks. But in the agent world, a small bug in permissions can lead to a massive data leak.

Next, we’re going to look at scaling and infrastructure so your system can handle the load.

Orchestration and scaling in the cloud

So you’ve built a logic loop that actually works on your laptop, but now you need to let it loose in the real world without the system crashing. Scaling agents isn't just about throwing more ram at the problem—it’s about making sure your infra doesn't choke when ten thousand users start asking for "personalized travel itineraries" at the same time.

Most people start with a simple script, but for real scale, you gotta think about containerization. Wrapping your agent in Docker means it runs the same in dev as it does in production. If you’re using Kubernetes, you can spin up "worker" pods that handle specific tasks, like one pod just for computer vision and another for data extraction.

• Serverless vs. Dedicated: Serverless (like AWS Lambda) is great for sporadic tasks, like an agent that only wakes up when a new invoice hits an S3 bucket. But for chatty agents that need to maintain state, a dedicated container is usually cheaper and faster.
• Load Balancing: You aren't just balancing traffic; you're balancing llm tokens. If one region hits a rate limit, your orchestrator needs to failover to a different provider or region instantly so the user doesn't just see a spinning wheel.

You can't just "set it and forget it." I've seen agents get stuck in infinite loops because they couldn't figure out a weird edge case, burning through hundreds of dollars in tokens in minutes. You need circuit breakers—using tools like Resilience4j or custom logic in your orchestrator—that kill an agent's process if it takes more than, say, five "thoughts" to answer a simple question. For rate limiting, something like Redis is perfect for keeping track of how many requests an agent is making in real-time.

• Latency Tracking: In retail, if a product recommendation takes more than two seconds, the customer is gone. Track the "time to first token" to see where the bottleneck is—usually it's a slow api call, not the ai itself.
• Cost Guardrails: According to a 2024 report by cncf, cloud cost management is the top priority for 49% of enterprises. For agents, this means setting hard caps on how many tokens a single session can consume.

The biggest headache is just keeping the thing alive when an external api changes its schema. Use a monitoring tool like LangSmith or Arize to watch how the agent "reasons" in the wild. If it starts hallucinating more than usual, it’s probably time to tweak your prompt or switch models.

Next, we’re wrapping things up with how to keep your agents relevant long-term through lifecycle management and continuous testing.

The lifecycle and governance of agents

So you finally got your agent running in production—congrats, but the work isn't done. The truth is, ai agents are more like toddlers than software; they need constant watching or they’ll eventually wander off and do something weird.

Since llms are non-deterministic, your old "if X then Y" tests won't cut it here. You need evals—basically a set of "golden" answers that you compare against the agent's output to make sure it hasn't started hallucinating.

• A/B Testing: Don't just swap models overnight. Run the new version alongside the old one for 10% of your traffic to see if it actually handles healthcare patient queries or retail returns better.
• Human-in-the-loop: For high-stakes stuff like finance, have a human review a random 5% of agent logs. It’s the only way to catch the "vibe" shifts that automated tools miss.

Governance sounds like a corporate headache, but it’s really just about making sure your ai doesn't become a liability. According to IBM, the average cost of a data breach is hitting record highs, and a rogue agent with too many permissions is a massive target.

You gotta watch for algorithmic bias too. If you're using agents for hr automation, regularly check if the ai is favoring certain resumes based on weird patterns it picked up. The goal is to build a "firewall" of policies that keep the agent inside the lines without killing its ability to actually solve problems. Just keep it simple, keep it transparent, and don't let the tech outrun your oversight.