Hidden component for CSS inclusion
project coco 03: post mortem
what has happened
it has been a couple weeks since my last update on Coco. this is to serve more as a post mortem on the project.
the infrastructure has had numerous changes. in each iteration, we realized its flaws, and completely shifted. the high level journey looks like this: openai agents sdk, Claude Code agent, leveraging skills, and using sprites.
in terms of tangible results, we got Coco to a point where it was very effective at researching manufacturers that aligned with the brand identity and goal for the product.
Coco’s manufacturer research.
we even got it to autonomously set up meetings with potential vendors with the ceo of the brand we were working with. we had some fun and even set up elevenlabs to call her.
Coco calling the ceo.
we were also impressed with its ability to negotiate down to get it into our specified target range.
Coco negotiating.
this is about as far as we took it. while Coco showed real promise, we decided this wasn’t the right direction for us to continue pursuing.
what we learned about agent harnesses
the first version of Coco was a long running agent built from scratch. we used the openai agents SDK, pointed it to claude, and wrote all of the necessary function tool code the agent needed to run the production operation smoothly.
the more we built Coco, the more we had to refine Coco’s behaviour by tweaking the system prompt, or improving the function tools we made. it became apparent that we had made a suboptimal version of Claude Code.
Coco-v2 is built on top of Claude Code, since so much of the functionality we need is already natively built in. most notably though, it also has the skill system where you could easily drop in the SKILL.md file and a script and the agent would know how to use it.
we also realized we were building a very similar product to Clawdbot (or now Openclaw), and so we did some research on Openclaw’s infrastructure
sessions and context
what actually is context? we decided it fits into the following formula:
when we talk about sessions, we’re really talking about the chat history. the question is: what is the ideal history the agent needs to produce the right output at any given moment?
for coding agents, the challenge is the cold start. whenever you start a new Claude Code session, you need to quickly orient the agent in the codebase with the use of CLAUDE.md files and the like. Coco’s problem was different because it was initialized once, and ran continuously throughout the lifetime of the production cycle. instead of a cold start problem, the context just keeps on growing.
in the first iteration of Coco, our approach was to keep the past few messages verbatim and compact the rest of the history with a separate summarization call. in Coco-v2, we handed context management entirely to Claude Code and instead focused on persistent memory: Coco’s system prompt instructed it to actively maintain its own CLAUDE.md file. it updated the file whenever it learned something new about the business, made a decision, or discovered something that didn’t work. this meant that even when a session was lost or compacted, the important information survived because Coco had already written it down itself.
there is no single right way to architect this. as long as the relevant information reaches the model, it doesn’t particularly matter whether it comes from a database, a filesystem, or a markdown file the agent wrote for itself.
agent runtime
the other big question was: what capabilities should be given to the agent and how?
we found it helpful to think about this in two tiers. function tools are the hands and feet of the model: they are used almost every turn, so they should always be in context. for Coco, this was very similar to the core tools that Claude Code had available: file i/o, bash, web search, etc. on the other hand, skills are like a car that needs to be driven every once in a while to pick up groceries on the weekend. it’s not integral to the agent for everyday use, but it can definitely come in handy.
our initial set of communication methods (email, slack) started out as a custom set of function tools, but later became a standalone CLI with a SKILL.md file that Coco could pick up on its own. same with the agent-browser skill.
this delineation matters because it is related to context preparation and scaling. naively, we could expose a complete tool set to the agent, but flooding the context like that is not scalable. this burns through context and wastes the model’s attention. the goal is to minimize the number of native tools to just what the agent needs every turn, and let it discover the rest through skills when it needs it.
we want to minimize the amount of wasted intelligence to maximize the scalability of the agent.
thoughts on agents
originally, we set out to build a niche agent for fashion manufacturing. in the end, i ended up building a general-purpose agent (Claude Code wrapper) with domain-specific tools and relevant context bolted on. the niche part of the model was actually in the CLAUDE.md, the skills, and the system prompt.
there is a question i keep finding myself going back to:
if your “specialized agent” is just a frontier model paired with the right tooling, then what is the defensibility of a “vertical AI agent”?
i experienced it firsthand where i tried to make my own custom agent, but ended up replacing it with Claude Code because it was just much better out of the box.
the defensibility for these generalized agent wrappers is not going to be in the technology itself. it’s in the things that compound over time like distribution, network effects, proprietary data, and user trust. the harness is commoditized but what you plug into it isn’t.
with that being said, this experiment revealed to me the importance of long running agents. METR’s research shows that the length of tasks AI can complete is doubling every 7 months. Anthropic built a C compiler by running agents in a loop that never stops. Cursor ran a thousand agents for a week. Project Coco was a small version of this same trend. these experiments are valuable because there is an inevitable future where agents are running the operations of businesses, research, and perhaps even governments.
whats next
this is the end of Project Coco. the experiment validated that an agent can meaningfully run parts of a business operation such as sourcing, outreach, negotiation.
follow me on X for more updates. any feedback is appreciated!
shoutout to Robin Cunningham, Jess Yin, and Rubaina Singla.