Development

I think this project is exemplary for how I often start smaller greenfield projects. A heavy push in the first few days early on with main functionality working okay-ish, but then a longer trailing off at the end which takes much longer with small, focused, targeted tasks. I think this is how software development works in general. There is no one-shot solution; you always discover new things and need to steer as you go along. You learn what works and what doesn't. You perform refactors because files become too big. What really helped me, and this is a constant across most of my projects, is a true decoupling of test infra from production. It's such a convenient safety net. (More on this later.)

For anything substantious I heavily used a plan first loop. Plan, review the plan, implementation and then adverserial review or a 'break' prompt. I hopped models often between tasks, sometimes because of their strenghts, sometimes as second opinions while reviewing.

fdsfds

What else I think is noteworhty:

The prompts are part of the production code

The bootstrap instruction block, which tells the agent to run the prime command at session start, is a critical piece in the entire mechanism of loading skills. If the model skips or only partially follows it, the skills are not loaded and will not work. So I went through some extra lengths and treated this prompt as a testable artifact. I don't know, but this feels a bit unusual. I think most projects just add this to the README. But since it was critical, an interesting exercise and I do think the prompt is part of what makes the tool work, I exercised some extra testing discipline. I introduced a testing loop where I could test the instructions by running twenty sessions, running an end-to-end test using the zero stack agent, and it would report back the success rate. Since the end-to-end tests were different from the other acceptance tests, they have a different testing DSL.

Reflection

Good practices take time, also with agents

Honestly, I'm not entirely sure how other people do this, but sticking to best practices still feels like it takes a lot of time. For example I invest a lot in writing tests. While I think this is definitely worthwhile for implementing and verifying functionality, I did have some doubts along the way whether I should keep adopting or putting in the effort of doing things right. Of course, this is a simple hobby project, but I tend to use these also to explore how, let's say, traditional best practices map to practices when developing with agents. Should we rethink completely or not? I'm not sure, but what I do know is that it is worthwhile to invest time in proper test design with the key property being a decoupling of the test suite from the actual implementation. With my current test setup in this project, I can confidently ask it to make changes.

I have now ended up with twice as much test code as production code.

❯ scripts/test_to_source_ratio.sh
Test-to-source ratio: 2.00×  (100% more test code than source)
  Source lines: 1329
  Test lines:   2663

I think testing is where agents can help a great deal but only when you stay engaged with them as much as with the production code. It comes up with test cases easily and adds them well if it has good examples to follow. (Just to note, not every exception condition it comes up with is worthwhile pursuing. Sure, in theory it was right to think of all permission or file read/write things that could go wrong, but setting those up would be very exhaustive and frankly quite exotic. For those cases, I've decided to take the pragmatic approach.)

I am aiming for speed-up in the practices that help us to build better products but I often feel in a battle to go faster and rely or trust that the agent will be able to keep on fixing things for me as we keep going. This is not sustainable in the long run but I do wonder if there is a middle ground.

Start of well and stay in control

Starting off a new project, a definite best practice is to start with a lot of hand holding and giving good examples to the agent. Providing a good structure will help the model decide how to implement the functionality and the tests. This requires constant review to ensure the agent stays in line.

Even when you have a substantial body of examples, for example for the tests, it's always good to keep on reviewing and adjusting. By adjusting, I mean updating the instructions to the agent, for example the skills or the AGENTS.md file.

See Implementing and understanding Agent Skills for the extracted section on how skills integration works under the hood.