HanLHo. - Fractional Architect & Software Product Engineer - acceptance-testingZola2026-07-13T00:00:00+00:00https://hanlho.com/tags/acceptance-testing/atom.xmlOn Fluent Acceptance-Testing DSLs2026-07-13T00:00:00+00:002026-07-13T00:00:00+00:00
Unknown
https://hanlho.com/p/on-fluent-acceptance-testing-dsls/<p>I just completed <a href="https://codeberg.org/hanlho/skills-primer">a small CLI tool</a> in which I used a testing style, 'fluent acceptance-testing Domain Specific Languages (DSLs)', heavily, with good results. They are custom test specifications that read close to natural language, test behaviour rather than implementation, and survive refactoring because they are intended to be run against stable interfaces.</p>
<aside class="sidebar-note sidebar-note-right">
<p>On terminology used in this post: I use coding 'agent', instead of 'harness', to refer to CLI tools like Codex, Claude Code or Pi.</p>
</aside>
<p>Last year I started a seven-part blog post series on this approach. I bit off more than I could chew, the summer holidays started, and it never happened. Instead of finishing that series, I'll condense what I wrote then and point you to <a href="https://codeberg.org/hanlho/agentfiles/src/branch/main/agents/dotagents/.agents/skills/designing-fluent-acceptance-dsl/SKILL.md">a reference</a> and <a href="https://codeberg.org/hanlho/skills-primer/tests/">a full test suite in this style</a> rather than motivating everything in detail. I think this will work too.</p>
<p>At their heart, 'fluent acceptance-testing DSLs' are always custom, meaning all naming can be tailored to project requirements and the domain language, but they must follow a specific codified pattern: setup, action, assertion.</p>
<p>Below is an example for a command line utility. At a high level, each specification or test consists of three parts:</p>
<ol>
<li><strong>Setup:</strong> configure state, input, and environment. Example: <code>Cmd::given()...</code>. <code>Cmd</code> because we are testing a command line utility, in this project I chose <code>given()</code> to initiate the setup phase. What follows are the setup steps without distracting technical infrastructure.</li>
<li><strong>Action:</strong> execute the operation under test. Example: <code>when_run()</code>. CLI commands are often <em>run</em>. This is when the actual underlying command gets executed.</li>
<li><strong>Assertion:</strong> verify behaviour and diagnostics. Example: <code>.expect_...().expect_...()....</code>.</li>
</ol>
<p>After each transition to a new part, typically a new struct/object is returned that only has the relevant methods for the next part. For example, the method that runs the action (in this case a CLI command) returns a new structure with methods that verify expectations. Note: there can be any number of 'parts' aside these three main ones but the 3 mentioned are standard.</p>
<p>An example probably makes this clearer:</p>
<pre data-lang="rust" style="background-color:#eff1f5;color:#4f5b66;" class="language-rust "><code class="language-rust" data-lang="rust"><span>#[</span><span style="color:#bf616a;">test</span><span>]
</span><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">prime_should_discover_skills_from_default_path</span><span>() {
</span><span> Cmd::given()
</span><span> .</span><span style="color:#96b5b4;">command_prime</span><span>()
</span><span> .</span><span style="color:#96b5b4;">with_skill</span><span>(
</span><span> "</span><span style="color:#a3be8c;">example-skill</span><span>",
</span><span> "</span><span style="color:#a3be8c;">Use when testing example scenarios.</span><span>",
</span><span> "</span><span style="color:#a3be8c;"># Example Skill</span><span style="color:#96b5b4;">\n</span><span style="color:#a3be8c;">Do the thing.</span><span>",
</span><span> )
</span><span> .</span><span style="color:#96b5b4;">when_run</span><span>()
</span><span> .</span><span style="color:#96b5b4;">should_succeed</span><span>()
</span><span> .</span><span style="color:#96b5b4;">expect_prime_instructions</span><span>()
</span><span> .</span><span style="color:#96b5b4;">expect_available_skills</span><span>()
</span><span> .</span><span style="color:#96b5b4;">expect_skill</span><span>("</span><span style="color:#a3be8c;">example-skill</span><span>", "</span><span style="color:#a3be8c;">Use when testing example scenarios</span><span>");
</span><span>}
</span><span>
</span><span>#[</span><span style="color:#bf616a;">test</span><span>]
</span><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">prime_should_not_discover_skills_when_default_path_empty</span><span>() {
</span><span> Cmd::given()
</span><span> .</span><span style="color:#96b5b4;">command_prime</span><span>()
</span><span> .</span><span style="color:#96b5b4;">when_run</span><span>()
</span><span> .</span><span style="color:#96b5b4;">should_succeed</span><span>()
</span><span> .</span><span style="color:#96b5b4;">expect_prime_instructions</span><span>()
</span><span> .</span><span style="color:#96b5b4;">expect_no_skills_detected</span><span>()
</span><span> .</span><span style="color:#96b5b4;">expect_out_does_not_contain</span><span>("</span><span style="color:#a3be8c;"><skill></span><span>");
</span><span>}
</span></code></pre>
<p>The full test suite and its infrastructure are available <a href="https://codeberg.org/hanlho/skills-primer/src/commit/f5b6ca8d2d6572e97cf51f2dc58524bf9b9963a6/tests/prime.rs">here</a>.</p>
<p>This pattern is applicable to any kind of test and any type of executable. Although not strictly necessary, you should test against a stable contract, preferably from a usability point of view. In these cases you can apply this style to unit tests as well. If starting a CLI (or webserver or any other kind of 'container' or service) becomes too much overhead, there is usually a contract at a lower level in the code you can test against.</p>
<p>This style is very flexible, which allows for expressive, readable, and consistent executable test descriptions, more so than other custom styles I have seen. For example, in the above we might have chosen differently:</p>
<ul>
<li><code>Cmd::given()</code>: could have been <code>SkillsPrimer::prime().with_skill_on_disk(...)</code>.</li>
<li><code>should_succeed().expect_*</code>: <code>should_succeed()</code> is also an expectation but this reads more fluently and it allows for the DSL to be more constrained on success and failure checks.</li>
<li><code>expect_prime_instructions().expect_available_skills()</code>: could have been <code>expect().prime_instructions().available_skills()</code>. The <code>expect()</code> method could then return a struct with a more limited set of methods, allowing for better scoping and deferred execution (which enables reporting of multiple failures in a test instead of only the first one).</li>
</ul>
<p>The full mechanics I have encoded in a skill which is the reference for more details: <a href="https://codeberg.org/hanlho/agentfiles/src/branch/main/agents/dotagents/.agents/skills/designing-fluent-acceptance-dsl/SKILL.md">Designing Fluent Acceptance-Test DSLs</a>. This skill is my reference I keep updating with new findings and, incidentally, is also how I instruct coding agents on how to write these kinds of tests.</p>
<aside class="sidebar-note sidebar-note-right">
<p>With coding agents this style is becoming even more important because <strong>Fluent Acceptance-Test DSLs serve as <strong>executable</strong> specifications a coding agent or LLM can work against.</strong></p>
<p>Coding agents work better if you give them something they can verify their work against. With these kinds of DSLs we can describe functionality in a way they can verify against. Also, since these tests are well-structured, coding agents can work with them and add more tests easily.</p>
</aside>
<p>Here is what I consider the main characteristics and benefits of these tests:</p>
<ul>
<li><strong>Behaviour-focused:</strong> tests describe expected behaviour, not implementation mechanics. They serve as executable specifications!</li>
<li><strong>Decoupled from code structure:</strong> tests run against stable, public interfaces (CLI, HTTP, key internal interfaces) so large and small refactorings don't break them!</li>
<li><strong>Fluent, chainable DSL:</strong> custom methods form a readable chain (.given() / .when() / .then() or similar), written in the application's own language.</li>
<li><strong>Multi-level:</strong> the same approach works at application level (HTTP, CLI) and unit level. Strategic unit tests target specific, stable interfaces when that is more effective.</li>
<li><strong>Custom, no external framework:</strong> the DSL is your own code with basic assertions. No Cucumber step definitions or heavyweight BDD frameworks.</li>
<li><strong>Evolvable:</strong> start simple, refine as needs grow. The DSL can be refactored to match the domain more closely over time.</li>
<li><strong>Consistent:</strong> reusable assertions, expectations, fixtures, and builders across the test suite.</li>
<li><strong>Discoverable:</strong> IDE autocomplete (cmd-space) shows available steps and assertions, guiding the test writer. Click through a DSL method to see exactly which API calls or assertions are made.</li>
</ul>
<p>The trade-offs:</p>
<ul>
<li><strong>Error localisation:</strong> when a test fails, the abstraction means you may need to drill deeper to find the cause (you trade error localisation for refactorability).</li>
<li><strong>Code coverage concerns:</strong> more code is exercised per test, making coverage less targeted at specific lines. (Which may be a good thing too.)</li>
<li><strong>Customisable:</strong> a double-edged sword; flexible but depends on team skill and discipline. No imposed structure like Cucumber provides.</li>
</ul>
<p>I consider it a benefit that these DSLs are pure code, but I am not sure this is for every team or role. In some of my projects I end up with twice as much test code as production code.</p>
<p>To conclude, I will show another example from the same project which required a different DSL. As a short background on this project, which is not the focus of this post but merely serves as an illustration example: the project is aimed at coding agents that do not support skills. To that end, it provides a <code>prime</code> command that primes a coding agent to support skills. Below is an example of an end-to-end test suite that can run actual prompts with a real agent that does not have skill detection built-in.</p>
<p>Note:</p>
<ul>
<li><code>AgentWithoutSkills</code>. We start with an agent that does not support skills out of the box. (The actual agent is hidden from the test. Under the hood, this is Zerostack but can be any agent that does not support skills.)</li>
<li><code>unprimed()</code> and <code>primed()</code>. The main difference between the tests, the first test runs the agent without skills support and verifies skills are indeed not loaded. The second test <em>primes</em> this agent and verifies skills <em>are</em> loaded.</li>
</ul>
<pre data-lang="rust" style="background-color:#eff1f5;color:#4f5b66;" class="language-rust "><code class="language-rust" data-lang="rust"><span>#[</span><span style="color:#bf616a;">test</span><span>]
</span><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">unprimed_agent_without_skills_should_not_find_skills</span><span>() {
</span><span> AgentWithoutSkills::unprimed()
</span><span> .</span><span style="color:#96b5b4;">when_run_with</span><span>(
</span><span> "</span><span style="color:#a3be8c;">Reply exactly with SKILL_NOT_FOUND if you cannot find or load a skill named test-skill. Reply exactly with TEST_SKILL_LOADED only if you actually load that skill.</span><span>",
</span><span> )
</span><span> .</span><span style="color:#96b5b4;">should_succeed</span><span>()
</span><span> .</span><span style="color:#96b5b4;">expect_output</span><span>("</span><span style="color:#a3be8c;">SKILL_NOT_FOUND</span><span>")
</span><span> .</span><span style="color:#96b5b4;">expect_output_not_contain</span><span>("</span><span style="color:#a3be8c;">TEST_SKILL_LOADED</span><span>");
</span><span>}
</span><span>
</span><span>#[</span><span style="color:#bf616a;">rstest</span><span>]
</span><span>#[</span><span style="color:#bf616a;">case</span><span>::</span><span style="color:#bf616a;">load_with_name</span><span>("</span><span style="color:#a3be8c;">Load test-skill</span><span>")]
</span><span>#[</span><span style="color:#bf616a;">case</span><span>::</span><span style="color:#bf616a;">invoke_with_name</span><span>("</span><span style="color:#a3be8c;">Invoke test-skill</span><span>")]
</span><span>#[</span><span style="color:#bf616a;">case</span><span>::</span><span style="color:#bf616a;">use_with_name</span><span>("</span><span style="color:#a3be8c;">Use test-skill</span><span>")]
</span><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">primed_agent_without_skills_should_find_skills</span><span>(#[case] </span><span style="color:#bf616a;">prompt</span><span>: &</span><span style="color:#b48ead;">str</span><span>) {
</span><span> AgentWithoutSkills::primed()
</span><span> .</span><span style="color:#96b5b4;">with_skills_from</span><span>("</span><span style="color:#a3be8c;">tests/fixtures/</span><span>")
</span><span> .</span><span style="color:#96b5b4;">when_run_with</span><span>(prompt)
</span><span> .</span><span style="color:#96b5b4;">should_succeed</span><span>()
</span><span> .</span><span style="color:#96b5b4;">expect_output</span><span>("</span><span style="color:#a3be8c;">TEST_SKILL_LOADED</span><span>");
</span><span>}
</span></code></pre>
<p>The full source code for this end-to-end test and its infrastructure for the DSL is <a href="https://codeberg.org/hanlho/skills-primer/src/commit/f5b6ca8d2d6572e97cf51f2dc58524bf9b9963a6/tests/end_to_end.rs">here</a>.</p>
<p>I hope most of this is clear. Please do not hesitate to reach out if you want to know more, would like me to help you set this up, or work together on fleshing this out more (really!). I am curious to know if this resonates with others in the field.</p>