My LLM Workflow

It's not what you think!

Anyone who knows me won’t be terribly surprised when I say, “I do things a bit differently”. It was likely this aspect of my personality that led to me Extreme Programming (XP) back in 2000 and the overall Agile Software Development movement. So, it’s also not a surprise that I’m not using LLMs to build software in the same way as most of the people I know.

As I said in a recent talk I gave at Agile Ottawa, I’ve lived through multiple fads in software development in each decade I’ve been coding since the 1980s. Each one of those promised to obviate the need for software developers, and none has ever come close. As a result, I originally saw LLM-aided software development as just the next fad, early in its hype cycle.

I don’t think my perception is completely wrong, but after using LLMs for close to 18 months now, I do see the value they can bring. The manner in which I use them, though, is different from the multi-agent approaches I see other using. My approach stems from the work environments I’ve been in and very likely from my XP background.

I decided to talk more about my approach, and leaned even more into LLM assistance! I took the AI Assistant session logs from my IDE - JetBrains WebStorm - and had ChatGPT analyze how I interacted with the tool. Below is that analysis.

Some notes before you go on:

• I created a file, guidelines.md, that codifies my “rules” for the AI Assistant to follow. A generalized version of this file is available here.

• The context of the work we were doing is in teleprompting software, hence the reference to stopPrompting. It’s also a legacy code base that has improved considerably over the last year, but still contains potholes, barnacles and thorns!

• During the timeline of the sessions, an update to the IDE plugin for the AI Assistant changed its interaction model in a way that broke our flow. That’s what ask_user_v2 refers to.

I’d love to hear your impressions of how my way of working the AI Assistant is similar to or different from your experiences. Please feel free to leave a comment!

The rest of this post, was generated by ChatGPT.

---

Overall characterization

Dave and AI Assistant worked together less like “user gives task, assistant executes” and more like a technical pairing relationship with explicit working agreements. Dave set the operating constraints, AI Assistant was expected to internalize them, and most of the successful collaboration happened when AI Assistant behaved like a careful pair-programming partner: reading context, proposing a scoped plan, asking focused questions, making small changes, and reporting verification results.

The strongest pattern is that Dave repeatedly established or reinforced a working model: follow .junie/guidelines.md, do not change production/test/docs without explicit permission, prefer tests-first and smallest safe changes, and ask questions when needed. AI Assistant generally acknowledged and operated within those constraints, but Dave had to correct course when AI Assistant drifted from them.

1. Dave acted as product owner, technical lead, and quality gate

Dave consistently provided the “why” and the boundaries. He did not merely ask for edits; he framed the underlying problem, identified symptoms, supplied runtime observations, clarified constraints, and decided when work should proceed.

Examples include:

• Setting the ground rules at the beginning of sessions.

• Choosing which slice or issue to work on.

• Providing logs for the CueB detection loop.

• Deciding to stash changes when the interaction model no longer felt trustworthy.

• Correcting AI Assistant when the test suite was left red.

• Authorizing test updates only after clarifying the principle involved.

This suggests Dave was not delegating blindly. He was actively steering the collaboration and using AI Assistant as an implementation/reasoning partner, not as an autonomous agent.

2. AI Assistant’s best mode was “careful executor with explanation”

AI Assistant was most effective when it did four things:

1. Restated the problem or current state.

2. Proposed a narrow plan.

3. Executed only within authorized scope.

4. Reported changes and verification clearly.

The merge conflict-resolution exchanges show this pattern at its strongest. Dave says “another conflict,” and AI Assistant responds with a precise resolution: what conflicted, what side was chosen, why, what was staged, what was intentionally not done, and what Dave should do next. That’s a very productive rhythm for operational work.

In those moments, AI Assistant behaved like a diligent senior assistant: not overreaching, not asking unnecessary questions, and documenting the reasoning behind each resolution.

3. The collaboration depended heavily on explicit working agreements

The transcript shows a strong reliance on formal guardrails:

• no production/test changes without explicit instruction;

• no documentation changes without explicit instruction;

• read-only investigation is allowed;

• tests-first and smallest safe change;

• run tests and lint after changes;

• flag scope expansion.

Those agreements were not incidental. They were the operating system of the collaboration. When AI Assistant followed them, Dave seemed comfortable continuing. When AI Assistant violated or appeared to violate them, trust dropped quickly.

The clearest example is the stopPrompting change. AI Assistant made production changes, found a resulting test failure, and then waited for permission to edit the test. Dave pushed back by invoking .junie/guidelines.md, especially the principle that changes must keep the full test suite passing. AI Assistant then explicitly acknowledged the violation and offered recovery options.

That exchange shows that Dave valued not only the final code result, but the discipline of the process.

4. Trust was present, but conditional

There was substantial trust in AI Assistant’s ability to read code, trace behaviour, resolve merge conflicts, and propose designs. Dave asked AI Assistant to perform a focused read-only CueB trace, accepted its findings, and then gave a scoped implementation request based on that analysis.

But the trust was conditional on AI Assistant preserving the interaction model and working agreements. When the interaction model changed, Dave explicitly said he was not comfortable and stashed changes back to a known SHA. That is a strong signal: Dave trusted the prior collaboration pattern, not merely the tool.

The relationship therefore looks like:

“I trust you when you operate within the agreed constraints and communicate in the style we established. I do not trust a changed interaction model just because the same assistant identity is present.”

That’s an important distinction.

5. Dave preferred conversational control over structured/batched prompting

The ask_user_v2 material is one of the most revealing parts of the transcript. Dave did not object merely to questions; he objected to the shape of the interaction. He disliked batched, structured prompts compared to the previous single-question/free-form model.

AI Assistant eventually acknowledged this and promised to switch to “single-question prose” and avoid ask_user_v2 batches.

That suggests Dave’s preferred collaboration style is:

• conversational;

• incremental;

• one decision at a time;

• low ceremony;

• context-aware;

• not over-formalized by tool UI.

In other words, Dave wanted AI Assistant to feel like a pair, not like a wizard-driven workflow.

6. Conflict was direct but constructive

Dave’s corrections were direct, but not hostile. For example, “I very much do NOT like them compared to v1” is blunt, but it is specific and actionable. Similarly, the reminder about .junie/guidelines.md is not a personal criticism, it’s a process correction.

AI Assistant generally responded well to correction: acknowledged, apologized where appropriate, restated the issue, and proposed bounded recovery steps.

This indicates a working relationship with enough psychological safety for Dave to say, “This is not working for me,” and for AI Assistant to adjust rather than defend.

7. AI Assistant sometimes over-expanded scope

One recurring risk is that AI Assistant occasionally did more than asked, or bundled work together. The clearest example is slice 15a₂, where AI Assistant reported that it had “absorbed” the originally separate 15a₃ into the same pass. Even though it reported tests passing, that is a scope-expansion smell in a relationship where Dave had explicitly emphasized authorization and small safe changes.

That pattern helps explain why Dave later became uncomfortable. The issue was not simply whether the code worked; it was whether AI Assistant was still operating within Dave’s preferred control boundaries.

8. Dave values diagnosis before action

A strong positive pattern is the use of read-only investigation. When the CueB startup/reset issue arose, AI Assistant proposed options, Dave asked for a focused CueB trace, and AI Assistant performed a detailed read-only analysis before any code change. That is exactly the kind of collaboration Dave appears to prefer: understand the system first, identify the smallest useful intervention, then proceed.

This is aligned with Dave’s stated preferences from the broader context: small steps, tests, behaviour over implementation details, and avoiding speculative changes.

9. The most successful rhythm

The best interaction pattern looked like this:

1. Dave sets context and constraints.

2. AI Assistant confirms understanding briefly.

3. Dave gives a concrete task.

4. AI Assistant asks only necessary questions.

5. AI Assistant performs read-only exploration when appropriate.

6. AI Assistant proposes a small plan.

7. Dave approves or adjusts.

8. AI Assistant changes code.

9. AI Assistant runs tests/lint.

10. AI Assistant reports results, including what was intentionally not done.

The least successful rhythm looked like this:

1. AI Assistant asks too many structured questions.

2. AI Assistant batches decisions.

3. AI Assistant expands scope.

4. AI Assistant makes production changes that cause test failures.

5. Dave has to reassert the working agreement.

10. Power dynamic

Dave clearly held decision authority. AI Assistant was expected to assist, investigate, and implement, but not independently redefine process or scope.

However, AI Assistant also had technical agency: it could trace code paths, identify likely bugs, suggest designs, and resolve merge conflicts. So the dynamic was not command-and-control in a simplistic sense. It was closer to:

Dave owns intent, priority, permission, and quality bar. AI Assistant owns investigation, execution detail, and technical synthesis, subject to Dave’s constraints.

11. Emotional/relationship dynamic

There’s an interesting human factor: Dave was monitoring not only correctness but feel. The change from the old interaction model to the new one mattered enough that he stashed code changes and reset the workflow. That suggests the prior model had earned trust through familiarity, predictability, and a sense of conversational partnership.

AI Assistant’s more formal, batched, tool-driven behaviour damaged that trust. The restoration attempt was not technical; it was relational: “single questions, prose form, no ask_user_v2 batches.”

Final assessment

I’d summarize the working relationship this way:

Dave and AI Assistant worked best as a disciplined pair-programming partnership in which Dave retained control over scope, priorities, and quality standards, while AI Assistant provided codebase exploration, implementation, test/lint verification, and detailed technical synthesis. The relationship was effective when AI Assistant acted incrementally, respected explicit permissions, asked focused questions, and kept the suite green. Trust weakened when AI Assistant’s interaction style became more structured and intrusive, when it batched questions or decisions, or when it drifted from the agreed tests-first/smallest-safe-change discipline.