Agent 101 - AI SWE Agent
Agentic IDE designed before Claude Code
Overview
- Role & Responsibilities
- UX Engineer
- Period
- 2024 Q2 (1 week)
- Platform
- Desktop IDE prototype
What I did
- Researched the existing IDEs with AI agents and challenges in communicating novel AI workflow concepts
- Led the UI and UX design for the IDE
- Developed interactive prototype to illustrate the core SWE agent concept
Problem
The SWE Agent research team needed a tangible, real-world expression of their coding models: proof that agentic coding could work inside an authentic developer workflow, enough to secure executive sponsorship. Explained abstractly, the novel workflow was hard for stakeholders to grasp, stalling buy-in.
What I designed & built
Designed and prototyped an IDE showing a human-in-the-loop coding agent inside a real developer workflow. Visualizing the agent's Plan-Code-Execute process step-by-step made the abstract concept tangible for executives seeing it for the first time.
Presented at IBM's Technology Review, Agent 101 proved the SWE agent research viable and secured continued funding and executive sponsorship. It was later demoed publicly by IBM Research leadership at TechXchange, and the team has since ranked at the top of multiple SWE agent leaderboards.
Process
Start Where the Work Already Is: GitHub
No new surface to learn
The experience starts from a GitHub issue, not a new tool. Assigning a label is the entire onboarding, and the agent reports its progress (Creating plan → Executing plan → Pull request created) inside the issue thread, where the team already looks.
The bet was to introduce an autonomous agent without changing where developers work: adoption stories land harder than capability stories, and executives asked 'how does my team use this?' before 'how smart is it?' Enter through GitHub, exit through a pull request: a teammate inside existing rituals, not a new tool beside them.
Dispatch from the issue
Assigning the agent-101 label is the whole onboarding: the bot acknowledges in the thread and keeps its status where the team already looks.
A Plan You Can Watch, and Rewrite
The plan drafts itself, step by step
The agent's first act is a plan: steps appear one at a time, visible work instead of a spinner, and nothing executes until you press Execute Plan.
Plans are negotiable
Type a change into the command bar and the agent redrafts the plan in front of you. The run starts only when the plan is yours too.
Plans are negotiable
When a fix could go two ways, the agent asks instead of guessing with one-click chips; your pick lands in the trace and the plan redrafts to Execute Plan. The run starts only when the plan is yours too.
Execution You Can Audit
Thought → Action → Observation
Every turn shows the agent's reasoning, the exact tool call, and the real output, failures included, because showing the work is what makes the result credible.
Real tests, real diffs
Failing tests render red and every edit is a reviewable diff. Output is evidence, not narration.
Two altitudes of progress
Press Execute Plan and the sidebar mirrors the run: each step grows sub-turns that tick off live, so progress stays legible without reading the trace.
Human in the Loop
Ambiguity is a UI state, not a failure
When a decision could go more than one way, the agent doesn't guess. Its blocked reasoning renders as a red thought block: a silent wrong guess is the error, and it asks, with one-click answer chips. The decision lands in the trace and the run resumes: a teammate in the loop, not an approval modal.
Blocked, asking, resumed
The red block signals 'I won't guess'; option chips make answering a one-click decision that's recorded in the trace like a code-review thread.
Close the Loop: Slash Commands & the PR
Every run ends with a verdict
When the plan completes, a terminal-style command bar offers two slash-command exits: /feedback records your judgment as evaluation data for the agent, and /createpr packages the change as a pull request linked back to the issue where the run began.
A finished run is an artifact to judge, not a chat to scroll, so the exits are ship it or grade it, not thumbs on individual turns. And feedback captured at the moment of completion actually gets written; a survey later doesn't.
/feedback grades the run
A one-line /feedback note is recorded against the run. Human judgment becomes evaluation data for the agent.
From /createpr to a linked PR
One command produces a reviewable pull request and updates the issue timeline on GitHub. The loop closes where work started.
Learnings
- Effective methods for visualizing and explaining complex, autonomous AI workflows to audiences unfamiliar with the concept.
- Designing interfaces focused on transparency and clarity when introducing novel AI agent interactions.
- Initial developer reactions and considerations for human-AI trust and collaboration in coding tasks.
Next steps
- Explore technical feasibility further based on positive concept reception.
- Design potential user controls for human-in-the-loop interactions and autonomous agent management.
- Explore more detailed interactions and add more features