Letting Claude open real pull requests from your feedback inbox

This is the post where the product earns its name. A user submits feedback, it’s tagged and dedup’d, and then a PR appears on the customer’s repo with Claude’s attempt at the fix. The magic trick isn’t the code generation — Claude is fine at that. The magic trick is all the plumbing around it: GitHub App installation, repo selection, branch protection, compilation, and retries.

GitHub App, not OAuth

I briefly considered shipping with a personal access token the user pastes in. This is the wrong call for four reasons: it’s a bad security posture, it can’t be scoped to a single repo, it can’t be revoked cleanly, and no enterprise customer will ever install it.

FeedbackIQ is a GitHub App. The customer installs it from github.com/apps/feedbackiq, picks specific repos, and grants contents: write, pull-requests: write, metadata: read, and nothing else. We store only the installation ID; we never see the user’s personal credentials. Installation tokens are short-lived, scoped to a single repo, and re-minted on every run.

The agent loop

When a feedback item is marked for PR generation (either manually from the dashboard or automatically if auto-mode is on), we enqueue a job that:

1. Mints a short-lived installation token for the target repo.
2. Clones the repo into a fresh ephemeral sandbox.
3. Hands the feedback text, screenshot (if any), and repo path to the Claude Agent SDK.
4. Lets Claude read files, propose edits, run the project’s build/typecheck, and iterate until it either succeeds or hits a step limit.
5. Commits on a branch like feedback/a3c8f2, pushes, opens the PR, and writes a link back to the feedback row.

The Agent SDK does the heavy lifting: tool use, file reads, edits, and shelling out to run npm run build. We wrap it so every action is logged and checkpointed, which makes debugging a failed run tractable.

Typecheck before PR

For the first month, we opened PRs regardless of whether the change compiled. About 30% of them were “close but not compiling” — a type error, a missing import, a renamed file the agent didn’t see. These PRs were worse than no PR because reviewers would open them, see the red CI, and lose trust.

Fix: before we commit, the agent runs the project’s own typecheck + build. If it fails, we feed the compiler output back into the agent, let it take one more pass, and try again. Only if the build then fails do we still commit — with a PR description that honestly says “compile failed, here’s the output.” Honesty beats hiding a broken branch.

Branch reuse and force-push

A single feedback item can go through the pipeline more than once — the user added more detail, the agent’s first pass was wrong, a repo change invalidated the old branch. We used to create a new branch each retry, which left a graveyard of feedback/* branches in the customer’s repo.

Fix: reuse the branch if it exists and force-push. Reuse the PR if it’s open. Each retry cleanly updates the same PR in place. Customers now see a single live PR that improves over time instead of a drawer full of stale ones.

Review-only or auto-mode

Auto-mode is a per-project toggle. In review-only mode, feedback items sit in the dashboard until a human clicks “Generate PR.” In auto-mode, the pipeline runs as soon as the item clears dedupe. Both are useful, to different teams:

• Review-only for teams that want a human to decide what’s worth shipping. PR generation becomes a button, not a daemon.
• Auto-mode for teams that trust the pipeline and treat PR review as the gate, not the classification. These tend to be smaller, scrappier teams — and they ship faster.

What we got wrong

The first version committed directly to main. Yes, really. No branch. It took exactly one customer asking “uh, did we skip review?” to fix it. That’s a mistake I will tell the story about for years.

Next up: upvote routing when feedback items get merged as duplicates. The parent owns the vote count, the child is hidden, and the voter never knows they clicked a stand-in.