Have AI fix failing tests and open a pull request
This guide has Alfred work on a repository the way a developer would. He clones it into his own machine, runs the tests, finds why they fail, fixes the code on a branch, and opens a pull request. You review it like any other PR.
Before you start
Section titled “Before you start”- Alfred is installed on the repository. Configure Alfred on GitHub and pick the account or organization, then the repositories he may reach.
- You’re on a paid plan. The work happens in his Sandbox VM, which is part of Power and up. See Patreon.
- The Sandbox VM module is on. Check it on the Modules page.
1. Ask for the fix
Section titled “1. Ask for the fix”In the Dashboard, name the repository and the result you want:
Clone my api repo, work out why the tests fail on main, and open a PR with the fix.
One repository and one task per request works best. If you already know something about the failure, say it: which test, since when, what changed.
What he does
Section titled “What he does”- Starts a sandbox, a Debian machine of his own with git, the GitHub CLI, Node, Python and the usual build tools already installed.
- Clones the repository and runs the tests to see the failure for himself.
- Reads the code involved, makes a change on a new branch, and runs the tests again.
- Opens a pull request with the fix.
He never pushes to your default branch. The change reaches you as a pull request, so nothing ships until you merge it.
If the project needs something the sandbox doesn’t have, he installs it. A project with an unusual setup goes faster when you tell him how to run it, or keep the steps in the README.
2. Review the pull request
Section titled “2. Review the pull request”Read the PR as you would a colleague’s. To ask for changes, go back to the same chat:
- “The fix works, but move the retry into the client class.”
- “Add a test for the case where the session has already expired.”
- “Reply to the review comment on the PR and push the change.”
He keeps the same sandbox for the conversation, so follow-up changes start from where he left off.
Other things to ask for
Section titled “Other things to ask for”- “Bump the dependencies in this repo and open a PR if the build still passes.”
- “Read through this repo and write me a summary of how the auth flow works.”
- “Find out why the build is slow and open a PR that speeds it up.”
For work that will take hours, such as a migration across a large codebase, ask for it as a job. He plans it, works through it in stages and reports back.
What he can reach
Section titled “What he can reach”- Only the repositories you installed him on. Change that at any time in your GitHub settings, under installed apps.
- Not your credentials. Your GitHub authentication is applied outside the sandbox, so neither the AI nor anything it runs in there ever holds your token.
This is real work on a real machine, so it uses more of your allowance than a chat does. A large test suite or a long build costs more than a small one. See Usage.
If it doesn’t work
Section titled “If it doesn’t work”- He can’t find the repository. He isn’t installed on it. Add it in your GitHub settings, under installed apps.
- He says he has no sandbox. Turn the Sandbox VM module on, and check that your plan includes it.
- The tests fail differently in his sandbox. Tell him about anything the tests need, such as a database, an environment variable or a specific Node version.