Azure DevOps Integration
Connect Otto to Azure DevOps to create pull requests and respond to code reviews. Otto uses a service principal (application identity) to access your Azure DevOps organization.
Prerequisites
Before connecting, you need:
- An Azure DevOps organization
- Admin access to your Azure DevOps organization settings (to add the service principal)
- Your Azure AD Tenant ID (found in Azure Portal → Microsoft Entra ID → Overview)
Connecting Azure DevOps
Step 1: Add Otto's Service Principal to Azure DevOps
- In Azure DevOps, go to Organization Settings → Users → Add users
- Search for Otto's service principal by app name or client ID
- Set access level to Basic
- Add the service principal to relevant projects as Contributor
Step 2: Connect in Otto
- Go to Settings → Integrations in your Otto dashboard
- Click Connect on the Azure Repos card
- Enter your Azure DevOps organization name (from
https://dev.azure.com/your-org-name) - Enter your Azure AD Tenant ID
- Click Connect
Otto validates access to your organization and automatically creates webhook subscriptions for PR events.
How Otto Uses Azure DevOps
Creating Branches
When Otto starts work on a work item, it:
- Creates a new branch from your default branch
- Branch name:
otto/[work-item-id]-[slug] - Example:
otto/1234-add-user-avatar
Making Commits
Otto commits changes with:
- Clear commit messages describing the change
- Linked work items in commit messages
- Co-authored attribution when responding to feedback
Opening Pull Requests
Otto's PRs include:
- Descriptive title matching the work item
- Summary of changes
- Link to the Jira or Azure DevOps work item
- Implementation notes
- Test instructions (if applicable)
Responding to Reviews
When you leave review comments, Otto:
- Reads the feedback
- Makes the requested changes
- Pushes a new commit
- Responds to the comment with a summary
The otto/review Status
Whenever Otto starts a fresh review of a pull request, it posts an otto/review status on the PR iteration it is reviewing. A fresh review happens when a PR is created, when new commits are pushed (if your reviewer trigger listens to PR updates), or when someone comments /otto review. Replies to comments don't touch the status.
The status says whether Otto is still reviewing. It never reflects Otto's verdict, which Otto records as its reviewer vote (approve or wait for author):
| State | Meaning |
|---|---|
| Pending | Otto is reviewing this iteration. |
| Succeeded | Otto finished reviewing (approved, requested changes, or commented). |
| Not applicable | Otto couldn't finish: it errored, timed out (after 60 minutes), the PR was too large, or the org is out of credits. |
A required status policy treats "not applicable" as not applying to the PR, so a finished or broken Otto review never blocks completion. Comment /otto review to retry.
Waiting for Otto before completing a PR
To stop PRs from being completed while Otto is reviewing, add a status check policy to the target branch (Branch policies > Status checks > +):
| Setting | Value |
|---|---|
| Status to check | otto/review |
| Policy requirement | Required |
| Authorized identity | Otto's service principal, so only Otto can satisfy the policy |
| Reset conditions | Optional; see the caveat below |
| Policy applicability | Apply by default |
Caveat: a required status that is never posted blocks completion. Otto only posts it on PRs it reviews, so require it only on branches where Otto reviews every PR. With Reset status whenever there are new changes turned on, every push resets the policy until Otto reviews the new iteration. Only turn it on when your PR reviewer trigger also runs on PR updates; otherwise each push blocks the PR until someone comments
/otto review.
Webhook Events
Otto automatically subscribes to these Azure DevOps events:
| Event | Description |
|---|---|
| PR created | New pull request opened |
| PR updated | Pull request modified |
| PR comment | Comment added to pull request |
Required Permissions
The Otto service principal needs these permissions in your Azure DevOps organization:
| Permission | Purpose |
|---|---|
| Code (read/write) | Read code, create branches, push commits |
| Pull requests (read/write) | Create and update pull requests |
| Work items (read/write) | Link PRs to work items |
| Identity (read) | Access user and team information |
Best Practices
Start with a Test Project
Initially, give Otto access to a test repository to validate your configuration before expanding to production repos.
Use Branch Policies
Always require PR reviews and build validation before completing PRs from Otto.
Enable CI/CD Pipelines
Set up automated tests and deployment pipelines that run on Otto's PRs to catch issues early.
Work Item Linking
Ensure work items are properly linked in PRs for traceability and audit purposes.
Team-Specific Agents
If you have multiple teams or products, create separate agents with team-specific context for better results.
Troubleshooting
"Failed to validate access"
The service principal has not been added to your Azure DevOps organization, or it lacks sufficient permissions.
Fix: Go to Azure DevOps → Organization Settings → Users → Add users, search for Otto's service principal, and grant Basic access level.
"Tenant ID not found" or token acquisition error
The Tenant ID may be incorrect.
Fix: In the Azure Portal, navigate to Microsoft Entra ID → Overview and copy the Tenant ID value.
Webhook subscriptions not created
Otto creates webhook subscriptions automatically during connection. If they fail, Otto will still connect but won't receive PR events.
Fix: Disconnect and reconnect the integration, or contact support.