OpenReview

An open-source, self-hosted AI code review bot. Deploy to Vercel, connect a GitHub App, and get on-demand PR reviews powered by your configured model provider.

Beta: OpenReview is currently in beta. It was built as an internal project to help the Vercel team test their technologies together. Expect rough edges and breaking changes.

Features

• On-demand reviews — Mention the app in any PR comment to trigger a review. The handle is your GitHub App's slug, e.g. @rajeevs-openreview-bot. Powered by Chat SDK
• Automatic frontier quality gate — An optional bounded, deterministic-quality gate that judges agent-written PRs with at most two frontier-model calls per review cycle
• Sandboxed execution — Runs in an isolated Vercel Sandbox with full repo access, including the ability to run linters, formatters, and tests
• Inline suggestions — Posts line-level comments with GitHub suggestion blocks for one-click fixes
• Code changes — Can directly fix formatting, lint errors, and simple bugs, then commit and push to your PR branch
• Reactions — React with 👍 or ❤️ to approve suggestions, or 👎 or 😕 to skip
• Durable workflows — Built on Vercel Workflow for reliable, resumable execution
• Extensible skills — Ships with built-in review skills and supports custom skills via .agents/skills/
• Configurable models — Uses your configured provider and model via the AI SDK
• Simple route handler — Easily define route handlers using Next.js Route Handlers for custom API endpoints and webhooks

How it works

1. Mention the app's slug in a PR comment, e.g. @rajeevs-openreview-bot (optionally with specific instructions)
2. OpenReview spins up a sandboxed environment and clones the repo on the PR branch
3. A configured AI agent reviews the diff, explores the codebase, and runs project tooling
4. The agent posts its findings as PR comments with inline suggestions
5. If changes are made (formatting fixes, lint fixes, etc.), they're committed and pushed to the branch
6. The sandbox is cleaned up

Setup

1. Deploy to Vercel

Click the button above or clone this repo and deploy it to your Vercel account.

In this repository, main deploys to production automatically; automatic previews are disabled for every other branch to avoid burning build quota. Run the Preview on demand workflow (or vercel deploy) when you need a preview. The policy and both escape hatches are in docs/vercel-deployments.md [blocked].

2. Create a GitHub App

Create a new GitHub App with the following configuration:

Webhook URL: https://your-deployment.vercel.app/api/webhooks

Repository permissions:

• Checks: Read & write (automatic frontier quality gate)
• Contents: Read & write
• Issues: Read & write
• Pull requests: Read & write
• Metadata: Read-only

Subscribe to events:

• Check run
• Issue comment
• Pull request
• Pull request review comment

Generate a private key and webhook secret, then note your App ID and Installation ID.

3. Configure environment variables

Add the following environment variables to your Vercel project:

Recommended Vercel setup:

OPENREVIEW_MODEL=anthropic/claude-sonnet-4.6OPENROUTER_API_KEY=your-openrouter-api-key

Example OPENREVIEW_MODEL values:

• anthropic/claude-sonnet-4.6
• openai/gpt-4o
• z-ai/glm-5

Fallback behavior:

• If OPENROUTER_API_KEY is present, OpenReview uses OpenRouter.
• Otherwise, if ANTHROPIC_API_KEY is present, OpenReview keeps the Anthropic path working.
• If neither key is configured, the agent fails fast with a clear configuration error.
• When using the Anthropic fallback path, OPENREVIEW_MODEL must be an anthropic/* model.

4. Install the GitHub App

Install the GitHub App on the repositories you want OpenReview to monitor. Once installed, mention the app's slug in any PR comment to trigger a review — the mention must match the App name exactly (@rajeevs-openreview-bot for this deployment), because the adapter matches on the App slug, not on the repository name.

Automatic frontier quality gate

Separate from the manual agent, the frontier quality gate automatically judges agent-written pull requests and reports a single stable frontier-quality check. It waits for your configured required CI before spending anything, skips low-value PRs (docs, assets, lockfiles) for free, and is bounded to two frontier calls per review cycle:

1. PR opened/updated → required CI green → one frontier review
2. Findings are surfaced on the check and as a PR comment; the originating agent fixes and pushes (any number of repair pushes costs $0)
3. Add the frontier-ready-final label → one delta-only review → PASS or BLOCK
4. After a BLOCK, a repair push is re-checked for free (changed files + green required CI, no model call); the check clears if every blocking finding resolves that way
5. Only the explicit frontier-new-cycle label can spend again

Configure it per repository with .github/frontier-review.yml:

frontier:  enabled: true  threshold: 5  always_review: ["src/agents/**", "benchmarks/**"]  never_review: ["generated/**"]

Set FRONTIER_DAILY_BUDGET_USD and FRONTIER_MONTHLY_BUDGET_USD to bound spend. Full design, signals, caps, labels and acceptance tests: docs/frontier-quality-gate.md [blocked]. Deployment, environment variables and operational traps: docs/frontier-quality-gate-operations.md [blocked].

Usage

Trigger a review: Comment the app's mention handle on any PR. The handle is the GitHub App slug — @rajeevs-openreview-bot on this deployment. The handle must match the App slug exactly, because the adapter matches on it — a bare @openreview does not work, and GitHub reserves that name for an existing account, which is why this App is rajeevs-openreview-bot. You can include specific instructions:

@rajeevs-openreview-bot check for security vulnerabilities@rajeevs-openreview-bot run the linter and fix any issues@rajeevs-openreview-bot explain how the authentication flow works

Reactions: React with 👍 or ❤️ on an OpenReview comment to approve and apply its suggestions. React with 👎 or 😕 to skip.

Skills

OpenReview uses a progressive skill system — the agent only loads specialized instructions when relevant, keeping context focused and reviews thorough. Skills are discovered from .agents/skills/ at runtime.

Built-in skills

Adding custom skills

Create a folder in .agents/skills/ with a SKILL.md file containing YAML frontmatter:

.agents/skills/└── my-custom-skill/    └── SKILL.md

---name: my-custom-skilldescription: When to use this skill — the agent reads this to decide whether to load it.---
# My Custom Skill
Your specialized review instructions here...

The agent sees only skill names and descriptions in its system prompt. When a request matches a skill, it calls loadSkill to get the full instructions — keeping the context window clean.

Tech stack

• Next.js — App framework
• Vercel Workflow — Durable execution
• Vercel Sandbox — Isolated code execution
• AI SDK — AI model integration
• Chat SDK — GitHub webhook handling
• Octokit — GitHub API client

Development

bun installbun dev

License

MIT