plugin

Plugin Eval

Start from chat, then evaluate or benchmark locally

.mdOpen in ChatGPT

Evaluate Codex skills and plugins from chat with a beginner-friendly start command, local-first reports, token budget explanations, and guided benchmarking. Skills for evaluating plugins and skills, improving skills and designing metric packs, with local-first reports and token budget explanations. Listed in OpenAI's curated Codex marketplace (openai-curated); the manifest declares skills.

Also available as

MCP App Basics (Apps SDK example)Eleven interactive widgets that each teach one MCP Apps APIapp · by OpenAI · 2.4k starsOpenAI Docs MCPSearch and read OpenAI's developer documentation from Codex or any MCP clientMCP server · by OpenAI · Remote

Install

  1. 1Add it in ChatGPT

    Open Plugins in the ChatGPT sidebar (web or desktop), search for “Plugin Eval” and select +. You'll be asked to sign in to any connected service on first use. Then type @ in a chat to call it.

  2. 2Or install it in Codex
    codex plugin add plugin-eval@openai-curated

    Start a new Codex session afterwards. You can also run /plugins inside the CLI to browse and install interactively.

Skills 5

  • Evaluate Plugin

    Evaluate a local Codex plugin

  • Evaluate Skill

    Evaluate a local Codex skill

  • Improve Skill

    Turn plugin-eval findings into a rewrite brief

  • Metric Pack Designer

    Design custom metric packs for plugin-eval

  • Plugin Eval

    Choose the right local evaluation workflow

Things to ask

  • “Give me an analysis of the game studio plugin.”
  • “Evaluate this plugin.”
  • “Why did this score that way?”

Learn more

How to package and publish a ChatGPT pluginPackage skills and an MCP server as a plugin with plugin.json, test it from a local marketplace, then upload, pass review and publish it to the directory.Advanced 60 minFor buildersWrite an Agent Skill for ChatGPT and CodexWrite a skill with SKILL.md, scripts and references, load it in Codex, test when it triggers, and bundle it in a plugin so it also works in ChatGPT.Beginner 20 minFor builders

Related

SuperpowersPlan, develop, and debug codeplugin · by Jesse Vincent · 294k starsCodex SecuritySecurity scanning for your codebaseplugin · by OpenAI · 7.2k starsBuild Web AppsBuild frontend-focused web apps with generated assets, browser testing, payments, and...plugin · by OpenAI · 7.2k starsGitHubTriage PRs, issues, CI, and publish flowsplugin · by OpenAI · 7.2k starsFigmaFigma design-to-code workflowsplugin · by Figma · 7.2k starsShopifyBuild and manage your storeplugin · by Shopify · 7.2k stars