Research Human Security Review in the Copilot App With Stop Conditions

Short summary

Proposes a research protocol for studying how humans review AI-generated security guidance in GitHub Copilot, focusing on evidence-backed decisions rather than acceptance rates. Defines scenarios with seeded true findings, plausible distractions, and critical omissions across diverse reviewer profiles. Specifies individual and study-level stop conditions to prevent unsafe actions during simulation sessions.

• •Research question is whether reviewers build evidence-backed decisions, not whether they click Accept
• •Scenarios seed true issues, plausible distractions, and critical omissions with varied mixes
• •Stop conditions cover credentials, live repos, and participants mistaking simulation for production

Generated with AI, which can make mistakes.