Learn From PR

Extracts lessons learned from a completed PR to improve repository documentation and agent capabilities.

Inputs

Outputs

1. Learning Analysis - Structured markdown with: - What happened (problem, attempts, solution) - Fix location analysis (attempted vs actual) - Failure modes identified - Prioritized recommendations

2. Actionable Recommendations - Each with: - Category, Priority, Location, Specific Change, Why It Helps

Completion Criteria

The skill is complete when you have: - [ ] Gathered PR diff and metadata - [ ] Analyzed fix location (attempted vs actual) - [ ] Identified failure modes - [ ] Generated at least one concrete recommendation - [ ] Presented findings to user

When to Use

• After agent failed to find the right fix
• After agent succeeded but took many attempts
• After agent succeeded quickly (to understand what worked)
• When asked "what can we learn from PR #XXXXX?"

When NOT to Use

• Before PR is finalized (use pr-finalize first)
• For trivial PRs (typo fixes, simple changes)
• When no agent was involved (nothing to analyze)

Workflow

Step 1: Gather Data

# Required: Get PR info
gh pr view XXXXX --json title,body,files
gh pr diff XXXXX

# Check for session markdown
ls .github/agent-pr-session/issue-XXXXX.md .github/agent-pr-session/pr-XXXXX.md 2>/dev/null

If session markdown exists, extract: - Fix Candidates table (what was tried) - Files each attempt targeted - Why attempts failed

Analyzing without session markdown:

When no session file exists, you can still learn from: 1. PR discussion - Comments reveal what was tried 2. Commit history - Multiple commits may show iteration 3. Code complexity - Non-obvious fixes suggest learning opportunities 4. Similar past issues - Search for related bugs

Focus on: "What would have helped an agent find this fix faster?"

Step 2: Fix Location Analysis

Critical question: Did agent attempts target the same files as the final fix?

# Where did final fix go?
gh pr view XXXXX --json files --jq '.files[].path' | grep -v test

# If session markdown exists, compare to attempted files

If different files: Answer these questions: - Why did agent think that was the right file? - What search would have found the correct file?

Step 3: Analyze Outcome

Determine which scenario applies and look for the relevant patterns:

Scenario A: Agent Failed

Scenario B: Agent Succeeded Slowly (many attempts)

Key question: What would have gotten agent to the solution faster?

Scenario C: Agent Succeeded Quickly

Key question: What made this work? Should we reinforce this pattern?

Step 4: Find Improvement Locations

# Discover where agent guidance lives
find .github/instructions -name "*.instructions.md" 2>/dev/null
find .github/skills -name "SKILL.md" 2>/dev/null
ls docs/design/ 2>/dev/null
ls .github/copilot-instructions.md 2>/dev/null

Step 5: Generate Recommendations

For each recommendation, provide:

1. Category: Instruction file / Skill / Architecture doc / Inline comment / Linting issue
2. Priority: High (prevents class of bugs) / Medium (helps discovery) / Low (nice to have)
3. Location: Exact file path
4. Specific Change: Exact text to add
5. Why It Helps: Which failure mode it prevents

Prioritization factors: - How common is this pattern? - Would future agents definitely hit this again? - How hard is it to implement?

Pattern-to-Improvement Mapping (Failures/Slow Success):

Pattern-to-Improvement Mapping (Quick Success - reinforce):

Step 6: Present Findings

Present your analysis covering: - What happened and what made it hard - Where agent looked vs actual fix location - Which patterns applied and evidence - Prioritized recommendations with full details (category, priority, location, exact change, why it helps)

Error Handling

Constraints

• Analysis only - Don't apply changes (use learn-from-pr agent for that)
• Actionable recommendations - Every recommendation must have specific file path and text
• Don't duplicate - Check existing docs before recommending new ones
• Focus on high-value learning - Skip trivial observations
• Respect PR scope - Don't recommend improvements unrelated to the PR's learnings

Examples

Example: Wrong File Entirely

PR #33352 - TraitCollectionDidChange crash on MacCatalyst

What happened: - Issue title: "ObjectDisposedException in ShellSectionRootRenderer" - Agent made 11 attempts, ALL in ShellSectionRootRenderer.cs - Actual fix was in PageViewController.cs

Failure Mode: Trusted issue title instead of searching for pattern.

Recommendation: - Category: Instruction File - Location: .github/instructions/ios-debugging.instructions.md - Change: "When fixing iOS crashes, search for the PATTERN across all files, not just the file named in the error" - Why: Prevents tunnel vision on named file

Example: Slow Success

PR #34567 - CollectionView scroll position not preserved

What happened: - Agent took 5 attempts to find fix - First 3 attempts were in wrong layer (handler vs core) - Eventually found it after reading more context - Final fix was simple once the right layer was identified

Pattern: Wrong abstraction layer - fixed handler when problem was in core.

Recommendation: - Category: Architecture Doc - Location: .github/architecture/handler-vs-core.md - Change: Document layer responsibilities - handlers map properties, core handles behavior - Why: Helps agent identify correct layer faster

Example: Quick Success

PR #35678 - Button disabled state not updating

What happened: - Agent found fix in 1 attempt - Searched for "IsEnabled" pattern across codebase immediately - Found similar past fix in another control and applied same approach - Simple, minimal change

Pattern: Good search strategy - recognized pattern from similar code.

Recommendation: - Category: Skill Enhancement - Location: .github/skills/try-fix/SKILL.md - Change: Add to search strategy: "Search for same property pattern in other controls" - Why: Reinforces successful discovery technique

Integration

• pr-finalize → Use first to verify PR is ready
• learn-from-pr skill → Analysis only (this skill)
• learn-from-pr agent → Analysis + apply changes

For automated application of recommendations, use the learn-from-pr agent instead.