Skip to content

Python: [BREAKING] Refactor FileSkillsSource for depth-based discovery and predicate filters#6488

Merged
giles17 merged 7 commits into
microsoft:mainfrom
giles17:giles17/refactor-file-skills-depth-based-discovery
Jun 24, 2026
Merged

Python: [BREAKING] Refactor FileSkillsSource for depth-based discovery and predicate filters#6488
giles17 merged 7 commits into
microsoft:mainfrom
giles17:giles17/refactor-file-skills-depth-based-discovery

Conversation

@giles17

@giles17 giles17 commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Motivation and Context

Ports the refactoring from .NET PR #6109 to the Python side. The old directory-whitelist approach (resource_directories/script_directories) was rigid — users had to know exact subdirectory names upfront. The new depth-based scanning with optional filter predicates is strictly more flexible and aligns both language implementations.

Description

Refactors FileSkillsSource to replace preconfigured directory whitelists with depth-controlled recursive scanning and user-supplied filter predicates.

Removed:

  • resource_directories / script_directories constructor options

Added:

  • search_depth option (>= 1, default 2): controls how deep the recursive scan goes within each skill directory. Depth 1 = root only; depth 2 = root + one level of subdirectories.
  • script_filter / resource_filter predicate options that receive a FileSkillFilterContext (skill_name + relative_file_path), allowing whitelist/blacklist filtering by file path.
  • FileSkillFilterContext class exported from agent_framework.

Preserved:

  • All security checks (path containment, symlink detection) using the skill root as the trusted boundary.
  • Extension-based filtering (resource_extensions / script_extensions).

Updated:

  • SkillsProvider.from_paths() accepts the new parameters.
  • Discovery methods refactored from @staticmethod to instance methods.
  • Tests rewritten to validate depth-based behavior and filter predicates.

Closes #6089

Contribution Checklist

  • The code builds clean without any errors or warnings
  • The PR follows the Contribution Guidelines
  • All unit tests pass, and I have added new tests where possible
  • Is this a breaking change? If yes, add [BREAKING] prefix to the title of the PR.

…y and predicate filters

Refactors FileSkillsSource to make script and resource discovery more flexible.

## Changes

- **Drops** resource_directories / script_directories options (preconfigured
  directory whitelists).
- **Adds** search_depth option (>= 1, default 2): controls how deep the
  recursive scan goes within each skill directory.
- **Adds** script_filter / resource_filter predicate options that receive a
  FileSkillFilterContext (skill_name + relative_file_path), allowing
  whitelist/blacklist filtering by file path.
- **Adds** FileSkillFilterContext class exported from agent_framework.

## Notes

- The Skills API is marked @experimental -- the option removals are intentional
  breaking changes within the experimental surface.
- Security checks (path containment, symlink detection) are preserved and
  continue to use the skill root directory as the trusted boundary.
- Ports the same refactoring from .NET PR microsoft#6109 while following Python
  conventions (instance methods, Callable type hints, __slots__).

Co-authored-by: Copilot <[email protected]>
Copilot AI review requested due to automatic review settings June 11, 2026 22:04
@moonbox3 moonbox3 added the python Usage: [Issues, PRs], Target: Python label Jun 11, 2026
@github-actions

github-actions Bot commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Python Test Coverage

Python Test Coverage Report •
FileStmtsMissCoverMissing
packages/core/agent_framework
   _skills.py10144096%294, 541, 1007, 1022, 1024–1025, 1381–1382, 1394–1395, 1624, 1653, 2121, 2569–2570, 2667, 2675, 2680, 2683, 2688, 2708, 2720, 2725, 2821, 2829, 2834, 2837, 2842, 2862, 2871, 2876, 3137–3138, 3487, 3714–3715, 3742–3743, 3750–3751
TOTAL42143498488% 

Python Unit Test Overview

Tests Skipped Failures Errors Time
8290 37 💤 0 ❌ 0 🔥 2m 6s ⏱️

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refactors Python file-based skill discovery to match the newer .NET behavior by replacing fixed subdirectory whitelists with depth-based recursive scanning and optional predicate filters, while keeping existing containment/symlink safety checks.

Changes:

  • Replace resource_directories / script_directories with search_depth (default 2) for recursive discovery within each skill directory.
  • Add script_filter / resource_filter predicates using the new exported FileSkillFilterContext to allow per-file inclusion/exclusion decisions.
  • Rewrite/extend unit tests to validate depth behavior, filtering, and existing safety guarantees (path traversal + symlink skipping).

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File Description
python/packages/core/tests/core/test_skills.py Updates tests to validate depth-based discovery and filter predicate behavior.
python/packages/core/agent_framework/_tools.py Minor refactor in approval-request state handling (no functional change intended).
python/packages/core/agent_framework/_skills.py Implements search_depth + predicate filters for FileSkillsSource, adds FileSkillFilterContext, updates SkillsProvider.from_paths() surface.
python/packages/core/agent_framework/init.py Exports FileSkillFilterContext from the public package surface.

Comment thread python/packages/core/agent_framework/_skills.py

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated Code Review

Reviewers: 5 | Confidence: 89%

✓ Correctness

The refactoring from directory-whitelist to depth-based scanning with filter predicates is well-implemented and correct. The depth logic correctly maps search_depth=1 to root-only, search_depth=2 to root+one-level, etc. Security checks (path containment, symlink detection) are preserved at all recursion levels. Filter predicates are applied after security checks, and the FileSkillFilterContext validation is sound. The from_paths factory method correctly forwards new parameters. No correctness bugs found.

✓ Security Reliability

The refactoring correctly preserves all security invariants (path containment, symlink detection at both directory and file levels). The depth-based recursion applies directory-level security checks before descending and file-level checks before including any resource. One minor reliability concern: unhandled exceptions from user-provided filter predicates would crash the entire skill discovery process rather than gracefully skipping the affected file.

✓ Test Coverage

Test coverage for the refactored FileSkillsSource is generally strong. Depth-based discovery is well-tested for resources at both unit and integration levels. Filter predicates are tested for both resources and scripts. However, there's an asymmetry: resource_filter has a dedicated context-verification test (test_resource_filter_receives_correct_context) confirming that skill_name and relative_file_path are passed correctly, but script_filter lacks an equivalent test. Similarly, from_paths has a passthrough test for resource_filter and search_depth but not for script_filter. These are low-severity gaps since the code paths are structurally identical.

✓ Failure Modes

The refactoring from directory-whitelists to depth-based recursive scanning is well-implemented from a failure-mode perspective. Error handling for OS-level failures (iterdir OSError), security checks (symlinks, path traversal), and input validation (search_depth >= 1) are all preserved or improved. User-provided filter predicates follow standard Python fail-fast callback semantics. No silent failures, lost errors, or partial-write scenarios were identified.

✗ Design Approach

The refactor mostly hangs together, but the new recursive file discovery no longer preserves skill boundaries when one skill directory contains another nested skill directory. Because nested SKILL.md folders are still discovered as separate skills elsewhere, the current approach can silently attach a child skill’s resources/scripts to its parent as well, which is a design regression worth fixing before merge.

Flagged Issues

  • Nested skill directories are treated as separate skills by _discover_skill_directories (lines 3161-3195), but the new recursive resource/script scanners descend into every subdirectory (lines 2796-2807 and 2945-2956) without checking for a nested SKILL.md. With parent/SKILL.md and parent/child/SKILL.md, the child's files will be surfaced under both skills. The scaners should stop recursing into subdirectories that contain their own SKILL.md.

Automated review by giles17's agents

Comment thread python/packages/core/agent_framework/_skills.py
@github-actions

Copy link
Copy Markdown
Contributor

Flagged issue

Nested skill directories are treated as separate skills by _discover_skill_directories (lines 3161-3195), but the new recursive resource/script scanners descend into every subdirectory (lines 2796-2807 and 2945-2956) without checking for a nested SKILL.md. With parent/SKILL.md and parent/child/SKILL.md, the child's files will be surfaced under both skills. The scaners should stop recursing into subdirectories that contain their own SKILL.md.


Source: automated DevFlow PR review

@giles17 giles17 changed the title Python: [Breaking] Refactor FileSkillsSource for depth-based discovery and predicate filters Python: [BREAKING] Refactor FileSkillsSource for depth-based discovery and predicate filters Jun 12, 2026
…rectories

- Add clarifying comments distinguishing MAX_SEARCH_DEPTH (SKILL.md
  discovery) from DEFAULT_SEARCH_DEPTH (per-skill resource/script scanning).
- Stop recursing into subdirectories that contain their own SKILL.md,
  preventing child skill files from being attached to the parent skill.
- Add test verifying nested skill boundary is respected.

Co-authored-by: Copilot <[email protected]>
@giles17 giles17 enabled auto-merge June 13, 2026 02:33
Comment thread python/packages/core/agent_framework/_skills.py Outdated
@Ashutosh0x

Copy link
Copy Markdown

Review: FileSkillsSource Refactor

Great refactor @giles17! Depth-based discovery and predicate filters are a much cleaner API surface than the current approach.

Observations

  1. Depth-based discovery: This solves the recursive scanning problem elegantly. Does the depth parameter default to 1 (current directory only) or unlimited? A sensible default of 1-2 levels would prevent accidental scanning of deep node_modules-style trees.

  2. Predicate filters: Love this — it enables use cases like 'load only Python skills' or 'exclude test skills'. Consider adding a few built-in predicates as static helpers:

    • FileSkillsSource.ByLanguage('python')
    • FileSkillsSource.ExcludePattern('test_*')
  3. Breaking change handling: Since this is BREAKING, is there a migration path for existing users of the old API? A deprecation warning in the current version pointing to the new API would ease the transition.

Offer to Help

I'm actively contributing to the hosting/channels layer (PR #6642). Happy to help with:

  • Writing tests for edge cases (empty directories, symlink loops, permission errors)
  • Documenting the migration path from old to new API
  • Validating the discovery against the hosted agent skill loading pattern

@moonbox3 moonbox3 added .NET Usage: [Issues, PRs], Target: .Net breaking change Usage: [PRs], Target: all PRs that introduce changes that are not backward compatible labels Jun 22, 2026
@github-actions github-actions Bot changed the title Python: [BREAKING] Refactor FileSkillsSource for depth-based discovery and predicate filters .NET: Python: [BREAKING] Refactor FileSkillsSource for depth-based discovery and predicate filters Jun 22, 2026
@giles17 giles17 force-pushed the giles17/refactor-file-skills-depth-based-discovery branch from 9de6829 to 9532465 Compare June 22, 2026 17:47
@giles17 giles17 removed the .NET Usage: [Issues, PRs], Target: .Net label Jun 22, 2026
@giles17 giles17 changed the title .NET: Python: [BREAKING] Refactor FileSkillsSource for depth-based discovery and predicate filters Python: [BREAKING] Refactor FileSkillsSource for depth-based discovery and predicate filters Jun 22, 2026
…ents

- Remove __slots__ from FileSkillFilterContext per reviewer feedback —
  the optimization is negligible and inconsistent with sibling classes.
- Add type: ignore[attr-defined] / ty: ignore[unresolved-attribute]
  comments to test lines accessing private _resources/_scripts attributes,
  matching the convention established on main.

Co-authored-by: Copilot <[email protected]>
Comment thread python/packages/core/agent_framework/_skills.py Outdated
Comment thread python/packages/core/agent_framework/_skills.py Outdated
Comment thread python/packages/core/agent_framework/_skills.py Outdated
…le[[str, str], bool]

Address reviewer feedback:
- Remove FileSkillFilterContext class — a dedicated class for two strings
  is overkill in Python. Filters now receive (skill_name, relative_file_path)
  directly as positional args.
- Update docstrings to describe behavior instead of referencing private
  instance attributes.
- Remove FileSkillFilterContext from exports and __all__.
- Update all test lambdas and remove TestFileSkillFilterContext class.

Co-authored-by: Copilot <[email protected]>
Comment thread python/packages/core/agent_framework/_skills.py Outdated
Instead of accepting int | None and resolving None to the default
internally, use DEFAULT_SEARCH_DEPTH as the parameter default value
on both FileSkillsSource.__init__() and SkillsProvider.from_paths().

Co-authored-by: Copilot <[email protected]>
@giles17 giles17 added this pull request to the merge queue Jun 24, 2026
Merged via the queue into microsoft:main with commit 9f1ee23 Jun 24, 2026
37 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking change Usage: [PRs], Target: all PRs that introduce changes that are not backward compatible python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python: Enhance skill resource and script discovery

7 participants