GitHub Copilot Names Custom Tools in Usage Reports: What Small Teams Can Learn

According to GitHub’s announcement, customer-defined items previously grouped under other or custom can now appear by name. The affected arrays are totals_by_skill, totals_by_custom_agent, totals_by_mcp, totals_by_slash_cmd, and totals_by_plugin. This covers skills, custom agents, MCP servers, slash commands, and plugins.

Each totals_by_* list now includes up to 25 entries per record, increased from five. Remaining activity is combined into a trailing other entry. GitHub says interaction_count and distinct_*_use_count calculations remain unchanged. The update therefore adds reporting detail without announcing a new counting method.

Our interpretation is that this helps teams ask more specific questions. A named entry can point to an internal tool worth investigating. It does not explain whether the tool completed its task correctly, whether developers liked it, or whether repeated interactions reflected difficulty.

GitHub lists enterprise and organization reports, including per-user one-day and 28-day reports, aggregated one-day reports, and day_totals within aggregated 28-day reports. Its REST API documentation describes daily report generation and endpoints that return download links. These are processed reports, so a same-session spot check is not a reliable expectation.

Client coverage needs equal attention. The announcement includes Copilot CLI interactive sessions and prompt mode, the GitHub Copilot app, Agent Client Protocol integrations such as JetBrains, and the Copilot SDK. Prompt mode, the app, and ACP integrations require Copilot CLI 1.0.97 or later. The SDK requirement is version 1.0.21 or later.

IDE agent experiences report names when they run on Copilot SDK 1.0.21 or later. An IDE’s built-in chat telemetry does not report names. Teams should identify the actual experience producing activity before comparing named entries across developers.

GitHub says names appear starting with report days processed after this release. Earlier days retain other or custom. That wording does not establish an exact activity-time cutoff, and it does not promise that historical reports will gain names.

This matters when reviewing a 28-day window spanning the change. A newly visible name may represent an existing tool that has become identifiable. Its appearance alone cannot establish that adoption began on that day. Similarly, a decline in a generic bucket may reflect more detailed labeling.

Our proposed approach is to record when each customization first becomes visible and which client versions are represented. Build subsequent comparisons from periods with comparable coverage. Keep any earlier generic totals as context, but avoid assigning that historical activity to a particular named tool without separate evidence.

The announcement says names are reported as configured, trimmed and limited to 128 characters. Names resembling file paths remain grouped under other or custom. Combined with the list limit, that means an absent name has several possible explanations beyond nonuse.

GitHub also warns that client telemetry can undercount activity, including when clients are outdated or telemetry is blocked. It recommends using the metrics for trends and adoption rather than exact accounting. We have not independently measured the size or frequency of these gaps.

For editorial purposes, the useful distinction is between visibility and effectiveness. A report can help locate activity to investigate. Assessing the resulting code still requires task evidence. Our AI code review workflow explains how to examine changes and verification separately from how frequently an assistant or customization was used.

For a small team already able to view organization reports, we propose a narrow pilot: choose one internal customization and ask whether teammates continue using it after an introduction. This is a suggested evaluation, not a test we performed.

Record its configured name, intended task, relevant client experience, and version. After a complete report day has been processed, inspect the applicable list for the name. If it is absent, examine coverage, naming rules, and the generic bucket before concluding that nobody used it. Repeat over comparable reporting periods.

Pair the report with a brief task review. Ask what developers attempted, whether the output was usable, and what correction remained necessary. Our coding-assistant evaluation guide offers task-based criteria for that discussion. Usage trends can guide the questions; review evidence should determine whether the workflow deserves continued attention.

This update is most relevant to teams maintaining several internal customizations and struggling to distinguish their use. GitHub identifies report access for enterprise owners and billing managers, organization owners, and custom roles with the relevant metrics permission. Individual Copilot use alone should not be treated as proof of organization-report access.

Report access also requires the Copilot usage metrics policy to be enabled. GitHub’s REST documentation specifies Enabled everywhere for the enterprise; confirm that prerequisite with the responsible administrator before expecting reports.

Our recommendation is to use the added names to prioritize investigation: which tool needs clearer instructions, which deserves a task review, and which appears to have limited visible adoption. Avoid ranking developers or declaring a tool successful from interaction counts alone. The release supplies more specific labels, while the judgment about usefulness still depends on the work those tools help produce.