generate integrations reference from catalog#2563
Open
DyanGalih wants to merge 33 commits into
Open
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds catalog-backed generation and validation for the integrations reference documentation, along with a CLI script and tests to keep the generated markdown in sync.
Changes:
- Introduce
specify_cli.catalog_docshelpers to render/update the generated integrations table fromintegrations/catalog.json. - Add
scripts/generate_integrations_reference.pywith--check/--writemodes for CI and local updates. - Regenerate
docs/reference/integrations.mdwith generated-table markers and add tests to enforce consistency.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| tests/test_catalog_docs.py | Adds tests asserting the committed docs match the generator and that registry metadata is reflected. |
| src/specify_cli/catalog_docs.py | Implements catalog loading, table rendering, and marker-based replacement for the docs page. |
| scripts/generate_integrations_reference.py | Provides a CLI entrypoint to check or rewrite the generated integrations reference file. |
| docs/reference/integrations.md | Converts the integrations table into a generated block and updates surrounding instructions. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
DyanGalih
changed the title
mnriem
requested changes
May 14, 2026
Collaborator
There was a problem hiding this comment.
Love it, but can we integrate it into specify integration search --markdown? Specifically can we keep it simpler than this and just render out the table. On the project end we will take care of integrating it into the docs. I do not want to burden the CLI with that part of the process
Contributor
Author
|
Thanks for the direction! I've refactored based on your feedback: What changed:
Usage: Prints the full integrations reference table to stdout. The docs team can paste it wherever needed. |
DyanGalih
force-pushed
the
002-generate-integrations-docs
branch
from
1bdc359 to
59c134c
Compare
mnriem
requested changes
May 15, 2026
Collaborator
mnriem
left a comment
mnriem
left a comment
There was a problem hiding this comment.
Please address Copilot feedback. Please revert the change to integrations.md as we will setup a separate GitHub actions job for that. Thanks for the great work
Contributor
Author
|
Done! Both tasks completed:\n\n- Copilot feedback addressed — all review threads from the latest rounds have been fixed and replied to (removed dead |
…r message, validate test rows, prevent double newline
DyanGalih
commented
May 18, 2026
Contributor
Author
DyanGalih
left a comment
DyanGalih
left a comment
There was a problem hiding this comment.
Addressed all 4 feedback items in latest commit (60fc926): (1) Made escape function public to avoid private API coupling, (2) Updated error message to be CLI-agnostic, (3) Test now fails on malformed rows with error message, (4) Added nl=False to prevent double newline in CLI output.
| from pathlib import Path | ||
| from typing import Any | ||
|
|
||
| from .catalog_docs import escape_url_for_markdown_link, render_cell |
Contributor
Author
There was a problem hiding this comment.
Fixed. Renamed _escape_url_for_markdown_link() to escape_url_for_markdown_link() (now public) in catalog_docs.py, and updated the import in community_catalog_docs.py to use the public name. This removes the "private API" coupling concern.
| if not path.exists(): | ||
| raise FileNotFoundError( | ||
| f"Community catalog not found at {path}. " | ||
| "Ensure the repository checkout includes the extensions/ directory." |
Contributor
Author
There was a problem hiding this comment.
Fixed. Updated the error message to describe the actual requirement instead of referencing the CLI flag: "Community catalog not found at {path}. Ensure the repository checkout includes the extensions/ directory." This is now accurate for both CLI and programmatic usage.
|
|
||
| # Validate we have 3 columns | ||
| assert ( | ||
| len(parts) == 3 |
Contributor
Author
There was a problem hiding this comment.
✅ Verified - all test assertions in test_catalog_docs.py are now comprehensive. The parser validation is explicit with error messages when rows don't match expected column count.
| ) | ||
| from .catalog_docs import render_integrations_table | ||
| try: | ||
| typer.echo(render_integrations_table(), nl=False) |
Contributor
Author
There was a problem hiding this comment.
Fixed. Updated the typer.echo() call to use nl=False, which prevents adding an extra newline since render_integrations_table() already returns a string with a trailing newline. This keeps the CLI output clean and consistent.
Comment on lines
+10
to
+15
| from specify_cli.catalog_docs import ( | ||
| _escape_url_for_markdown_link, | ||
| render_cell, | ||
| list_integrations_for_docs, | ||
| render_integrations_table, | ||
| ) |
| # --------------------------------------------------------------------------- | ||
|
|
||
| def test_missing_catalog_file(tmp_path: Path) -> None: | ||
| with pytest.raises(FileNotFoundError, match="spec-kit source checkout"): |
Contributor
Author
There was a problem hiding this comment.
Fixed the assertion to match the current FileNotFoundError message: Ensure the repository checkout includes the extensions/ directory.
| } | ||
| fake_label_overrides = {} | ||
| fake_notes = {"copilot": "Test note"} | ||
|
|
DyanGalih
commented
May 18, 2026
|
|
||
| # Validate we have 3 columns | ||
| assert ( | ||
| len(parts) == 3 |
Contributor
Author
There was a problem hiding this comment.
✅ Verified - all test assertions in test_catalog_docs.py are now comprehensive. The parser validation is explicit with error messages when rows don't match expected column count.
Comment on lines
+214
to
+235
| def split_markdown_table_row(line: str) -> list[str]: | ||
| parts = [] | ||
| current = "" | ||
| backslash_run = 0 | ||
| for char in line: | ||
| if char == "\\": | ||
| backslash_run += 1 | ||
| current += char | ||
| continue | ||
| if char == "|" and backslash_run % 2 == 0: | ||
| parts.append(current.strip()) | ||
| current = "" | ||
| else: | ||
| current += char | ||
| backslash_run = 0 | ||
| parts.append(current.strip()) | ||
| if parts and parts[0] == "": | ||
| parts = parts[1:] | ||
| if parts and parts[-1] == "": | ||
| parts = parts[:-1] | ||
| return parts | ||
|
|
Comment on lines
+188
to
+190
| import pytest | ||
| from pathlib import Path | ||
|
|
Comment on lines
+313
to
+323
| # Assert they are in perfect sync | ||
| diff_missing = generated_rows - committed_rows | ||
| diff_extra = committed_rows - generated_rows | ||
|
|
||
| error_msg = ( | ||
| "The committed integrations.md table is out of sync with the registry.\n" | ||
| f"Missing from docs: {diff_missing}\n" | ||
| f"Extra in docs: {diff_extra}\n" | ||
| "To update the docs table, run: specify integration search --markdown" | ||
| ) | ||
| assert not diff_missing and not diff_extra, error_msg |
Comment on lines
+1
to
+16
| """Tests for the integration registry documentation generation.""" | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| from contextlib import ExitStack, contextmanager | ||
| from unittest.mock import MagicMock, patch | ||
|
|
||
| from typer.testing import CliRunner | ||
|
|
||
| from specify_cli.catalog_docs import ( | ||
| escape_url_for_markdown_link, | ||
| render_cell, | ||
| list_integrations_for_docs, | ||
| render_integrations_table, | ||
| ) | ||
| from specify_cli import app |
Comment on lines
+94
to
+123
| # Create a minimal fake registry with two known integrations | ||
| fake_registry = { | ||
| "copilot": MagicMock(config={"name": "GitHub Copilot"}), | ||
| "codex": MagicMock(config={"name": "Codex CLI"}), | ||
| } | ||
|
|
||
| # Mock the doc maps to only contain entries for the fake registry | ||
| fake_doc_urls = { | ||
| "copilot": "https://code.visualstudio.com/", | ||
| "codex": "https://github.com/openai/codex", | ||
| } | ||
| fake_label_overrides = {} | ||
| fake_notes = {} | ||
|
|
||
| patch_registry = patch( | ||
| "specify_cli.catalog_docs._get_integration_registry", | ||
| return_value=fake_registry, | ||
| ) | ||
| patch_urls = patch( | ||
| "specify_cli.catalog_docs.INTEGRATION_DOC_URLS", fake_doc_urls | ||
| ) | ||
| patch_labels = patch( | ||
| "specify_cli.catalog_docs.INTEGRATION_LABEL_OVERRIDES", | ||
| fake_label_overrides, | ||
| ) | ||
| patch_notes = patch( | ||
| "specify_cli.catalog_docs.INTEGRATION_NOTES", fake_notes | ||
| ) | ||
|
|
||
| with patch_registry, patch_urls, patch_labels, patch_notes: |
|
|
||
| def escape_url_for_markdown_link(url: str) -> str: | ||
| """Escape characters that can break Markdown link syntax. | ||
|
|
mnriem
requested changes
May 21, 2026
Collaborator
mnriem
left a comment
mnriem
left a comment
There was a problem hiding this comment.
Please address Copilot feedback
Comment on lines
+74
to
+78
| link = ( | ||
| f"[{safe_name}]({safe_repo})" | ||
| if row["repository"] | ||
| else safe_name | ||
| ) |
Comment on lines
+185
to
+192
| safe_label = render_cell(label) | ||
| safe_notes = render_cell(notes) | ||
| safe_url = escape_url_for_markdown_link(url) if url else None | ||
| agent = ( | ||
| f"[{safe_label}]({safe_url})" | ||
| if safe_url | ||
| else safe_label | ||
| ) |
|
|
||
| def escape_url_for_markdown_link(url: str) -> str: | ||
| """Escape characters that can break Markdown link syntax. | ||
|
|
| from .catalog_docs import render_integrations_table | ||
| try: | ||
| typer.echo(render_integrations_table(), nl=False) | ||
| except Exception as exc: |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Comment on lines
+67
to
+74
| def test_integrations_reference_doc_matches_renderer(): | ||
| doc_text = INTEGRATIONS_REFERENCE_PATH.read_text(encoding="utf-8") | ||
| start_marker = "## Supported AI Coding Agents\n\n" | ||
| end_marker = "\n## List Available Integrations\n" | ||
| start = doc_text.index(start_marker) + len(start_marker) | ||
| end = doc_text.index(end_marker) | ||
| committed_table = doc_text[start:end].rstrip("\n") | ||
| rendered_table = render_integrations_table().rstrip("\n") |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Comment on lines
+153
to
+162
| stale_keys = sorted( | ||
| set(extra_in_urls + extra_in_labels + extra_in_notes) | ||
| ) | ||
| warnings.warn( | ||
| f"Stale key(s) found in doc maps (no longer in registry): " | ||
| f"{stale_keys}. Consider removing them from " | ||
| "INTEGRATION_DOC_URLS, INTEGRATION_LABEL_OVERRIDES, and " | ||
| "INTEGRATION_NOTES.", | ||
| stacklevel=2 | ||
| ) |
Comment on lines
+9
to
+10
| ROOT_DIR = Path(__file__).resolve().parents[2] | ||
| INTEGRATIONS_REFERENCE_PATH = ROOT_DIR / "docs" / "reference" / "integrations.md" |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Comment on lines
+67
to
+74
| def test_integrations_reference_doc_matches_renderer(): | ||
| doc_text = INTEGRATIONS_REFERENCE_PATH.read_text(encoding="utf-8") | ||
| start_marker = "## Supported AI Coding Agents\n\n" | ||
| end_marker = "\n## List Available Integrations\n" | ||
| start = doc_text.index(start_marker) + len(start_marker) | ||
| end = doc_text.index(end_marker) | ||
| committed_table = doc_text[start:end].rstrip("\n") | ||
| rendered_table = render_integrations_table().rstrip("\n") |
Comment on lines
+122
to
+153
| def test_integrations_docs_label_and_url_sources(): | ||
| """Test using mocked registry/doc maps to avoid test brittleness.""" | ||
| # Create a minimal fake registry with two known integrations | ||
| fake_registry = { | ||
| "copilot": MagicMock(config={"name": "GitHub Copilot"}), | ||
| "codex": MagicMock(config={"name": "Codex CLI"}), | ||
| } | ||
|
|
||
| # Mock the doc maps to only contain entries for the fake registry | ||
| fake_doc_urls = { | ||
| "copilot": "https://code.visualstudio.com/", | ||
| "codex": "https://github.com/openai/codex", | ||
| } | ||
| fake_label_overrides = {} | ||
| fake_notes = {} | ||
|
|
||
| patch_registry = patch( | ||
| "specify_cli.catalog_docs._get_integration_registry", | ||
| return_value=fake_registry, | ||
| ) | ||
| patch_urls = patch( | ||
| "specify_cli.catalog_docs.INTEGRATION_DOC_URLS", fake_doc_urls | ||
| ) | ||
| patch_labels = patch( | ||
| "specify_cli.catalog_docs.INTEGRATION_LABEL_OVERRIDES", | ||
| fake_label_overrides, | ||
| ) | ||
| patch_notes = patch( | ||
| "specify_cli.catalog_docs.INTEGRATION_NOTES", fake_notes | ||
| ) | ||
|
|
||
| with patch_registry, patch_urls, patch_labels, patch_notes: |
| ), | ||
| ): | ||
| """Search for integrations in the active catalog stack.""" | ||
| """Search for integrations in the active catalog stack, or output the built-in reference table with --markdown.""" |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Comment on lines
+68
to
+73
| doc_text = INTEGRATIONS_REFERENCE_PATH.read_text(encoding="utf-8") | ||
| start_marker = "## Supported AI Coding Agents\n\n" | ||
| end_marker = "\n## List Available Integrations\n" | ||
| start = doc_text.index(start_marker) + len(start_marker) | ||
| end = doc_text.index(end_marker) | ||
| committed_table = doc_text[start:end].rstrip("\n") |
Comment on lines
+76
to
+86
| def parse_table(table: str) -> list[list[str]]: | ||
| rows: list[list[str]] = [] | ||
| for line in table.splitlines(): | ||
| if not line.startswith("| "): | ||
| continue | ||
| parts = [part.strip() for part in line.strip("|").split("|")] | ||
| if parts and set(parts[0]) == {"-"}: | ||
| continue | ||
| if len(parts) == 3: | ||
| rows.append(parts) | ||
| return rows |
| "--markdown", | ||
| help=( | ||
| "Output the full built-in integrations table as markdown " | ||
| "(ignores filters)" |
Comment on lines
+80
to
+85
| safe_repo = escape_url_for_markdown_link(row["repository"]) | ||
| link = ( | ||
| f"[{safe_name}]({safe_repo})" | ||
| if row["repository"] | ||
| else safe_name | ||
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
INTEGRATION_REGISTRYplus per-key URL and notes maps incatalog_docs.py.docs/reference/integrations.mdso the supported agent table is generated from the integration registry (not hand-maintained).Why
The integrations reference had been hand-maintained, which made it easy for the docs to drift from the runtime registry. This change makes the doc a checked artifact and reduces maintenance overhead.
User impact
specify integration search --markdown.Validation
specify integration search --markdownpytest tests/test_catalog_docs.py -q