Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 14 additions & 14 deletions .claude/commands/docs-lint.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,21 @@
# Linting Documentation Notebooks
# Linting Documentation Python Snippets

Lint documentation notebooks for syntax and type errors without executing them using `docs/_scripts/lint_notebooks.py`.
Lint Markdown and MDX Python fenced code blocks for syntax and type errors without executing them using `docs/_scripts/lint_python_snippets.py`.

## Basic Usage

```sh
uv run python docs/_scripts/lint_notebooks.py <path>
uv run python docs/_scripts/lint_python_snippets.py <path>
```

Examples:

```sh
# Lint a single notebook
uv run python docs/_scripts/lint_notebooks.py docs/run-inference/tutorials/deploy-llm-nims.md
# Lint a single doc page
uv run python docs/_scripts/lint_python_snippets.py docs/run-inference/tutorials/deploy-llm-nims.md

# Lint all notebooks in a directory
uv run python docs/_scripts/lint_notebooks.py docs/run-inference/
# Lint all Markdown/MDX pages in a directory
uv run python docs/_scripts/lint_python_snippets.py docs/run-inference/
```

### Running in Cursor Agent Sandbox Mode
Expand All @@ -24,10 +24,10 @@ When running through the Cursor agent in sandbox mode, the agent will set `UV_CA

## Type Checking

Add `--type-check` to run `ty` type checker on the combined notebook cells:
Add `--type-check` to run `ty` type checker on the combined snippets:

```sh
uv run python docs/_scripts/lint_notebooks.py docs/run-inference/ --type-check
uv run python docs/_scripts/lint_python_snippets.py docs/run-inference/ --type-check
```

This catches:
Expand All @@ -45,10 +45,10 @@ Note: `ty` is alpha software and reports some false positives for SDK attributes
## How It Works

The script:
1. Finds notebooks with the `@nemo-nb: process` marker
2. Extracts all Python code cells
3. Combines them into a single file (so cross-cell context works)
4. Runs `ty check` and passes through the output
1. Finds Markdown and MDX files under the requested paths
2. Extracts all `python` and `py` fenced code blocks
3. Combines snippets from each page into a single file so earlier snippets can define later context
4. Runs `ty check` and maps diagnostics back to the original doc lines

## Fixing Linter Errors

Expand Down Expand Up @@ -92,4 +92,4 @@ The SDK's gateway methods return `object` type, causing false positives when acc

## Markers

Only notebooks with the `@nemo-nb: process` marker will be linted.
All Markdown and MDX pages under the requested paths are scanned. Use `<!-- @nemo-docs: skip-python-snippet-check -->` before a block to skip it.
14 changes: 8 additions & 6 deletions .claude/commands/docs-test.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,14 @@
# Testing Documentation Notebooks

You can test documentation from the repository root using `docs/_scripts/run_notebooks.py`, which executes notebooks marked with `@nemo-nb: process`.
You can test documentation from the repository root using `docs/fern/scripts/run_notebooks.py`, which executes notebooks marked with `@nemo-nb: process`.

## Basic Usage

The script takes a file path or directory path:

uv run python docs/_scripts/run_notebooks.py <input_path>
```sh
uv run python docs/fern/scripts/run_notebooks.py <input_path>
```

Environment Variables
The script automatically loads environment variables from a .env file in the repository root (if present).
Expand All @@ -22,20 +24,20 @@ NGC_API_KEY=your-ngc-key-here
You can also override environment variables inline:

```sh
NMP_BASE_URL=http://custom-url:8080 uv run python docs/_scripts/run_notebooks.py docs/run-inference/
NMP_BASE_URL=http://custom-url:8080 uv run python docs/fern/scripts/run_notebooks.py docs/run-inference/
```

## Language filters

```sh
# Run only Python cells (default)
uv run python docs/_scripts/run_notebooks.py docs/run-inference/ --language python
uv run python docs/fern/scripts/run_notebooks.py docs/run-inference/ --language python

# Run only shell cells
uv run python docs/_scripts/run_notebooks.py docs/run-inference/ --language shell
uv run python docs/fern/scripts/run_notebooks.py docs/run-inference/ --language shell
```


## Markers

Only notebooks with the @nemo-nb: process marker will be executed.
Only notebooks with the `@nemo-nb: process` marker will be executed.
16 changes: 16 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ endif
PYTEST_EXTRA ?=
PYTHON_VERSION ?= 3.11
BOOTSTRAP_CREATE_VENV ?= 1
BOOTSTRAP_EXPECTED_VIRTUAL_ENV := $(CURDIR)/.venv
BOOTSTRAP_ACTIVATION_REMINDER = if [ "$${VIRTUAL_ENV:-}" != "$(BOOTSTRAP_EXPECTED_VIRTUAL_ENV)" ]; then echo ""; echo "Next steps:"; echo " source .venv/bin/activate"; echo " nemo --help"; fi

# Display platform info
$(info local system architecture: $(PLATFORM)/$(ARCH))
Expand Down Expand Up @@ -120,6 +122,16 @@ docs-watch: ## Start Fern docs dev plus a repo-level watcher for docs/** changes
docs-check: ## Validate the Fern docs (fern check + validate-mdx + gated-link check)
cd docs/fern && npm run check

.PHONY: docs-check-python-snippets
docs-check-python-snippets: ## Syntax-check and type-check Python snippets in one doc (DOCS_PATH=...)
@if [ -z "$(strip $(DOCS_PATH))" ]; then echo "Usage: make docs-check-python-snippets DOCS_PATH=docs/customizer/tutorials/import-hf-model.mdx" >&2; exit 2; fi
uv run --frozen python docs/_scripts/lint_python_snippets.py "$(DOCS_PATH)"

.PHONY: docs-run-notebook
docs-run-notebook: ## Execute one Fern notebook source (DOCS_PATH=.mdx/.ipynb/.md, optional ARGS=...)
@if [ -z "$(strip $(DOCS_PATH))" ]; then echo "Usage: make docs-run-notebook DOCS_PATH=docs/customizer/tutorials/sft-customization-job.mdx" >&2; exit 2; fi
uv run --frozen python docs/fern/scripts/run_notebooks.py $(ARGS) "$(DOCS_PATH)"

.PHONY: docs-broken-links
docs-broken-links: ## Report broken links across the built docs
cd docs/fern && npm run broken-links
Expand Down Expand Up @@ -179,6 +191,9 @@ bootstrap-python: ## Bootstrap Python dependencies.
@if [ -n "$(strip $(BOOTSTRAP_LOCAL_PLUGIN_DIRS))" ]; then \
$(MAKE) bootstrap-plugins BOOTSTRAP_LOCAL_PLUGIN_DIRS="$(BOOTSTRAP_LOCAL_PLUGIN_DIRS)"; \
fi
@if [ "$(filter bootstrap-python,$(MAKECMDGOALS))" = "bootstrap-python" ]; then \
$(BOOTSTRAP_ACTIVATION_REMINDER); \
fi

.PHONY: verify-node-version
verify-node-version: ## Verify pnpm and Node.js satisfy Studio's package engine
Expand Down Expand Up @@ -222,6 +237,7 @@ bootstrap: bootstrap-python ## Bootstrap the local dev environment, including St
echo " make bootstrap-studio"; \
fi
@echo "bootstrap completed"
@$(BOOTSTRAP_ACTIVATION_REMINDER)

.PHONY: run
run: build-policy ## Run the NeMo Platform locally with Docker job backend
Expand Down
1 change: 1 addition & 0 deletions docs/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ _build
conf.py

# temporary ipynb files
*.tmp.md
*.tmp.ipynb

# base64 content
Expand Down
2 changes: 2 additions & 0 deletions docs/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ Run these from the repo root (they wrap `cd docs/fern && npm run …`):
| `make docs` | Local dev server (live preview) |
| `make docs-watch` | Local dev server plus repo-level watcher for `docs/**` changes outside `docs/fern/` |
| `make docs-check` | `fern check` + MDX validation + gated-link check (what CI runs) |
| `make docs-check-python-snippets DOCS_PATH=...` | Syntax-check and type-check Python fenced snippets in one doc |
| `make docs-run-notebook DOCS_PATH=...` | Execute the source notebook for one Fern `.mdx`/`.ipynb` doc using `nemo-nb` markers |
| `make docs-broken-links` | Report broken links |
| `make docs-fix-links` | Auto-delink references into gated pages |

Expand Down
Loading
Loading