Skip to content

AGENTS.md

Latest commit

 

History

History
196 lines (144 loc) · 10.3 KB

AGENTS.md

File metadata and controls

196 lines (144 loc) · 10.3 KB
Raw

Apify Documentation Standards

Project overview

Docusaurus-based documentation site combining multiple content types:

  • Platform docs (sources/platform/) - Product documentation
  • Academy (sources/academy/) - Educational courses and tutorials
  • API reference (apify-api/) - Generated from OpenAPI specs
  • Multi-repo architecture - SDK/Client docs from separate repos served via nginx

One unified site built from multiple repositories. See CONTRIBUTING.md for multi-repo setup.

Commands

pnpm install            # Install dependencies (runs patch-package via postinstall)
pnpm start              # Dev server (rebuilds API docs, port 3000)
pnpm build              # Production build (catches broken links, bad frontmatter)
pnpm lint               # Run all linters (markdownlint + oxlint)
pnpm lint:md            # Markdownlint only
pnpm lint:code          # oxlint only
pnpm lint:fix           # Auto-fix both linters
vale sync               # Download Vale styles (first time only)
vale "path/to/file.md" --minAlertLevel=error  # Prose style check
pnpm api:rebuild        # Regenerate API docs from OpenAPI specs
pnpm openapi:lint       # Validate OpenAPI spec (Redocly + Spectral + YAML)

Architecture

OpenAPI documentation pipeline

API docs are generated, NOT hand-written. The workflow:

  1. Source of truth: apify-api/openapi/openapi.yaml (splits into /paths and /components)
  2. Redocly plugins (apify-api/plugins/apify.mjs) inject custom behavior:
    • code-samples-decorator.mjs - Auto-adds code samples if files exist in /code_samples/{js,curl}/
    • legacy-doc-url-decorator.mjs - Adds backward-compatible URLs
    • client-references-links-decorator.mjs - Links to client library docs
  3. Build command: pnpm api:rebuild = clean + bundle with Redocly + generate with Docusaurus
  4. Output: Markdown files in apify-api/docs/ (gitignored, regenerated on each build)

Never edit generated API docs directly. Edit the OpenAPI YAML source or add code samples.

OpenAPI code samples

Add code samples by creating files in apify-api/openapi/code_samples/{javascript,curl}/:

  • Filename must match operationId from OpenAPI spec (e.g., actorRun_get.js)
  • Decorator auto-detects and adds x-codeSamples property
  • Missing samples are logged during build

OpenAPI specification changes

  • Target OpenAPI specification version should be extracted from /openapi/openapi.yaml. All specification changes should be compliant with syntax for that specific OpenAPI specification version.
  • Prefer re-use of existing objects via $ref over duplication. Reusable components can be found in /openapi/components.
  • Components most suitable for re-use are:
    • Request parameters and path parameters defined in /openapi/components/parameters
    • Request/response schemas defined in /openapi/components/schemas
    • Explicit non-automatic examples defined in /openapi/components/examples
    • Objects that are not standardized in the OpenAPI specification defined in /openapi/components/objects
  • Objects that exist in several variants with only minor differences across different files can be extracted into file in /openapi/components/objects. Within one file the YAML anchor syntax can be used to define a shared and unique portion of such objects and avoid some code duplication. These objects will be used only during bundling process through references, but they will not be standalone entities in the generated specification.
  • When changing files in /openapi/paths look for opportunities to extract shared duplicate objects into re-usable components saved in /openapi/components.
  • When adding new endpoints, check first if any existing path is similar and if yes, try to re-use same components. If by adding new paths you create new duplication, try to extract it into a new components and reference it instead.
  • Prefer automatically generated examples from schema over explicit examples.

Error responses

  • Re-use schemas for error responses defined in /apify-api/openapi/components/responses
  • Each endpoint should have at least following error responses: 400 (Bad Request), 405 (Method Not Allowed), 429 (Too Many Requests).
  • Endpoints that define security: [] do not use any authentication.
  • Each endpoint that uses authentication should have at least following error responses: 401 (Unauthorized), 403 (Forbidden).
  • Each endpoint that has runs/last in its path or that has any ID related parameter (for example actorId, buildId, runId, datasetId and so on) should have at least one 404 (Not Found) error.
  • Each endpoint that has requestBody should have at least following error responses: 413 (Payload Too Large), 415 (Unsupported Media Type).

Syntax hints

  • Instead of using one item enum, use const: Avoid this:
schema:
  type: string
  enum:
    - "constantValue"

Use this:

schema:
  type: string
  const: "constantValue"

Python API client model generation

apify-client-python generates its Pydantic models from the published spec, and it pulls them on its own schedule - this repo does not push anything to it:

  1. This repo (.github/workflows/openapi-ci.yaml): on a PR touching apify-api/openapi/**, lints, builds, and validates the bundled spec. Once merged and deployed, the bundle is served at https://docs.apify.com/api/openapi.json.

  2. apify-client-python (.github/workflows/on_schedule_regenerate_models.yaml): nightly at 02:00 UTC, downloads the spec from that URL, regenerates the models, and opens a PR when they change. Only the spec's version is recorded on that side, not the spec itself.

A spec change therefore reaches the Python client within a day of being deployed, with no coordination needed on this side. Nothing here needs to be merged in lockstep with a client PR.

This used to be a cross-repo dispatch that opened a companion client PR per docs PR. It was removed: the client can only generate from the published spec anyway, and having two mechanisms write the same generated files made them diverge.

Theme system

Uses @apify/docs-theme package - a shared theme across all 6+ documentation repos. Don't modify theme files directly. Changes to the theme propagate via CI to all projects.

LLMs.txt generation

Post-build scripts (scripts/joinLlmsFiles.mjs + indentLlmsFile.mjs) combine llms.txt files from all sections into root. Edit source llms.txt files in content directories, not the generated root one.

Slug collection

tools/utils/collectSlugs.js scans directories for slug: in frontmatter, used in docusaurus.config.js for dynamic navigation with activeBaseRegex. Ensure slugs are collected when adding new sections.

Multi-repo ecosystem

Repository Port Content
apify-docs (this repo) 3000 Platform, Academy, OpenAPI
apify-client-js 3001 JavaScript client docs
apify-client-python 3002 Python client docs
apify-sdk-js 3003 JavaScript SDK docs
apify-sdk-python 3004 Python SDK docs
apify-cli 3005 CLI documentation

Use pnpm start:dev + nginx to serve all repos together locally. See CONTRIBUTING.md for setup.

Deployment

  • Auto-deploy on merge to master
  • Preview builds on pull requests
  • PR titles must use Conventional Commits format (docs:, fix:, feat:, etc.) - enforced by CI
  • Keep PR descriptions short - one or two sentences covering what changed and why. Skip boilerplate headings (## Summary, ## Changes, ## Details), bullet lists that restate the diff, and filler text. The diff is the record of what changed; the description explains the why.

Common pitfalls

  1. Editing generated API docs - Always edit OpenAPI YAML source, never generated markdown in apify-api/docs/
  2. Broken links on build - onBrokenLinks: 'throw' fails CI. Check slugs match file paths
  3. Missing frontmatter - Description or slug errors break SEO and navigation
  4. Missing code block language - Always specify language for syntax highlighting
  5. Stale API docs locally - Run pnpm api:rebuild after changing OpenAPI specs

Quick reference

  • Add new doc: Create .md in sources/{platform,academy}/, add frontmatter with title/description/slug
  • Add API endpoint: Edit apify-api/openapi/paths/**/*.yaml, add code samples, run pnpm api:rebuild
  • Fix broken build: Check onBrokenLinks errors, verify slugs match file paths, validate frontmatter

Standards

Detailed writing and formatting standards are in standards/:

  • standards/writing-style.md - Prose voice, tone, headings, links, numbers
  • standards/content-standards.md - Front matter, admonitions, code blocks, images
  • standards/terminology.md - Product names, capitalization, article usage
  • standards/grammar-rules.md - Hyphenation, punctuation, numbers, brand spelling
  • standards/file-organization.md - File naming and directory structure
  • standards/quality-standards.md - Complete quality checklist before submitting

Key rules at a glance:

  • US English, active voice, imperative tone, no sales language
  • Sentence case headings, no gerunds
  • Bold for UI elements only; code for filenames, commands, variables
  • All admonitions require titles
  • 140-160 character descriptions in front matter
  • See standards/terminology.md for Apify product name capitalization
  • Don't use em dashes (—) - use hyphen with spaces ( - ) instead

Skills

Documentation skills live in .agents/skills/ (AgentSkills spec), each with its own references/ and scripts/:

  • .agents/skills/review-docs/ - Documentation review process and output format
  • .agents/skills/doc-write/ - Writing and editing documentation pages
  • .agents/skills/tutorial/ - Creating structured tutorials
  • .agents/skills/api-doc/ - OpenAPI specification and API documentation

Review checklist

When creating or reviewing documentation, verify:

  • Sentence case headings, no gerunds, proper hierarchy
  • Front matter complete (title, description 140-160 chars, sidebar_position, slug)
  • Bold used only for UI elements
  • All admonitions have titles
  • Code examples are complete with syntax highlighting
  • Links use descriptive text, internal links use relative paths
  • Images have alt text, use light theme
  • Terminology matches rules above
  • US English, active voice, no sales language
  • pnpm lint passes