Skip to content

fix(docs): render the Markdown SDK references as plain Markdown, not MDX - #22037

Merged
gewenyu99 merged 15 commits into
masterfrom
sdk-markdown-reference-plain
Nov 11, 2026
Merged

gewenyu99 merged 15 commits into
masterfrom
sdk-markdown-reference-plain

Conversation

@gewenyu99

@gewenyu99 gewenyu99 commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Changes

#21971 writes each Markdown SDK reference from Strapi to contents/docs/libraries/<sdk>/<page>.md. Gatsby then compiles that file as MDX, and MDX can run code at build time. A Strapi row must not run code in our build. This PR stops these pages from going through MDX. They render as plain Markdown.

  • gatsby-config.js blocks the pages from gatsby-plugin-mdx with shouldBlockNodeFromTransformation. The AgentSkill SKILL.md files use the same option.
  • gatsby/onCreateNode.ts makes an SdkMarkdownReference node for each page.
  • src/templates/sdk/SdkMarkdownReference.tsx renders the page with react-markdown and rehype-sanitize, as the community posts do. Raw HTML such as <details> stays. Scripts, event handlers, and javascript: links go.
  • Search indexes the latest SDK reference pages, through gatsby/algoliaConfig.js. Past and next majors, and the migration guides, stay out of search, as they stay out of the sitemap.
  • gatsby/createPages.ts makes the SDK reference pages before the minimal build returns, so Vercel previews have them.
  • The template parses the Markdown again only when the body changes, and renders nothing for a code block that the sanitizer leaves empty. A narrow window doesn't repeat the long section list above the page, because the Jump to panel has it.
  • A heading never gets the ID posthog. A browser makes an element ID a global when no script defines it, so that ID replaces window.posthog when posthog-js doesn't load, and the page crashes. The JavaScript reference has such a heading. It gets the ID posthog-heading, and the page's links and table of contents use the new ID.
  • Search reads the first 150,000 characters of a reference, which runs to about 400 KB, twice the largest page indexed today.

The pages keep the docs layout, the table of contents, and their .md copies for llms.txt.

PR tour
  1. gatsby/sdkMarkdownReferences.js finds the SDK reference files, splits the front matter from the body, and says which pages answer noindex.
  2. gatsby-config.js blocks these files from MDX.
  3. gatsby/onCreateNode.ts makes an SdkMarkdownReference node from each file.
  4. gatsby/createSchemaCustomization.ts types the node.
  5. gatsby/createPages.ts makes one page for each node. Line 145 runs it before the minimal build returns, so preview deploys have the pages.
  6. src/templates/sdk/SdkMarkdownReference.tsx renders the page in ReaderView. Headings get the same IDs as MDX pages, so the table of contents links work. Code blocks keep their language.
  7. gatsby/onPostBuild.ts adds the pages to the .md copies and llms.txt.
  8. gatsby/algoliaConfig.js adds the pages to site search.
  9. gatsby/onPreBootstrap.ts uses the shared page check.
Reviewer's guide

Start at the template. Most of the review is the sanitize schema. It is the default schema, plus className on code for the language.

Test Result
Hostile page through the template's plugins: export, import, <script>, onerror, javascript: link, {process.env.SECRET} export and import show as text. The script, the handler, and the link target are removed. The expression stays as text.
Local pnpm build on an older master, with Python, Go, Rust, iOS, Android, and Flutter rows in a local Strapi Exit 0. Every SDK page and its .md copy is made.
Python 7.x.x page in the browser The nav and the table of contents show. Heading IDs match the table of contents. 0 scripts in the content.
Code block languages python, swift, kotlin, dart, go, and rust are found on their pages.
Strapi fails or hangs The build logs a warning and continues without the pages.
pnpm format Clean.

Not tested: a production build on the current master, and the Vercel preview. The older master did not have the renderMdx prop on ReaderView, so the local build used a local copy of that prop.

Screenshots

The Node.js reference, from a local build. A narrow window no longer lists every section above the page. The wide window doesn't change.

State Before After
Light, narrow window
Light, wide window
Dark, narrow window
Dark, wide window

Merge order

🤖 Generated with Claude Code

gewenyu99 and others added 9 commits November 4, 2026 16:23
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rence

Keeps the hand-written library page in place.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replaces the intake script, version selector and nav code with one small step
in onPreBootstrap. The SDK repo writes each page, front matter included.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…est out of search

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A major's page, such as Python 7, is written to 7.x.x.md, so its URL reads
/docs/libraries/python/7.x.x. The noindex rule matches the new URLs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A referenceId or version with a slash or leading dot could write outside
contents/docs/libraries. The intake now skips such a row.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Time out after 30 seconds and skip the pages on an error, so a Strapi outage doesn't fail or stall the build.
- Skip rows that have no referenceId or version.
- Write only reference, migration and N.x.x pages, so a row can't replace a hand-written page.
- Gitignore the written pages and remove the last build's pages first, so a page that Strapi drops doesn't stay.
- Leave the noindex pages out of the sitemap.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The SDK repos generate these pages, and Strapi stores them. The build must
not run code from a row, so the pages are never compiled as MDX:

- gatsby-config.js blocks them from gatsby-plugin-mdx with
  shouldBlockNodeFromTransformation, like the AgentSkill SKILL.md files.
- gatsby/onCreateNode.ts turns each page into an SdkMarkdownReference node.
- src/templates/sdk/SdkMarkdownReference.tsx renders the body with
  react-markdown and rehype-sanitize, like the community posts.

The pages keep the docs layout, the table of contents, and the .md copies
for llms.txt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@gewenyu99
gewenyu99 requested a review from a team as a code owner November 6, 2026 16:20
@github-actions github-actions Bot added the website About the website (beyond just landing pages) label Nov 6, 2026
@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Deploy preview

Status Details Updated (UTC)
🟢 Ready View preview Nov 11, 2026 12:20AM

@posthog posthog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PostHog Review · Standard · reviewed 7b879f0: 4 Should fix.

See this review and its findings in PostHog Review. Comment @posthog review --deep for a Deep review. A Deep review takes longer and looks deeper than a Standard review.

Comment thread gatsby-config.js
(node.relativeDirectory || '').includes('/skills/')) ||
(node.url && new URL(node.url).hostname === 'raw.githubusercontent.com')),
// SDK references from Strapi render as plain Markdown, never as MDX (gatsby/sdkMarkdownReferences.js).
isSdkReferenceFile(node) ||

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 · Include SDK Markdown nodes in the search index

When SDK references stop producing Mdx nodes, they disappear from site search. retrieveSearchPages in gatsby/algoliaConfig.js queries only allMdx, and its fallback accepts only components under src/pages/, excluding SdkMarkdownReference.tsx. Index SdkMarkdownReference nodes and join their titles, headings, and bodies to their published pages.

Comment thread gatsby/createPages.ts Outdated
Comment on lines +602 to +604
sdkMarkdownReferences.data.allSdkMarkdownReference.nodes.forEach(({ id, slug }) => {
createPage({ path: slug, component: SdkMarkdownReferenceTemplate, context: { id } })
})

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 · Create SDK Markdown pages in minimal builds

With GATSBY_MINIMAL=true, as configured in .github/workflows/deploy-preview.yml, createPages returns through createMinimalPages before reaching this block. That function only queries allMdx and legacy SDK nodes, so the newly blocked Markdown references have no pages and their preview URLs return 404. Run SDK Markdown page creation in both build paths.

Comment on lines +58 to +64
const renderMarkdown = (markdown: string) => (
// rehype-sanitize sanitizes the HTML that rehype-raw parses, so no raw HTML is trusted.
// nosemgrep: typescript.react.security.react-markdown-insecure-html.react-markdown-insecure-html
<ReactMarkdown
remarkPlugins={[remarkGfm]}
rehypePlugins={[rehypeRaw, [rehypeSanitize, schema], headingIds] as any}
components={components}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 · Memoize Markdown rendering by document body

Resizing an SDK reference window updates appWindow on every drag event, which rerenders ReaderViewContent and calls renderMarkdown again. Each call reparses the entire reference, parses HTML, sanitizes it, and generates heading IDs synchronously. Large references therefore add main-thread work throughout resizing; memoize the Markdown component by its body.

Comment on lines +40 to +44
const code = node.children[0] as any
const className = (code?.properties?.className || []).find((name) => name.startsWith('language-'))
return (
<SingleCodeBlock language={className?.slice('language-'.length) || ''}>
{text(code).trimEnd()}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 · Handle pre elements emptied by sanitization

A reference containing <pre><iframe></iframe></pre> leaves an empty pre after rehype-sanitize removes the iframe. The custom renderer reads node.children[0] and passes undefined to text, which throws when accessing node.type. This aborts Gatsby's HTML build instead of safely rendering the sanitized content; guard empty children before extracting text.

…ow-window header

The Jump to panel lists the sections, so a narrow window doesn't repeat
dozens of them above the page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- gatsby/algoliaConfig.js indexes the SdkMarkdownReference pages for site
  search, in the shape of the MDX pages. Past and next majors, and the
  migration guides, stay out of search like they stay out of the sitemap.
- gatsby/createPages.ts makes the SDK reference pages before the minimal
  build returns, so Vercel previews have them too.
- The template parses the Markdown again only when the body changes, and
  renders nothing for a code block the sanitizer left empty.
- scripts/posthog-loader.test.cjs stubs ./sdkMarkdownReferences, which its
  sandbox can't resolve from gatsby/.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot removed the website About the website (beyond just landing pages) label Nov 6, 2026
@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

AEO/SEO: I'm in danger

This PR may affect SEO/AEO

@natalia-amorim, please review the items below before this PR merges. If Natalia is out, ask Ian or Andy to review instead.

🚨 Likely to hurt search or AI visibility (2)

  • noindex or X-Robots-Tag added in gatsby-config.js
    This change can remove existing pages from Google and AI search: const { isNoindexSdkReference, isSdkReferenceFile } = require('./gatsby/sdkMarkdownReferences')
    Fix: Confirm the pages must leave search results. If not, remove the directive.
  • noindex or X-Robots-Tag added in gatsby/algoliaConfig.js
    This change can remove existing pages from Google and AI search: const { isNoindexSdkReference } = require('./sdkMarkdownReferences')
    Fix: Confirm the pages must leave search results. If not, remove the directive.
Minor (1)

Minor (1)

  • llms.txt or Markdown pages changed in gatsby/onPostBuild.ts
    AI agents read llms.txt and the .md version of each docs page. A break here makes our docs harder for assistants to find and quote.
    Fix: Check /llms.txt and a few .md URLs in the Vercel preview.

This comment updates on each push. The check reads the diff only, so it can miss problems or flag safe changes. If this is a false alarm, reply here so the check can improve.

@github-actions

github-actions Bot commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Bundle report

Total JS (gzip)

12.05 MiB (+0.3 KiB / +0.0%)

Eager graph (modules shipped in each entrypoint's initial chunks)

Entrypoint Eager size Budget Modules
✅ app 10.41 MiB (no change) report-only 1099
Largest modules in the app closure
Module Size
css ./node_modules/.pnpm/css-loader@5.2.7_webpack@5.101.3/node_modules/css-loader/dist/cjs.js??ruleSet[1].rules[8].oneOf[1].use[1]!./node_modules/.pnpm/postcss-loader@4.3.0_postcss@8.5.6_webpack@5.101.3/node_modules/postcss-loader/dist/cjs.js??ruleSet[1].rules[8].oneOf[1].use[2]!./src/styles/global.css 1021.7 KiB
./src/components/Stickers/Stickers.tsx 696.4 KiB
./node_modules/.pnpm/@posthog+icons@0.36.6_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/@posthog/icons/dist/posthog-icons.es.js 354.8 KiB
./node_modules/.pnpm/react-markdown@8.0.7_@types+react@16.14.66_react@18.3.1/node_modules/react-markdown/lib/react-markdown.js + 88 modules 351.4 KiB
./node_modules/.pnpm/framer-motion@14.0.0_react-dom@18.3.1_react@18.3.1__react@18.3.1/node_modules/framer-motion/dist/es/render/components/motion/proxy.mjs + 126 modules 287.2 KiB
./node_modules/.pnpm/cloudinary-core@2.14.0_lodash@4.17.21/node_modules/cloudinary-core/cloudinary-core.js 281.9 KiB
./src/components/SearchUI/index.tsx + 87 modules 273.1 KiB
./gatsby-browser.tsx + 59 modules 259.3 KiB
./node_modules/.pnpm/@posthog+brand@0.13.0_react@18.3.1/node_modules/@posthog/brand/dist/generated/hoggies/svg/construction-1.mjs 248.5 KiB
./src/components/Pricing/PricingSlider/Slider.tsx + 87 modules 240.2 KiB
./node_modules/.pnpm/@posthog+brand@0.13.0_react@18.3.1/node_modules/@posthog/brand/dist/generated/hoggies/svg/construction-2.mjs 228.2 KiB
./node_modules/.pnpm/@posthog+brand@0.13.0_react@18.3.1/node_modules/@posthog/brand/dist/generated/hoggies/svg/magnifying-glass.mjs 221.7 KiB
./src/hooks/useProducts.tsx + 23 modules 153.9 KiB
./src/hooks/skills.tsx + 2 modules 150.7 KiB
./src/hooks/useProduct.ts + 9 modules 149.8 KiB

Eager-graph budgets are report-only until a baseline is established. Sizes are gzip of public/**/*.js; eager size is webpack module source bytes for the modules actually shipped in the entrypoint's initial chunks (post-tree-shake).

…earch body

- gatsby/sdkMarkdownReferences.js has one isNoindexSdkReference, which the
  sitemap and site search both use, so the Python reference stays out of
  search until #21973 links it.
- Search reads the first 150,000 characters of a reference. A reference runs
  to about 400 KB, twice the largest page indexed today.
- The createPages comment names the preview deploys, not Vercel.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@smallbrownbike smallbrownbike moved this from New to In Progress in PostHog.com planning Nov 9, 2026
Base automatically changed from markdown-sdk-references to master November 10, 2026 20:08
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gewenyu99 and others added 2 commits November 10, 2026 18:50
The JavaScript reference has a heading with the ID `posthog`. A browser
makes an element ID a global when no script defines it, so when
posthog-js doesn't load, window.posthog is that heading and the route
update's capture call crashes the page. Give that heading the ID
`posthog-heading`, and move the page's links and table of contents to it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…emap

The master merge brought back a local copy of isNoindexSdkReference that
shadowed the shared one from gatsby/sdkMarkdownReferences.js.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@gewenyu99
gewenyu99 merged commit 86540bf into master Nov 11, 2026
22 checks passed
@gewenyu99
gewenyu99 deleted the sdk-markdown-reference-plain branch November 11, 2026 01:07
@github-project-automation github-project-automation Bot moved this from In Progress to Done in PostHog.com planning Nov 11, 2026
gewenyu99 added a commit that referenced this pull request Nov 12, 2026
master has #22037's final version, so keep it and reapply this branch's own changes on top.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gewenyu99 added a commit that referenced this pull request Nov 12, 2026
Picks up master, which has #22037's final version.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants