KSES: Reimplement with Tag Processor

[dmsnell]

Trac ticket: Core-66208
Trac ticket: Core-65984

Replaces #6577

Description

Rewrites wp_kses() to rely on the HTML API for structural and reliable application of sanitization rules, normalizing the output for improved downstream parsing.

Notables

• A new filter wp_kses_force_legacy_parser provides the choice of whether to use this new parser or stick with the legacy code.

Fixes

• Core-25851 Large attribute values may crash PCRE patterns and cause content loss.
• Core-37698 Global pollution in wp_kses_split() calls.
• Core-48873 CSS contents are corrupted by wp_kses().
• Core-51482 wp_kses() turns SCRIPT and STYLE content into renderable text.
• Core-52333 wp_kses() and HTML disagree on the set of named character references.
• Core-58377 Block names with consecutive hyphens are corrupted.
• Core-58921 Valid tag names are rejected from the allow-list.
• Core-59310 parse_blocks() called unnecessarily.
• Core-61246 wp_kses() un-comments HTML comments.
• Core-62024 wp_kses_post() incorrectly escapes "<" attributes values.

Todo

Merge after #13273, which accounts for three of the failing tests.

• self-closing non-HTML elements
• [~] remove opening tag when required attributes are missing, and closing tag
  • while this would be a nice enhancement it’s going to be left out of this work to preserve existing behaviors. with the HTML Processor powering wp_kses(), it’s possible to simply wait until an opened element is closed based on depth, and skip that closing element if it exists.
• replace C0 controls
  • replace C0 controls with their escapes, rather than stripping them away
  • replace C0 controls in attribute values?
  • original commit removing C0 controls is c7fd8c7
  • C0 controls are left in-place to prevent problems with creating new syntax through their removal
• [~] handle incomplete parsing, including closing all open elements
  • plenty of existing code in Core calls wp_kses() with intentionally-incomplete input, for example, a wrapper opening tag with part of the content, separately from the closer. closing open elements does a good job of isolating content, but legacy behaviors depend too much on the more procedural use of wp_kses() so isolation cannot be reasonably added without mangling websites.
• if SVG or MATH are not allowed, the entire element should disappear
• remove default pre_kses filters but then call pre_kses
• [~] Change character reference handling for XML context (even though this function is not and was never appropriate for XML parsing — use serialize_to_xml() instead).
  • The legacy implementation doesn’t parse a different set of character reference names. Instead, for the xml context, it only leaves the five syntax characters as names, and decodes everything else. This is correlated with the default behavior for the new implementation for XML and HTML.
  • It shouldn’t need saying here, but wp_kses() is entirely inappropriate for XML inputs or XML outputs. That requires serialize_to_xml().

Notes

• Branch tip with HTML Processor bdbae3a

[github-actions]

Test using WordPress Playground

The changes in this pull request can previewed and tested using a WordPress Playground instance.

WordPress Playground is an experimental project that creates a full WordPress instance entirely within the browser.

Some things to be aware of

• All changes will be lost when closing a tab with a Playground instance.
• All changes will be lost when refreshing the page.
• A fresh instance is created each time the link below is clicked.
• Every time this pull request is updated, a new ZIP file containing all changes is created. If changes are not reflected in the Playground instance,
  it's possible that the most recent build failed, or has not completed. Check the list of workflow runs to be sure.

For more details about these limitations and more, check out the Limitations page in the WordPress Playground documentation.

Test this pull request with WordPress Playground.