Use HtmlKit tokenizer for HTML5 parsing - #243
Conversation
e76e74f to
0bfe73e
Compare
|
Just FYI, I've added some regression testing, so we can detect what this change "breaks" or rather what has already been broken in the existing samples. As such you can expect a bunch of failures that have likely nothing to do with your contribution. 😉 |
|
After these first 2 PRs get merged, I plan on bringing over the relevant parts of the regression suite PeachPDF has. It's up to around 7,000 tests and verifies all sorts of HTML, CSS, and SVG spec conformance things. Not all of the tests are useful, and there's some missing tests for GDI and WPF specific codepaths, but its a good initial corpus. |
|
Oh that sounds awesome! I just quickly hacked something together, so we have at least some baseline for the HtmlKit and ExCSS PRs beyond the manual testing. |
0bfe73e to
0a6d568
Compare
|
I noticed that non-breaking spaces weren't handled anymore with the removal of |
|
@eXpl0it3r I just found a similar issue in PeachPDF -- let me see about porting it over. See: jhaygood86/PeachPDF#431. The fix was to "do nothing" since HtmlKit already decoded everything correctly. |
|
@eXpl0it3r Looks like your change is more or less equivalent for now, so I'll just leave it. |
|
Fixed another issue where consecutive line break tags (e.g. A lot of the other "regressions" are just from invalid HTML (e.g. |
|
That's one that is fixed in PeachPDF already in the "fix up" phase. When I do the next stage, I'll make sure that all of those fixes come in. There's quite a bit of "take this parsed and cascaded document and make it actually be structured correctly" work that HTML-Renderer kind of only does |
Replaces the hand-rolled HTML scanner with HtmlKit tokenization and updates DOM parsing to better follow HTML5 behavior, including omitted end-tag handling (`p`/`td`/`tr`), safer comment/script handling, and `noscript` re-parsing. Adds anonymous table box correction logic and related DOM/CSS utilities, migrates text storage from `SubString` to `string` (removing `SubString.cs`), and includes a new demo sample showcasing malformed-HTML rendering improvements.
Improve the HTML5 parsing demo snippet by expanding the inline `if` into a multi-line block and refining the demo string text. The escaped source example and live script output were updated together to stay consistent and easier to read.
Delete the `InternalsVisibleTo` entry for `HtmlParserVerify` from `HtmlRenderer.csproj`, narrowing assembly internals access and keeping project metadata aligned with current test/verification setup.
c88f600 to
7a70b46
Compare
2 participants
This is the first of many pull requests to bring in the various HTML and CSS fixes from PeachPDF (which is a stripped down fork) to upstream. The goal is as each change gets merged in, I'll start work on the next change. The eventual goal will be parity where possible for features. There are many cases where that won't be possible, especially on the PDF side, but where it is possible, I'll make sure it works.
Background:
PeachPDF replaced HTML 5 parsing with using the tokenizer in MimeKit. We also use various other functionalities from MimeKit, so we bring in the full library. HTML-Renderer just needs the tokenization, so I used HtmlKit. If any of the other changes that requiring MimeKit gets ported in, then it would make sense to use MimeKit. The only difference is a namespace import and package import. They are the same codebase from the same maintainer.
The reason for this is that the hand-rolled regex based parser is not compliant with HTML 5 parsing rules at all. There are some related changes in the DOM layer that were brought in as well to make it spec compliant.
Changes:
p/td/tr), safer comment/script handling, andnoscriptre-parsing.SubStringtostring(removingSubString.cs)