Feature/directive/handle url encoding #12

Merged

bazaah merged 6 commits from feature/directive/handle-url-encoding into master

2021-06-27 16:35:03 +00:00

bazaah commented

2021-06-27 16:31:15 +00:00

(Migrated from github.com)

This PR allows the scanner to properly consume percent encoded UTF8 in tag directives. It also makes a change to the Scanner.next_token return, moving from a Token<'a> to a Ref<'borrow, 'copy>, allowing the scanner to copy into the given scratch space and return a Token that borrows from it.

Finally, it includes a unit test for tag directive percent encoding.

This PR allows the scanner to properly consume percent encoded UTF8 in tag directives. It also makes a change to the `Scanner.next_token` return, moving from a `Token<'a>` to a `Ref<'borrow, 'copy>`, allowing the scanner to copy into the given `scratch` space and return a `Token` that borrows from it. Finally, it includes a unit test for tag directive percent encoding.