Commit Graph

334 Commits

Author SHA1 Message Date
Paul Stemmet f6f7102a16 lib/scanner: add InvalidAnchorName error variant 2021-06-06 10:54:28 +01:00
Paul Stemmet 8ce016f320 lib/scanner: clippy lints 2021-06-06 08:44:15 +01:00
Paul Stemmet 5240cb7330 lib/scanner: scan tag directives
+ add unit tests for tag directives
2021-06-06 08:44:15 +01:00
Paul Stemmet a66664dd27 lib/scanner: mv cow! to normal macros, add check!
- cow! is useful for non test
- check! is a macro for determining if a buffer has the given byte at
  the given pos (default 0)
- also add an advance! variant that returns the removed slice
2021-06-06 08:44:15 +01:00
Paul Stemmet e852509921 lib/scanner: add InvalidTagHandle, InvalidTagPrefix variants 2021-06-06 08:44:15 +01:00
Paul Stemmet 2596a79e3f lib/scanner: improve version directive parsing
- handle multibyte version numbers correctly
- only advance Scanner.buffer on Token boundaries
- Unit tests for the above items
2021-06-06 08:44:15 +01:00
Paul Stemmet 8c453afcb1 lib/scanner: add UnexpectedEOF error variant 2021-06-06 08:44:15 +01:00
Paul Stemmet 3a93cb006a lib/scanner: add tokens! > variant for matching Results 2021-06-06 08:44:15 +01:00
Paul Stemmet 2768e5c86c lib/deps: add atoi
for fast slice -> ascii conversions
2021-06-06 08:44:15 +01:00
Paul Stemmet 3348cd8fed lib/scanner: clippy lints 2021-06-06 08:44:15 +01:00
Paul Stemmet ea21c0824f lib/scanner: add support for version directive
- refactor common buffer advance pattern into a macro, advance!
- fixup tokens! macro to work with scanner returning results
- unit test for version directive
2021-06-06 08:44:15 +01:00
Paul Stemmet be4c6ad52c lib/token: fix mixup in directive variants
I misunderstood, Version should have (u8, u8) for the major and minor
versions, while Tag contains the borrowed content
2021-06-06 08:44:15 +01:00
Paul Stemmet 7beda497d0 lib/scanner: add error::ScanError
In preparation for moving the scanner iterator to return
Result<Token, ScanError>
2021-06-06 08:44:15 +01:00
Paul Stemmet 2146f55f28 style/fmt: apply rustfmt rules 2021-06-05 10:17:19 +01:00
Paul Stemmet 8dc9d10dfe style/fmt: add .rustfmt to root 2021-06-05 10:17:19 +01:00
Paul Stemmet 832ef5eb33 lib/reader: comments 2021-06-03 17:52:32 +01:00
Paul Stemmet a09c983d56 lib/reader: fix column @newline 2021-06-03 17:52:32 +01:00
Paul Stemmet 2e3562c28e lib/reader: fmt 2021-06-03 17:52:32 +01:00
Paul Stemmet 95e1edd194 lib/reader: add unit tests 2021-06-03 17:52:32 +01:00
Paul Stemmet 7c8b0cdc43 lib/reader: first go at a reader impl
We need to track our column and mark how far through we've read at the
very least.
2021-06-03 17:52:32 +01:00
Paul Stemmet b541afbf91 lib/error: use anyhow as an error stub 2021-06-03 17:52:32 +01:00
Paul Stemmet 85c9a9f0c5 lib/scanner: improve eat_whitespace to chomp comments 2021-06-02 19:34:12 +01:00
Paul Stemmet 68cc3f7342 lib/scanner: improve tokens! macro error reporting 2021-06-02 19:34:12 +01:00
Paul Stemmet 9ecea201c8 lib/scanner: add fn for chomping whitespace between tokens 2021-06-02 16:09:24 +01:00
Paul Stemmet bee55887f0
ci/prtasks: use install over mv 2021-06-02 14:52:43 +00:00
Paul Stemmet 7594379488
ci/prtasks: fix sccache version, improve curl output 2021-06-02 14:44:47 +00:00
Paul Stemmet d4e13dd167
ci: move actions to repo root 2021-06-02 14:39:52 +00:00
Paul Stemmet 5cdf7bfc10
ci/prtasks: add actions for PR unit testing 2021-06-02 14:34:25 +00:00
Paul Stemmet e90176db4b
add Cargo.lock file 2021-06-02 14:33:51 +00:00
Paul Stemmet 3e2135ce79
remove .lock from ignores 2021-06-02 14:33:37 +00:00
Paul Stemmet dee8a69b71
lib/mod: add scanner + token 2021-06-01 19:10:44 +00:00
Paul Stemmet bdc70312f1
scanner: add Scanner struct + tokens! macro
The scanner is the engine that will drive the underlying byte stream,
converting it to tokens. This struct will not attempt to impose semantic
restrictions on token ordering, that will be for high level data
structures.
2021-06-01 19:07:47 +00:00
Paul Stemmet f4608a8588
token: add Token enum of possible tokens emitted 2021-06-01 19:04:29 +00:00
Paul Stemmet 847cf54400
Genesis 2021-06-01 13:31:00 +00:00