lib/scanner: fix missing token production for zero indents
So, the underlying bug here is caused because we check for zero indents *before* unwinding the indentation stack. Take the following example: ``` 0|Map1: 1|- Map2: value 2| Map2.1: value 3|Map3: ... ---------- 0123456789 ``` At (1,0) we encounter to indentation level increases -- map, sequence -- but zero actual indentation increase until (1,3) ('M' in 'Map2'). So, when we arrive at (3,0), Scanner::unroll_indent/2 only unrolls one indent level... the second one (sequence) which contains the information we use to determine whether an indent level has a zero indent that needs to be added to the queue. Thus, when we hit the call to Scanner::pop_zero_indent_sequence/2 it looks at the current indent (map), which it correctly decides isn't zero indented. The fix is simple, just reorder the operations to check for zero indents before we unroll. That said, I probably should create some additional test cases to check that we handle these edge cases correctly. 1. Zero indent before and after real indent: (Z,N,Z) 2. Double zero indent -- not sure this is possible in YAML (Z,Z) Issue: #54
This commit is contained in:
parent
1dfa3def22
commit
835ceb0b97
|
@ -139,8 +139,8 @@ impl Scanner
|
||||||
self.expire_stale_saved_key()?;
|
self.expire_stale_saved_key()?;
|
||||||
|
|
||||||
// Handle indentation unrolling
|
// Handle indentation unrolling
|
||||||
self.pop_zero_indent_sequence(*base, tokens)?;
|
|
||||||
self.unroll_indent(tokens, self.stats.column)?;
|
self.unroll_indent(tokens, self.stats.column)?;
|
||||||
|
self.pop_zero_indent_sequence(*base, tokens)?;
|
||||||
|
|
||||||
// Is it the end of a stream?
|
// Is it the end of a stream?
|
||||||
if base.is_empty() || self.state == StreamState::Done
|
if base.is_empty() || self.state == StreamState::Done
|
||||||
|
|
Loading…
Reference in New Issue