While the previous commit did add support for _adding_ zero indented
sequences to the token stream, it unfortunately relied on the indent
stack flush that happens once reaching end of stream to push the stored
BlockEnd tokens.
This commit adds better support for removing zero indented sequences
from the stack once finished.
The heuristic used here is:
A zero_indented BlockSequence starts when:
- The top stored indent is for a BlockMapping
- A BlockEntry occupies the same indentation level
And terminates when:
- The top indent stored is a BlockSequence & is tagged as zero indented
- A BlockEntry _does not_ occupy the same indentation level
This fixes the edge case YAML allows where a sequence may be zero
indented, but still start a sequence.
E.g using the following YAML:
key:
- "one"
- "two"
The following tokens would have been produced (before this commit):
StreamStart
BlockMappingStart
Key
Scalar('key')
Value
BlockEntry
Scalar('one')
BlockEntry
Scalar('two')
BlockEnd
StreamEnd
Note the the lack of any indication that the values are in a sequence.
Post commit, the following is produced:
StreamStart
BlockMappingStart
Key
Scalar('key')
Value
BlockSequenceStart <--
BlockEntry
Scalar('one')
BlockEntry
Scalar('two')
BlockEnd <--
BlockEnd
StreamEnd
Paul Stemmet