rocksdb/table/block_based
Peter Dillinger ef443cead4 Refactor to avoid confusing "raw block" (#10408)
Summary:
We have a lot of confusing code because of mixed, sometimes
completely opposite uses of of the term "raw block" or "raw contents",
sometimes within the same source file. For example, in `BlockBasedTableBuilder`,
`raw_block_contents` and `raw_size` generally referred to uncompressed block
contents and size, while `WriteRawBlock` referred to writing a block that
is already compressed if it is going to be. Meanwhile, in
`BlockBasedTable`, `raw_block_contents` either referred to a (maybe
compressed) block with trailer, or a maybe compressed block maybe
without trailer. (Note: left as follow-up work to use C++ typing to
better sort out the various kinds of BlockContents.)

This change primarily tries to apply some consistent terminology around
the kinds of block representations, avoiding the unclear "raw". (Any
meaning of "raw" assumes some bias toward the storage layer or toward
the logical data layer.) Preferred terminology:

* **Serialized block** - bytes that go into storage. For block-based table
(usually the case) this includes the block trailer. WART: block `size` may or
may not include the trailer; need to be clear about whether it does or not.
* **Maybe compressed block** - like a serialized block, but without the
trailer (or no promise of including a trailer). Must be accompanied by a
CompressionType.
* **Uncompressed block** - "payload" bytes that are either stored with no
compression, used as input to compression function, or result of
decompression function.
* **Parsed block** - an in-memory form of a block in block cache, as it is
used by the table reader. Different C++ types are used depending on the
block type (see block_like_traits.h).

Other refactorings:
* Misc corrections/improvements of internal API comments
* Remove a few misleading / unhelpful / redundant comments.
* Use move semantics in some places to simplify contracts
* Use better parameter names to indicate which parameters are used for
outputs
* Remove some extraneous `extern`
* Various clean-ups to `CacheDumperImpl` (mostly unnecessary code)

Pull Request resolved: https://github.com/facebook/rocksdb/pull/10408

Test Plan: existing tests

Reviewed By: akankshamahajan15

Differential Revision: D38172617

Pulled By: pdillinger

fbshipit-source-id: ccb99299f324ac5ca46996d34c5089621a4f260c
2022-09-22 11:25:32 -07:00
..
binary_search_index_reader.cc Set Read rate limiter priority dynamically and pass it to FS (#9996) 2022-05-18 19:41:44 -07:00
binary_search_index_reader.h Extend Get/MultiGet deadline support to table open (#6982) 2020-06-29 14:53:17 -07:00
block.cc Add API for writing wide-column entities (#10242) 2022-06-25 15:30:47 -07:00
block.h Refactor to avoid confusing "raw block" (#10408) 2022-09-22 11:25:32 -07:00
block_based_table_builder.cc Refactor to avoid confusing "raw block" (#10408) 2022-09-22 11:25:32 -07:00
block_based_table_builder.h Refactor to avoid confusing "raw block" (#10408) 2022-09-22 11:25:32 -07:00
block_based_table_factory.cc Always verify SST unique IDs on SST file open (#10532) 2022-09-07 22:52:42 -07:00
block_based_table_factory.h Account memory of big memory users in BlockBasedTable in global memory limit (#9748) 2022-04-06 10:33:00 -07:00
block_based_table_iterator.cc Provide support for direct_reads with async_io (#10197) 2022-07-06 11:42:59 -07:00
block_based_table_iterator.h More testing w/prefix extractor, small refactor (#10122) 2022-06-16 16:41:25 -07:00
block_based_table_reader.cc Refactor to avoid confusing "raw block" (#10408) 2022-09-22 11:25:32 -07:00
block_based_table_reader.h Refactor to avoid confusing "raw block" (#10408) 2022-09-22 11:25:32 -07:00
block_based_table_reader_impl.h Updated NewDataBlockIterator to not fetch compression dict for non-da… (#10310) 2022-07-06 09:30:25 -07:00
block_based_table_reader_sync_and_async.h Refactor to avoid confusing "raw block" (#10408) 2022-09-22 11:25:32 -07:00
block_based_table_reader_test.cc Add support for wide-column point lookups (#10540) 2022-08-19 11:51:12 -07:00
block_builder.cc Improve data block construction performance (#9040) 2021-10-19 12:36:21 -07:00
block_builder.h Improve data block construction performance (#9040) 2021-10-19 12:36:21 -07:00
block_like_traits.h Revert "Avoid dynamic memory allocation on read path (#10453)" (#10541) 2022-08-19 11:02:54 -07:00
block_prefetcher.cc Fix stress test failure for async_io (#10660) 2022-09-12 14:48:06 -07:00
block_prefetcher.h Provide support for direct_reads with async_io (#10197) 2022-07-06 11:42:59 -07:00
block_prefix_index.cc Fix bug with kHashSearch and changing prefix_extractor with SetOptions (#10128) 2022-06-10 08:51:45 -07:00
block_prefix_index.h Fix bug with kHashSearch and changing prefix_extractor with SetOptions (#10128) 2022-06-10 08:51:45 -07:00
block_test.cc Improve / clean up meta block code & integrity (#9163) 2021-11-18 11:43:44 -08:00
block_type.h Remove deprecated block-based filter (#10184) 2022-06-16 15:51:33 -07:00
cachable_entry.h Refactor to avoid confusing "raw block" (#10408) 2022-09-22 11:25:32 -07:00
data_block_footer.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
data_block_footer.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
data_block_hash_index.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
data_block_hash_index.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
data_block_hash_index_test.cc Add support for wide-column point lookups (#10540) 2022-08-19 11:51:12 -07:00
filter_block.h Pass rate_limiter_priority through filter block reader functions to FS (#10251) 2022-06-24 16:13:44 -07:00
filter_block_reader_common.cc Pass rate_limiter_priority through filter block reader functions to FS (#10251) 2022-06-24 16:13:44 -07:00
filter_block_reader_common.h Pass rate_limiter_priority through filter block reader functions to FS (#10251) 2022-06-24 16:13:44 -07:00
filter_policy.cc Have Cache use Status::MemoryLimit (#10262) 2022-07-06 14:41:46 -07:00
filter_policy_internal.h Remove deprecated block-based filter (#10184) 2022-06-16 15:51:33 -07:00
flush_block_policy.cc Restore Regex support for ObjectLibrary::Register, rename new APIs to allow old one to be deprecated in the future (#9362) 2022-01-11 06:33:48 -08:00
flush_block_policy.h Make FlushBlockPolicyFactory into a Customizable class (#8432) 2021-07-12 09:04:59 -07:00
full_filter_block.cc Pass rate_limiter_priority through filter block reader functions to FS (#10251) 2022-06-24 16:13:44 -07:00
full_filter_block.h Pass rate_limiter_priority through filter block reader functions to FS (#10251) 2022-06-24 16:13:44 -07:00
full_filter_block_test.cc Pass rate_limiter_priority through filter block reader functions to FS (#10251) 2022-06-24 16:13:44 -07:00
hash_index_reader.cc Fix bug with kHashSearch and changing prefix_extractor with SetOptions (#10128) 2022-06-10 08:51:45 -07:00
hash_index_reader.h Extend Get/MultiGet deadline support to table open (#6982) 2020-06-29 14:53:17 -07:00
index_builder.cc Make InternalKeyComparator not configurable (#10342) 2022-07-14 10:09:31 -07:00
index_builder.h Make InternalKeyComparator not configurable (#10342) 2022-07-14 10:09:31 -07:00
index_reader_common.cc Seek parallelization (#9994) 2022-05-20 16:09:33 -07:00
index_reader_common.h Set Read rate limiter priority dynamically and pass it to FS (#9996) 2022-05-18 19:41:44 -07:00
mock_block_based_table.h Remove deprecated block-based filter (#10184) 2022-06-16 15:51:33 -07:00
parsed_full_filter_block.cc Hide FilterBits{Builder,Reader} from public API (#9592) 2022-02-17 16:34:46 -08:00
parsed_full_filter_block.h Use new Insert and Lookup APIs in table reader to support secondary cache (#8315) 2021-05-21 18:29:12 -07:00
partitioned_filter_block.cc Add new option num_file_reads_for_auto_readahead in BlockBasedTableOptions (#10556) 2022-09-01 11:56:00 -07:00
partitioned_filter_block.h Update passing rate_limiter_priority for a PartitionedFilterBlockReader function to FS (#10438) 2022-07-29 11:32:54 -07:00
partitioned_filter_block_test.cc Pass rate_limiter_priority through filter block reader functions to FS (#10251) 2022-06-24 16:13:44 -07:00
partitioned_index_iterator.cc Provide support for direct_reads with async_io (#10197) 2022-07-06 11:42:59 -07:00
partitioned_index_iterator.h Make initial auto readahead_size configurable (#9836) 2022-04-15 17:28:09 -07:00
partitioned_index_reader.cc Add new option num_file_reads_for_auto_readahead in BlockBasedTableOptions (#10556) 2022-09-01 11:56:00 -07:00
partitioned_index_reader.h Meta-internal folly integration with F14FastMap (#9546) 2022-04-13 07:34:01 -07:00
reader_common.cc Remove own ToString() (#9955) 2022-05-06 13:03:58 -07:00
reader_common.h Bring the Configurable options together (#5753) 2020-09-14 17:01:01 -07:00
uncompression_dict_reader.cc Seek parallelization (#9994) 2022-05-20 16:09:33 -07:00
uncompression_dict_reader.h Fb 9718 verify checksums is ignored (#9767) 2022-03-29 11:54:54 -07:00