mirror of https://github.com/facebook/rocksdb.git
4da34b97ee
Summary: ### Context: Background compactions and flush generate large reads and writes, and can be long running, especially for universal compaction. In some cases, this can impact foreground reads and writes by users. ### Solution User, Flush, and Compaction reads share some code path. For this task, we update the rate_limiter_priority in ReadOptions for code paths (e.g. FindTable (mainly in BlockBasedTable::Open()) and various iterators), and eventually update the rate_limiter_priority in IOOptions for FSRandomAccessFile. **This PR is for the Read path.** The **Read:** dynamic priority for different state are listed as follows: | State | Normal | Delayed | Stalled | | ----- | ------ | ------- | ------- | | Flush (verification read in BuildTable()) | IO_USER | IO_USER | IO_USER | | Compaction | IO_LOW | IO_USER | IO_USER | | User | User provided | User provided | User provided | We will respect the read_options that the user provided and will not set it. The only sst read for Flush is the verification read in BuildTable(). It claims to be "regard as user read". **Details** 1. Set read_options.rate_limiter_priority dynamically: - User: Do not update the read_options. Use the read_options that the user provided. - Compaction: Update read_options in CompactionJob::ProcessKeyValueCompaction(). - Flush: Update read_options in BuildTable(). 2. Pass the rate limiter priority to FSRandomAccessFile functions: - After calling the FindTable(), read_options is passed through GetTableReader(table_cache.cc), BlockBasedTableFactory::NewTableReader(block_based_table_factory.cc), and BlockBasedTable::Open(). The Open() needs some updates for the ReadOptions variable and the updates are also needed for the called functions, including PrefetchTail(), PrepareIOOptions(), ReadFooterFromFile(), ReadMetaIndexblock(), ReadPropertiesBlock(), PrefetchIndexAndFilterBlocks(), and ReadRangeDelBlock(). - In RandomAccessFileReader, the functions to be updated include Read(), MultiRead(), ReadAsync(), and Prefetch(). - Update the downstream functions of NewIndexIterator(), NewDataBlockIterator(), and BlockBasedTableIterator(). ### Test Plans Add unit tests. Pull Request resolved: https://github.com/facebook/rocksdb/pull/9996 Reviewed By: anand1976 Differential Revision: D36452483 Pulled By: gitbw95 fbshipit-source-id: 60978204a4f849bb9261cb78d9bc1cb56d6008cf |
||
|---|---|---|
| .. | ||
| binary_search_index_reader.cc | ||
| binary_search_index_reader.h | ||
| block.cc | ||
| block.h | ||
| block_based_filter_block.cc | ||
| block_based_filter_block.h | ||
| block_based_filter_block_test.cc | ||
| block_based_table_builder.cc | ||
| block_based_table_builder.h | ||
| block_based_table_factory.cc | ||
| block_based_table_factory.h | ||
| block_based_table_iterator.cc | ||
| block_based_table_iterator.h | ||
| block_based_table_reader.cc | ||
| block_based_table_reader.h | ||
| block_based_table_reader_impl.h | ||
| block_based_table_reader_test.cc | ||
| block_builder.cc | ||
| block_builder.h | ||
| block_like_traits.h | ||
| block_prefetcher.cc | ||
| block_prefetcher.h | ||
| block_prefix_index.cc | ||
| block_prefix_index.h | ||
| block_test.cc | ||
| block_type.h | ||
| cachable_entry.h | ||
| data_block_footer.cc | ||
| data_block_footer.h | ||
| data_block_hash_index.cc | ||
| data_block_hash_index.h | ||
| data_block_hash_index_test.cc | ||
| filter_block.h | ||
| filter_block_reader_common.cc | ||
| filter_block_reader_common.h | ||
| filter_policy.cc | ||
| filter_policy_internal.h | ||
| flush_block_policy.cc | ||
| flush_block_policy.h | ||
| full_filter_block.cc | ||
| full_filter_block.h | ||
| full_filter_block_test.cc | ||
| hash_index_reader.cc | ||
| hash_index_reader.h | ||
| index_builder.cc | ||
| index_builder.h | ||
| index_reader_common.cc | ||
| index_reader_common.h | ||
| mock_block_based_table.h | ||
| parsed_full_filter_block.cc | ||
| parsed_full_filter_block.h | ||
| partitioned_filter_block.cc | ||
| partitioned_filter_block.h | ||
| partitioned_filter_block_test.cc | ||
| partitioned_index_iterator.cc | ||
| partitioned_index_iterator.h | ||
| partitioned_index_reader.cc | ||
| partitioned_index_reader.h | ||
| reader_common.cc | ||
| reader_common.h | ||
| uncompression_dict_reader.cc | ||
| uncompression_dict_reader.h | ||