dolysis/rocksdb
3
0
Fork 0
mirror of https://github.com/facebook/rocksdb.git synced 2024-11-27 02:44:18 +00:00
Code Issues Packages Projects Releases Wiki Activity
rocksdb/table/block_based
History
Akanksha Mahajan a0e0feca62 Improve BlockPrefetcher to prefetch only for sequential scans (#7394) 
Summary:
BlockPrefetcher is used by iterators to prefetch data if they
anticipate more data to be used in future and this is valid for forward sequential
scans. But BlockPrefetcher tracks only num_file_reads_ and not if reads
are sequential. This presents problem for MultiGet with large number of
keys when it reseeks index iterator and data block. FilePrefetchBuffer
can end up doing large readahead for reseeks as readahead size
increases exponentially once readahead is enabled. Same issue is with
BlockBasedTableIterator.

Add previous length and offset read as well in BlockPrefetcher (creates
FilePrefetchBuffer) and FilePrefetchBuffer (does prefetching of data) to
determine if reads are sequential and then  prefetch.

Update the last block read after cache hit to take reads from cache also
in account.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/7394

Test Plan: Add new unit test case

Reviewed By: anand1976

Differential Revision: D23737617

Pulled By: akankshamahajan15

fbshipit-source-id: 8e6917c25ed87b285ee495d1b68dc623d71205a3
2021-04-28 12:53:46 -07:00
..
binary_search_index_reader.cc Separate internal and user key comparators in BlockIter (#6944) 2020-07-07 17:26:16 -07:00
binary_search_index_reader.h Extend Get/MultiGet deadline support to table open (#6982) 2020-06-29 14:53:17 -07:00
block.cc Remove unused includes (#7604) 2020-10-28 23:22:27 -07:00
block.h Add EnvTestWithParam::OptionsTest to the ASSERT_STATUS_CHECKED passes (#7283) 2020-08-20 19:18:35 -07:00
block_based_filter_block.cc Exclude timestamp from prefix extractor (#7668) 2020-12-01 14:07:15 -08:00
block_based_filter_block.h Exclude timestamp from prefix extractor (#7668) 2020-12-01 14:07:15 -08:00
block_based_filter_block_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
block_based_table_builder.cc Rename variables in ImmutableCFOptions to avoid conflicts with ImmutableDBOptions (#8227) 2021-04-26 12:43:45 -07:00
block_based_table_builder.h Apply sample_for_compression to all block-based tables (#8105) 2021-03-25 15:00:45 -07:00
block_based_table_factory.cc Move RegisterOptions into the Configurable API (#8223) 2021-04-26 03:13:24 -07:00
block_based_table_factory.h Create a Customizable class to load classes and configurations (#6590) 2020-11-11 15:10:41 -08:00
block_based_table_iterator.cc Clean up InternalIterator upper bound logic a little bit (#7200) 2020-08-05 10:44:57 -07:00
block_based_table_iterator.h Exclude timestamp from prefix extractor (#7668) 2020-12-01 14:07:15 -08:00
block_based_table_reader.cc Improve BlockPrefetcher to prefetch only for sequential scans (#7394) 2021-04-28 12:53:46 -07:00
block_based_table_reader.h Improve BlockPrefetcher to prefetch only for sequential scans (#7394) 2021-04-28 12:53:46 -07:00
block_based_table_reader_impl.h Divide block_based_table_reader.cc (#6527) 2020-03-12 21:41:50 -07:00
block_based_table_reader_test.cc Apply sample_for_compression to all block-based tables (#8105) 2021-03-25 15:00:45 -07:00
block_builder.cc Add pipelined & parallel compression optimization (#6262) 2020-04-01 16:40:18 -07:00
block_builder.h Add pipelined & parallel compression optimization (#6262) 2020-04-01 16:40:18 -07:00
block_prefetcher.cc Improve BlockPrefetcher to prefetch only for sequential scans (#7394) 2021-04-28 12:53:46 -07:00
block_prefetcher.h Improve BlockPrefetcher to prefetch only for sequential scans (#7394) 2021-04-28 12:53:46 -07:00
block_prefix_index.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
block_prefix_index.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
block_test.cc More Makefile Cleanup (#7097) 2020-07-09 14:35:17 -07:00
block_type.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
cachable_entry.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
data_block_footer.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
data_block_footer.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
data_block_hash_index.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
data_block_hash_index.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
data_block_hash_index_test.cc Apply sample_for_compression to all block-based tables (#8105) 2021-03-25 15:00:45 -07:00
filter_block.h Exclude timestamp from prefix extractor (#7668) 2020-12-01 14:07:15 -08:00
filter_block_reader_common.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
filter_block_reader_common.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
filter_policy.cc Revert Ribbon starting level support from #8198 (#8212) 2021-04-20 19:46:40 -07:00
filter_policy_internal.h Revert Ribbon starting level support from #8198 (#8212) 2021-04-20 19:46:40 -07:00
flush_block_policy.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
flush_block_policy.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
full_filter_block.cc Fix a bug for SeekForPrev with partitioned filter and prefix (#8137) 2021-04-06 12:14:08 -07:00
full_filter_block.h Fix a bug for SeekForPrev with partitioned filter and prefix (#8137) 2021-04-06 12:14:08 -07:00
full_filter_block_test.cc Use size_t for filter APIs, protect against overflow (#7726) 2020-12-11 22:18:12 -08:00
hash_index_reader.cc Separate internal and user key comparators in BlockIter (#6944) 2020-07-07 17:26:16 -07:00
hash_index_reader.h Extend Get/MultiGet deadline support to table open (#6982) 2020-06-29 14:53:17 -07:00
index_builder.cc Move break into block (#7468) 2020-09-30 20:24:23 -07:00
index_builder.h Make db_basic_test pass assert status checked (#7452) 2020-09-29 09:49:04 -07:00
index_reader_common.cc Divide block_based_table_reader.cc (#6527) 2020-03-12 21:41:50 -07:00
index_reader_common.h Divide block_based_table_reader.cc (#6527) 2020-03-12 21:41:50 -07:00
mock_block_based_table.h Rename variables in ImmutableCFOptions to avoid conflicts with ImmutableDBOptions (#8227) 2021-04-26 12:43:45 -07:00
parsed_full_filter_block.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
parsed_full_filter_block.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
partitioned_filter_block.cc Improve BlockPrefetcher to prefetch only for sequential scans (#7394) 2021-04-28 12:53:46 -07:00
partitioned_filter_block.h Return error if Get/Multi() fails in Prefetching Filter blocks (#7543) 2020-10-14 10:45:36 -07:00
partitioned_filter_block_test.cc Remove unused includes (#7604) 2020-10-28 23:22:27 -07:00
partitioned_index_iterator.cc Fix misspelling of PartitionedIndexIterator (#7450) 2020-09-29 16:28:13 -07:00
partitioned_index_iterator.h Fix misspelling of PartitionedIndexIterator (#7450) 2020-09-29 16:28:13 -07:00
partitioned_index_reader.cc Improve BlockPrefetcher to prefetch only for sequential scans (#7394) 2021-04-28 12:53:46 -07:00
partitioned_index_reader.h Get() to fail with underlying failures in PartitionIndexReader::CacheDependencies() (#7297) 2020-08-25 19:01:05 -07:00
reader_common.cc Fix block checksum for >=4GB, refactor (#6978) 2020-06-19 16:18:24 -07:00
reader_common.h Bring the Configurable options together (#5753) 2020-09-14 17:01:01 -07:00
uncompression_dict_reader.cc Rename variables in ImmutableCFOptions to avoid conflicts with ImmutableDBOptions (#8227) 2021-04-26 12:43:45 -07:00
uncompression_dict_reader.h Extend Get/MultiGet deadline support to table open (#6982) 2020-06-29 14:53:17 -07:00