dolysis/rocksdb
3
0
Fork 0
mirror of https://github.com/facebook/rocksdb.git synced 2024-11-26 16:30:56 +00:00
Code Issues Packages Projects Releases Wiki Activity
rocksdb/table/block_based
History
Ziyue Yang c384c08a4f Add tests for compression failure in BlockBasedTableBuilder (#6709) 
Summary:
Currently there is no check for whether BlockBasedTableBuilder will expose
compression error status if compression fails during the table building.
This commit adds fake faulting compressors and a unit test to test such
cases.

This check finds 5 bugs, and this commit also fixes them:

1. Not handling compression failure well in
   BlockBasedTableBuilder::BGWorkWriteRawBlock.
2. verify_compression failing in BlockBasedTableBuilder when used with ZSTD.
3. Wrongly passing the same reference of block contents to
   BlockBasedTableBuilder::CompressAndVerifyBlock in parallel compression.
4. Wrongly setting block_rep->first_key_in_next_block to nullptr in
   BlockBasedTableBuilder::EnterUnbuffered when there are still incoming data
   blocks.
5. Not maintaining variables for compression ratio estimation and first_block
   in BlockBasedTableBuilder::EnterUnbuffered.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/6709

Reviewed By: ajkr

Differential Revision: D21236254

fbshipit-source-id: 101f6e62b2bac2b7be72be198adf93cd32a1ff46
2020-05-12 09:27:35 -07:00
..
binary_search_index_reader.cc Divide block_based_table_reader.cc (#6527) 2020-03-12 21:41:50 -07:00
binary_search_index_reader.h Divide block_based_table_reader.cc (#6527) 2020-03-12 21:41:50 -07:00
block.cc Fix some typos in code comments (#6733) 2020-04-23 12:28:49 -07:00
block.h fix comparison count for format_version=3 indexes (#6650) 2020-04-13 11:18:37 -07:00
block_based_filter_block.cc Silence false alarms in db_stress fault injection (#6741) 2020-04-24 13:06:12 -07:00
block_based_filter_block.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
block_based_filter_block_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
block_based_table_builder.cc Add tests for compression failure in BlockBasedTableBuilder (#6709) 2020-05-12 09:27:35 -07:00
block_based_table_builder.h Disallow BlockBasedTableBuilder to set status from non-OK (#6776) 2020-04-30 15:37:03 -07:00
block_based_table_factory.cc Add OptionTypeInfo::Enum and related methods (#6423) 2020-05-05 15:04:04 -07:00
block_based_table_factory.h Add a ConfigOptions for use in comparing objects and converting to/from strings (#6389) 2020-04-21 17:38:17 -07:00
block_based_table_iterator.cc Properly report IO errors when IndexType::kBinarySearchWithFirstKey is used (#6621) 2020-04-15 17:40:44 -07:00
block_based_table_iterator.h Properly report IO errors when IndexType::kBinarySearchWithFirstKey is used (#6621) 2020-04-15 17:40:44 -07:00
block_based_table_reader.cc Fix false NotFound from batched MultiGet with kHashSearch (#6821) 2020-05-07 15:41:37 -07:00
block_based_table_reader.h Stats for redundant insertions into block cache (#6681) 2020-04-27 13:20:27 -07:00
block_based_table_reader_impl.h Divide block_based_table_reader.cc (#6527) 2020-03-12 21:41:50 -07:00
block_builder.cc Add pipelined & parallel compression optimization (#6262) 2020-04-01 16:40:18 -07:00
block_builder.h Add pipelined & parallel compression optimization (#6262) 2020-04-01 16:40:18 -07:00
block_prefetcher.cc De-template block based table iterator (#6531) 2020-03-16 12:20:50 -07:00
block_prefetcher.h De-template block based table iterator (#6531) 2020-03-16 12:20:50 -07:00
block_prefix_index.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
block_prefix_index.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
block_test.cc Fix range deletion tombstone ingestion with global seqno (#6429) 2020-02-25 15:31:48 -08:00
block_type.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
cachable_entry.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
data_block_footer.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
data_block_footer.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
data_block_hash_index.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
data_block_hash_index.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
data_block_hash_index_test.cc Fix range deletion tombstone ingestion with global seqno (#6429) 2020-02-25 15:31:48 -08:00
filter_block.h Fix iterator reading filter block despite read_tier == kBlockCacheTier (#6562) 2020-03-26 15:21:26 -07:00
filter_block_reader_common.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
filter_block_reader_common.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
filter_policy.cc Add Functions to OptionTypeInfo (#6422) 2020-04-28 18:04:26 -07:00
filter_policy_internal.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
flush_block_policy.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
flush_block_policy.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
full_filter_block.cc Basic MultiGet support for partitioned filters (#6757) 2020-04-28 14:49:34 -07:00
full_filter_block.h Fix iterator reading filter block despite read_tier == kBlockCacheTier (#6562) 2020-03-26 15:21:26 -07:00
full_filter_block_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
hash_index_reader.cc Divide block_based_table_reader.cc (#6527) 2020-03-12 21:41:50 -07:00
hash_index_reader.h Divide block_based_table_reader.cc (#6527) 2020-03-12 21:41:50 -07:00
index_builder.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
index_builder.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
index_reader_common.cc Divide block_based_table_reader.cc (#6527) 2020-03-12 21:41:50 -07:00
index_reader_common.h Divide block_based_table_reader.cc (#6527) 2020-03-12 21:41:50 -07:00
mock_block_based_table.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
parsed_full_filter_block.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
parsed_full_filter_block.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
partitioned_filter_block.cc Basic MultiGet support for partitioned filters (#6757) 2020-04-28 14:49:34 -07:00
partitioned_filter_block.h Basic MultiGet support for partitioned filters (#6757) 2020-04-28 14:49:34 -07:00
partitioned_filter_block_test.cc Fix range deletion tombstone ingestion with global seqno (#6429) 2020-02-25 15:31:48 -08:00
partitioned_index_iterator.cc Fix regression bug in partitioned index reseek caused by #6531 (#6551) 2020-03-17 12:33:10 -07:00
partitioned_index_iterator.h Fix compiler warning treated as error (#6547) 2020-03-17 09:59:28 -07:00
partitioned_index_reader.cc Silence false alarms in db_stress fault injection (#6741) 2020-04-24 13:06:12 -07:00
partitioned_index_reader.h Divide block_based_table_reader.cc (#6527) 2020-03-12 21:41:50 -07:00
reader_common.cc Divide block_based_table_reader.cc (#6527) 2020-03-12 21:41:50 -07:00
reader_common.h Divide block_based_table_reader.cc (#6527) 2020-03-12 21:41:50 -07:00
uncompression_dict_reader.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
uncompression_dict_reader.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00