dolysis/rocksdb
3
0
Fork 0
mirror of https://github.com/facebook/rocksdb.git synced 2024-11-26 07:30:54 +00:00
Code Issues Packages Projects Releases Wiki Activity
rocksdb/db
History
Manuel Ung eae832740b WriteUnPrepared: improve read your own write functionality (#5573) 
Summary:
There are a number of fixes in this PR (with most bugs found via the added stress tests):
1. Re-enable reseek optimization. This was initially disabled to avoid infinite loops in https://github.com/facebook/rocksdb/pull/3955 but this can be resolved by remembering not to reseek after a reseek has already been done. This problem only affects forward iteration in `DBIter::FindNextUserEntryInternal`, as we already disable reseeking in `DBIter::FindValueForCurrentKeyUsingSeek`.
2. Verify that ReadOption.snapshot can be safely used for iterator creation. Some snapshots would not give correct results because snaphsot validation would not be enforced, breaking some assumptions in Prev() iteration.
3. In the non-snapshot Get() case, reads done at `LastPublishedSequence` may not be enough, because unprepared sequence numbers are not published. Use `std::max(published_seq, max_visible_seq)` to do lookups instead.
4. Add stress test to test reading own writes.
5. Minor bug in the allow_concurrent_memtable_write case where we forgot to pass in batch_per_txn_.
6. Minor performance optimization in `CalcMaxUnpreparedSequenceNumber` by assigning by reference instead of value.
7. Add some more comments everywhere.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/5573

Differential Revision: D16276089

Pulled By: lth

fbshipit-source-id: 18029c944eb427a90a87dee76ac1b23f37ec1ccb
2019-07-23 08:08:19 -07:00
..
compaction Add more callers for table reader. (#5454) 2019-06-20 14:31:48 -07:00
db_impl WriteUnPrepared: improve read your own write functionality (#5573) 2019-07-23 08:08:19 -07:00
builder.cc Add more callers for table reader. (#5454) 2019-06-20 14:31:48 -07:00
builder.h Move some logging related files to logging/ (#5387) 2019-05-31 17:23:59 -07:00
c.cc Fix memorty leak in rocksdb_wal_iter_get_batch function (#5515) 2019-07-15 12:59:39 -07:00
c_test.c Add C binding for secondary instance (#5505) 2019-06-27 08:58:54 -07:00
column_family.cc Integrate block cache tracer into db_impl (#5433) 2019-06-13 15:43:10 -07:00
column_family.h Integrate block cache tracer into db_impl (#5433) 2019-06-13 15:43:10 -07:00
column_family_test.cc Make format 2019-05-31 15:24:43 -07:00
compact_files_test.cc Organizing rocksdb/db directory 2019-05-31 11:57:01 -07:00
compacted_db_impl.cc Organizing rocksdb/db directory 2019-05-31 11:57:01 -07:00
compacted_db_impl.h Export Import sst files (#5495) 2019-07-17 12:27:14 -07:00
comparator_db_test.cc Move some memory related files from util/ to memory/ (#5382) 2019-05-30 17:44:09 -07:00
convenience.cc Add more callers for table reader. (#5454) 2019-06-20 14:31:48 -07:00
corruption_test.cc simplify include directive involving inttypes (#5402) 2019-06-06 13:56:07 -07:00
cuckoo_table_db_test.cc Organizing rocksdb/db directory 2019-05-31 11:57:01 -07:00
db_basic_test.cc Support GetAllKeyVersions() for non-default cf (#5544) 2019-07-07 22:43:52 -07:00
db_blob_index_test.cc fix lite build 2017-10-17 08:57:09 -07:00
db_block_cache_test.cc Move the filter readers out of the block cache (#5504) 2019-07-16 13:14:58 -07:00
db_bloom_filter_test.cc Unordered Writes (#5218) 2019-05-13 17:47:21 -07:00
db_compaction_filter_test.cc Apply modernize-use-override (2nd iteration) 2019-02-14 14:41:36 -08:00
db_compaction_test.cc Combine the read-ahead logic for user reads and compaction reads (#5431) 2019-06-19 14:10:46 -07:00
db_dynamic_level_test.cc Fix flaky DBDynamicLevelTest.DynamicLevelMaxBytesBase2 (#4668) 2018-11-12 16:42:16 -08:00
db_encryption_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
db_filesnapshot.cc Add missing check before calling PurgeObsoleteFiles in EnableFileDeletions (#5448) 2019-06-13 14:43:13 -07:00
db_flush_test.cc Ref and unref cfd before and after calling WaitForFlushMemTables (#5513) 2019-07-01 14:12:02 -07:00
db_info_dumper.cc simplify include directive involving inttypes (#5402) 2019-06-06 13:56:07 -07:00
db_info_dumper.h Change RocksDB License 2017-07-15 16:11:23 -07:00
db_inplace_update_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_io_failure_test.cc Disable DBIOFailureTest.NoSpaceCompactRange in LITE (#4596) 2018-10-29 14:36:31 -07:00
db_iter.cc WriteUnPrepared: improve read your own write functionality (#5573) 2019-07-23 08:08:19 -07:00
db_iter.h Make format 2019-05-31 15:24:43 -07:00
db_iter_stress_test.cc Move some memory related files from util/ to memory/ (#5382) 2019-05-30 17:44:09 -07:00
db_iter_test.cc Move some memory related files from util/ to memory/ (#5382) 2019-05-30 17:44:09 -07:00
db_iterator_test.cc Fix lower bound check error when iterate across file boundary (#5540) 2019-07-04 17:28:30 -07:00
db_log_iter_test.cc Apply modernize-use-override (2nd iteration) 2019-02-14 14:41:36 -08:00
db_memtable_test.cc Fix tsan complaint in ConcurrentMergeWrite test (#5308) 2019-05-15 11:21:48 -07:00
db_merge_operator_test.cc WriteUnPrepared: less virtual in iterator callback (#5049) 2019-04-02 14:47:16 -07:00
db_options_test.cc fix rocksdb lite and clang contrun test failures (#5477) 2019-06-17 21:16:29 -07:00
db_properties_test.cc Deprecate ttl option from CompactionOptionsFIFO (#4965) 2019-02-15 09:51:41 -08:00
db_range_del_test.cc Fix merging range tombstone covering put during flush/compaction (#5406) 2019-06-04 10:24:14 -07:00
db_sst_test.cc Fix bugs in WAL trash file handling (#5520) 2019-07-06 21:07:32 -07:00
db_statistics_test.cc Make statistics's stats_level change thread-safe (#5030) 2019-03-01 10:42:09 -08:00
db_table_properties_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
db_tailing_iter_test.cc Remove managed iterator 2018-07-17 14:43:18 -07:00
db_test.cc Export Import sst files (#5495) 2019-07-17 12:27:14 -07:00
db_test2.cc row_cache to share entry for recent snapshots (#5600) 2019-07-22 18:56:19 -07:00
db_test_util.cc Unordered Writes (#5218) 2019-05-13 17:47:21 -07:00
db_test_util.h simplify include directive involving inttypes (#5402) 2019-06-06 13:56:07 -07:00
db_universal_compaction_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
db_wal_test.cc Fix bugs in DBWALTest.kTolerateCorruptedTailRecords triggered by #5520 (#5550) 2019-07-09 11:18:32 -07:00
db_write_test.cc Move some memory related files from util/ to memory/ (#5382) 2019-05-30 17:44:09 -07:00
dbformat.cc simplify include directive involving inttypes (#5402) 2019-06-06 13:56:07 -07:00
dbformat.h Add support for timestamp in Get/Put (#5079) 2019-06-05 23:10:47 -07:00
dbformat_test.cc Move some logging related files to logging/ (#5387) 2019-05-31 17:23:59 -07:00
deletefile_test.cc Organizing rocksdb/db directory 2019-05-31 11:57:01 -07:00
error_handler.cc Make format 2019-05-31 15:24:43 -07:00
error_handler.h Fix typos in comments (#4456) 2018-10-04 20:46:50 -07:00
error_handler_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
event_helpers.cc Log file_creation_time table property (#5232) 2019-04-22 15:30:07 -07:00
event_helpers.h Move some logging related files to logging/ (#5387) 2019-05-31 17:23:59 -07:00
experimental.cc Organizing rocksdb/db directory 2019-05-31 11:57:01 -07:00
external_sst_file_basic_test.cc Fix ingested file and direcotry not being sync (#5435) 2019-06-21 10:15:38 -07:00
external_sst_file_ingestion_job.cc Fix ingested file and direcotry not being sync (#5435) 2019-06-21 10:15:38 -07:00
external_sst_file_ingestion_job.h Fix ingested file and direcotry not being sync (#5435) 2019-06-21 10:15:38 -07:00
external_sst_file_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
fault_injection_test.cc Move some logging related files to logging/ (#5387) 2019-05-31 17:23:59 -07:00
file_indexer.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
file_indexer.h Move some memory related files from util/ to memory/ (#5382) 2019-05-30 17:44:09 -07:00
file_indexer_test.cc Move some memory related files from util/ to memory/ (#5382) 2019-05-30 17:44:09 -07:00
filename_test.cc Move some logging related files to logging/ (#5387) 2019-05-31 17:23:59 -07:00
flush_job.cc simplify include directive involving inttypes (#5402) 2019-06-06 13:56:07 -07:00
flush_job.h Move some logging related files to logging/ (#5387) 2019-05-31 17:23:59 -07:00
flush_job_test.cc Integrate block cache tracer into db_impl (#5433) 2019-06-13 15:43:10 -07:00
flush_scheduler.cc Remove global locks from FlushScheduler (#5372) 2019-06-10 16:50:26 -07:00
flush_scheduler.h Unordered Writes (#5218) 2019-05-13 17:47:21 -07:00
forward_iterator.cc Add more callers for table reader. (#5454) 2019-06-20 14:31:48 -07:00
forward_iterator.h Move some memory related files from util/ to memory/ (#5382) 2019-05-30 17:44:09 -07:00
forward_iterator_bench.cc simplify include directive involving inttypes (#5402) 2019-06-06 13:56:07 -07:00
import_column_family_job.cc Export Import sst files (#5495) 2019-07-17 12:27:14 -07:00
import_column_family_job.h Export Import sst files (#5495) 2019-07-17 12:27:14 -07:00
import_column_family_test.cc Fix tsan and valgrind failures in import_column_family_test 2019-07-19 13:25:36 -07:00
internal_stats.cc simplify include directive involving inttypes (#5402) 2019-06-06 13:56:07 -07:00
internal_stats.h Fix MyRocks compile warnings-treated-as-errors on Fedora 30, gcc 9.1.1 (#5553) 2019-07-12 17:30:51 -07:00
job_context.h WritePrepared: Fix visible key compacted out by compaction (#4883) 2019-01-15 21:34:38 -08:00
listener_test.cc Move some logging related files to logging/ (#5387) 2019-05-31 17:23:59 -07:00
log_format.h Fix an inaccurate comment (#4315) 2018-08-24 18:13:20 -07:00
log_reader.cc Support for single-primary, multi-secondary instances (#4899) 2019-03-26 16:45:31 -07:00
log_reader.h secondary instance: add support for WAL tailing on OpenAsSecondary 2019-04-24 12:08:44 -07:00
log_test.cc Move some memory related files from util/ to memory/ (#5382) 2019-05-30 17:44:09 -07:00
log_writer.cc LogWriter to only flush after finish generating whole record (#5328) 2019-05-21 12:33:17 -07:00
log_writer.h Close WAL files before deletion (#5233) 2019-04-25 10:11:41 -07:00
logs_with_prep_tracker.cc Skip deleted WALs during recovery 2018-05-03 15:43:09 -07:00
logs_with_prep_tracker.h Skip deleted WALs during recovery 2018-05-03 15:43:09 -07:00
lookup_key.h Introduce a new MultiGet batching implementation (#5011) 2019-04-11 14:28:26 -07:00
malloc_stats.cc Support jemalloc compiled with --with-jemalloc-prefix (#5521) 2019-07-02 12:07:01 -07:00
malloc_stats.h Change RocksDB License 2017-07-15 16:11:23 -07:00
manual_compaction_test.cc Move some memory related files from util/ to memory/ (#5382) 2019-05-30 17:44:09 -07:00
memtable.cc Add support for timestamp in Get/Put (#5079) 2019-06-05 23:10:47 -07:00
memtable.h Move some memory related files from util/ to memory/ (#5382) 2019-05-30 17:44:09 -07:00
memtable_list.cc simplify include directive involving inttypes (#5402) 2019-06-06 13:56:07 -07:00
memtable_list.h Move some logging related files to logging/ (#5387) 2019-05-31 17:23:59 -07:00
memtable_list_test.cc Integrate block cache tracer into db_impl (#5433) 2019-06-13 15:43:10 -07:00
merge_context.h Introduce a new MultiGet batching implementation (#5011) 2019-04-11 14:28:26 -07:00
merge_helper.cc Fix merging range tombstone covering put during flush/compaction (#5406) 2019-06-04 10:24:14 -07:00
merge_helper.h Remove v1 RangeDelAggregator (#4778) 2018-12-17 17:33:46 -08:00
merge_helper_test.cc Move some memory related files from util/ to memory/ (#5382) 2019-05-30 17:44:09 -07:00
merge_operator.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
merge_test.cc Make format 2019-05-31 15:24:43 -07:00
obsolete_files_test.cc Organizing rocksdb/db directory 2019-05-31 11:57:01 -07:00
options_file_test.cc Organizing rocksdb/db directory 2019-05-31 11:57:01 -07:00
perf_context_test.cc Move some memory related files from util/ to memory/ (#5382) 2019-05-30 17:44:09 -07:00
pinned_iterators_manager.h Change RocksDB License 2017-07-15 16:11:23 -07:00
plain_table_db_test.cc Move some logging related files to logging/ (#5387) 2019-05-31 17:23:59 -07:00
pre_release_callback.h WritePrepared: reduce prepared_mutex_ overhead (#5420) 2019-06-10 11:53:31 -07:00
prefix_test.cc Organizing rocksdb/db directory 2019-05-31 11:57:01 -07:00
range_del_aggregator.cc Organizing rocksdb/db directory 2019-05-31 11:57:01 -07:00
range_del_aggregator.h Organizing rocksdb/db directory 2019-05-31 11:57:01 -07:00
range_del_aggregator_bench.cc Move some memory related files from util/ to memory/ (#5382) 2019-05-30 17:44:09 -07:00
range_del_aggregator_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
range_tombstone_fragmenter.cc simplify include directive involving inttypes (#5402) 2019-06-06 13:56:07 -07:00
range_tombstone_fragmenter.h Add compaction logic to RangeDelAggregatorV2 (#4758) 2018-12-17 13:20:51 -08:00
range_tombstone_fragmenter_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
read_callback.h WriteUnPrepared: improve read your own write functionality (#5573) 2019-07-23 08:08:19 -07:00
repair.cc Add more callers for table reader. (#5454) 2019-06-20 14:31:48 -07:00
repair_test.cc Organizing rocksdb/db directory 2019-05-31 11:57:01 -07:00
snapshot_checker.h WritePrepared: fix issue with snapshot released during compaction (#4858) 2019-01-16 09:55:32 -08:00
snapshot_impl.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
snapshot_impl.h Refresh snapshot list during long compactions (2nd attempt) (#5278) 2019-05-03 17:30:22 -07:00
table_cache.cc row_cache to share entry for recent snapshots (#5600) 2019-07-22 18:56:19 -07:00
table_cache.h Remove RandomAccessFileReader.for_compaction_ (#5572) 2019-07-16 16:32:18 -07:00
table_properties_collector.cc Feature for sampling and reporting compressibility (#4842) 2019-03-18 12:15:34 -07:00
table_properties_collector.h Feature for sampling and reporting compressibility (#4842) 2019-03-18 12:15:34 -07:00
table_properties_collector_test.cc Organizing rocksdb/db directory 2019-05-31 11:57:01 -07:00
transaction_log_impl.cc Replace Corruption with TryAgain status when new tail is not visible to TransactionLogIterator (#5474) 2019-06-19 08:10:08 -07:00
transaction_log_impl.h Move some file related files outside util/ (#5375) 2019-05-29 20:47:06 -07:00
version_builder.cc simplify include directive involving inttypes (#5402) 2019-06-06 13:56:07 -07:00
version_builder.h Support for single-primary, multi-secondary instances (#4899) 2019-03-26 16:45:31 -07:00
version_builder_test.cc Move some logging related files to logging/ (#5387) 2019-05-31 17:23:59 -07:00
version_edit.cc Move some logging related files to logging/ (#5387) 2019-05-31 17:23:59 -07:00
version_edit.h Fix MyRocks compile warnings-treated-as-errors on Fedora 30, gcc 9.1.1 (#5553) 2019-07-12 17:30:51 -07:00
version_edit_test.cc Move some memory related files from util/ to memory/ (#5382) 2019-05-30 17:44:09 -07:00
version_set.cc Added log_readahead_size option to control prefetching for Log::Reader (#5592) 2019-07-19 12:00:19 -07:00
version_set.h Add more callers for table reader. (#5454) 2019-06-20 14:31:48 -07:00
version_set_test.cc Integrate block cache tracer into db_impl (#5433) 2019-06-13 15:43:10 -07:00
wal_manager.cc Fix bugs in WAL trash file handling (#5520) 2019-07-06 21:07:32 -07:00
wal_manager.h Fix bugs in WAL trash file handling (#5520) 2019-07-06 21:07:32 -07:00
wal_manager_test.cc Replace Corruption with TryAgain status when new tail is not visible to TransactionLogIterator (#5474) 2019-06-19 08:10:08 -07:00
write_batch.cc Make format 2019-05-31 15:24:43 -07:00
write_batch_base.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
write_batch_internal.h WriteUnPrepared: Add new WAL marker kTypeBeginUnprepareXID (#4069) 2018-06-28 18:58:29 -07:00
write_batch_test.cc Move some memory related files from util/ to memory/ (#5382) 2019-05-30 17:44:09 -07:00
write_callback.h Change RocksDB License 2017-07-15 16:11:23 -07:00
write_callback_test.cc WritePrepared: reduce prepared_mutex_ overhead (#5420) 2019-06-10 11:53:31 -07:00
write_controller.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
write_controller.h Change RocksDB License 2017-07-15 16:11:23 -07:00
write_controller_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
write_thread.cc Move some memory related files from util/ to memory/ (#5382) 2019-05-30 17:44:09 -07:00
write_thread.h Fix skip WAL for whole write_group when leader's callback fail (#4838) 2019-01-03 12:40:42 -08:00