dolysis
/
rocksdb
mirror of https://github.com/facebook/rocksdb.git
3
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
rocksdb/db
History
krad de85e4cadf Introduce WAL recovery consistency levels 
Summary:
The "one size fits all" approach with WAL recovery will only introduce inconvenience for our varied clients as we go forward. The current recovery is a bit heuristic. We introduce the following levels of consistency while replaying the WAL.

1. RecoverAfterRestart (kTolerateCorruptedTailRecords)

This mocks the current recovery mode.

2. RecoverAfterCleanShutdown (kAbsoluteConsistency)

This is ideal for unit test and cases where the store is shutdown cleanly. We tolerate no corruption or incomplete writes.

3. RecoverPointInTime (kPointInTimeRecovery)

This is ideal when using devices with controller cache or file systems which can loose data on restart. We recover upto the point were is no corruption or incomplete write.

4. RecoverAfterDisaster (kSkipAnyCorruptRecord)

This is ideal mode to recover data. We tolerate corruption and incomplete writes, and we hop over those sections that we cannot make sense of salvaging as many records as possible.

Test Plan:
(1) Run added unit test to cover all levels.
(2) Run make check.

Reviewers: leveldb, sdong, igor

Subscribers: yoshinorim, dhruba

Differential Revision: https://reviews.facebook.net/D38487
 		2015-06-22 15:28:12 -07:00
..
builder.cc Add TablePropertiesCollector::NeedCompact() to suggest DB to further compact output files 2015-06-05 20:18:21 -07:00
builder.h Add more table properties to EventLogger 2015-05-12 15:53:55 -07:00
c.cc Use CompactRangeOptions for CompactRange 2015-06-17 14:36:14 -07:00
c_test.c Block c_test in ROCKSDB_LITE 2015-06-17 10:54:51 -07:00
column_family.cc Fail DB::Open() when the requested compression is not available 2015-06-18 14:55:05 -07:00
column_family.h Fail DB::Open() when the requested compression is not available 2015-06-18 14:55:05 -07:00
column_family_test.cc Use CompactRangeOptions for CompactRange 2015-06-17 14:36:14 -07:00
compact_files_test.cc Make "make all" work for CYGWIN 2015-06-09 16:36:07 -07:00
compaction.cc Use nullptr for default compaction_filter_factory 2015-06-08 16:34:26 -07:00
compaction.h Allowing L0 -> L1 trivial move on sorted data 2015-06-04 16:51:25 -07:00
compaction_job.cc Fixed a bug of CompactionStats in multi-level universal compaction case 2015-06-17 23:40:34 -07:00
compaction_job.h Fixed a bug of CompactionStats in multi-level universal compaction case 2015-06-17 23:40:34 -07:00
compaction_job_stats_test.cc Skip bottommost level compaction if possible 2015-06-18 11:03:31 -07:00
compaction_job_test.cc Add TablePropertiesCollector::NeedCompact() to suggest DB to further compact output files 2015-06-05 20:18:21 -07:00
compaction_picker.cc Don't let two L0->L1 compactions run in parallel 2015-06-11 15:42:16 -07:00
compaction_picker.h Optimize GetRange Function 2015-05-05 09:57:47 -07:00
compaction_picker_test.cc Allowing L0 -> L1 trivial move on sorted data 2015-06-04 16:51:25 -07:00
comparator_db_test.cc Make "make all" work for CYGWIN 2015-06-09 16:36:07 -07:00
corruption_test.cc fix typos 2015-04-25 18:14:27 +09:00
cuckoo_table_db_test.cc Allowing L0 -> L1 trivial move on sorted data 2015-06-04 16:51:25 -07:00
db_bench.cc Add --benchmark_write_rate_limit option to db_bench 2015-06-17 16:44:52 -07:00
db_filesnapshot.cc Add wal files to Checkpoint for multiple column families. 2015-06-19 16:08:31 -07:00
db_impl.cc Introduce WAL recovery consistency levels 2015-06-22 15:28:12 -07:00
db_impl.h Use CompactRangeOptions for CompactRange 2015-06-17 14:36:14 -07:00
db_impl_debug.cc Allowing L0 -> L1 trivial move on sorted data 2015-06-04 16:51:25 -07:00
db_impl_experimental.cc Clean up InstallSuperVersion 2015-06-17 12:37:59 -07:00
db_impl_readonly.cc Move GetThreadList() feature under Env. 2014-12-22 12:20:17 -08:00
db_impl_readonly.h Use CompactRangeOptions for CompactRange 2015-06-17 14:36:14 -07:00
db_iter.cc fix typos 2015-04-25 18:14:27 +09:00
db_iter.h reduce references to cfd->options() in DBImpl 2014-09-08 15:04:34 -07:00
db_iter_test.cc rocksdb: Remove #include "util/string_util.h" from util/testharness.h 2015-03-19 17:29:37 -07:00
db_test.cc Introduce WAL recovery consistency levels 2015-06-22 15:28:12 -07:00
dbformat.cc Replace %llu with format macros in ParsedInternalKey::DebugString()) 2015-06-17 20:44:26 -07:00
dbformat.h Optimistic Transactions 2015-05-29 14:36:35 -07:00
dbformat_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
deletefile_test.cc Use CompactRangeOptions for CompactRange 2015-06-17 14:36:14 -07:00
event_helpers.cc Add EventListener::OnTableFileDeletion() 2015-06-03 19:57:01 -07:00
event_helpers.h Add EventListener::OnTableFileDeletion() 2015-06-03 19:57:01 -07:00
experimental.cc Implement DB::PromoteL0 method 2015-04-23 12:10:36 -07:00
fault_injection_test.cc Use CompactRangeOptions for CompactRange 2015-06-17 14:36:14 -07:00
file_indexer.cc Fix possible SIGSEGV in CompactRange (github issue #596) 2015-04-29 10:52:31 -07:00
file_indexer.h Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
file_indexer_test.cc Fix possible SIGSEGV in CompactRange (github issue #596) 2015-04-29 10:52:31 -07:00
filename.cc Sync manifest file when initializing it 2015-01-22 14:32:03 -08:00
filename.h Sync manifest file when initializing it 2015-01-22 14:32:03 -08:00
filename_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
flush_job.cc Add largest sequence to FlushJobInfo 2015-06-11 15:22:22 -07:00
flush_job.h Add largest sequence to FlushJobInfo 2015-06-11 15:22:22 -07:00
flush_job_test.cc Optimistic Transactions 2015-05-29 14:36:35 -07:00
flush_scheduler.cc Don't return (or dereference) dangling pointer 2014-10-02 14:33:16 -07:00
flush_scheduler.h Fix data race #1 2015-01-26 11:48:07 -08:00
forward_iterator.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
forward_iterator.h rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
internal_stats.cc Fixed a bug of CompactionStats in multi-level universal compaction case 2015-06-17 23:40:34 -07:00
internal_stats.h Fixed a bug of CompactionStats in multi-level universal compaction case 2015-06-17 23:40:34 -07:00
job_context.h Clean up old log files in background threads 2015-03-30 15:04:10 -04:00
listener_test.cc Use CompactRangeOptions for CompactRange 2015-06-17 14:36:14 -07:00
log_format.h Some minor refactoring on the code 2014-01-02 16:32:31 -08:00
log_reader.cc Introduce WAL recovery consistency levels 2015-06-22 15:28:12 -07:00
log_reader.h Introduce WAL recovery consistency levels 2015-06-22 15:28:12 -07:00
log_test.cc Introduce WAL recovery consistency levels 2015-06-22 15:28:12 -07:00
log_writer.cc Fix comparison between signed and usigned integers 2015-05-19 10:59:30 -07:00
log_writer.h Log writer record format doc. 2015-04-07 16:25:56 -07:00
managed_iterator.cc Fix compile error on MacOS. 2015-02-24 16:24:53 -08:00
managed_iterator.h Fixed xfunc related compile errors in ROCKSDB_LITE 2015-04-09 21:05:18 -07:00
memtable.cc Allow GetApproximateSize() to include mem table size if it is skip list memtable 2015-06-16 18:13:23 -07:00
memtable.h Allow GetApproximateSize() to include mem table size if it is skip list memtable 2015-06-16 18:13:23 -07:00
memtable_allocator.cc Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
memtable_allocator.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
memtable_list.cc Allow GetApproximateSize() to include mem table size if it is skip list memtable 2015-06-16 18:13:23 -07:00
memtable_list.h Allow GetApproximateSize() to include mem table size if it is skip list memtable 2015-06-16 18:13:23 -07:00
memtable_list_test.cc Slow down writes by bytes written 2015-06-11 20:42:18 -07:00
memtablerep_bench.cc build: avoid unused-variable warning 2015-05-02 13:19:10 -07:00
merge_context.h API to fetch from both a WriteBatchWithIndex and the db 2015-05-11 14:51:51 -07:00
merge_helper.cc Allow EventListener::OnCompactionCompleted to return CompactionJobStats. 2015-06-02 17:07:16 -07:00
merge_helper.h Helper function to time Merges 2015-04-27 20:23:50 -07:00
merge_operator.cc Some small cleaning up to make some compiling environment happy 2014-03-26 18:11:41 -07:00
merge_test.cc Use CompactRangeOptions for CompactRange 2015-06-17 14:36:14 -07:00
perf_context_test.cc Makefile minor cleanup 2015-03-30 16:05:35 -04:00
plain_table_db_test.cc rocksdb: Remove #include "util/string_util.h" from util/testharness.h 2015-03-19 17:29:37 -07:00
prefix_test.cc rocksdb: Remove #include "util/string_util.h" from util/testharness.h 2015-03-19 17:29:37 -07:00
repair.cc Add TablePropertiesCollector::NeedCompact() to suggest DB to further compact output files 2015-06-05 20:18:21 -07:00
skiplist.h Allow GetApproximateSize() to include mem table size if it is skip list memtable 2015-06-16 18:13:23 -07:00
skiplist_test.cc rocksdb: switch to gtest 2015-03-17 14:08:00 -07:00
slice.cc Create an abstract interface for write batches 2015-03-17 19:23:08 -07:00
snapshot.h GetSnapshot() and ReleaseSnapshot() to move new and free out of DB mutex 2015-06-08 21:57:02 -07:00
table_cache.cc TableMock + framework for mock classes 2014-10-28 17:52:32 -07:00
table_cache.h use GetContext to replace callback function pointer 2014-09-29 11:09:09 -07:00
table_properties_collector.cc A new call back to TablePropertiesCollector to allow users know the entry is add, delete or merge 2015-04-06 10:27:21 -07:00
table_properties_collector.h Add TablePropertiesCollector::NeedCompact() to suggest DB to further compact output files 2015-06-05 20:18:21 -07:00
table_properties_collector_test.cc A new call back to TablePropertiesCollector to allow users know the entry is add, delete or merge 2015-04-06 10:27:21 -07:00
transaction_log_impl.cc Turn -Wshadow back on 2014-11-06 11:14:28 -08:00
transaction_log_impl.h rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
version_builder.cc Fix deleting obsolete files 2015-02-06 08:44:30 -08:00
version_builder.h Move VersionBuilder logic to a separate .cc file 2014-10-31 16:34:38 -07:00
version_builder_test.cc Add TablePropertiesCollector::NeedCompact() to suggest DB to further compact output files 2015-06-05 20:18:21 -07:00
version_edit.cc Turn on -Wshadow 2014-10-31 11:59:54 -07:00
version_edit.h Add TablePropertiesCollector::NeedCompact() to suggest DB to further compact output files 2015-06-05 20:18:21 -07:00
version_edit_test.cc Add TablePropertiesCollector::NeedCompact() to suggest DB to further compact output files 2015-06-05 20:18:21 -07:00
version_set.cc Slow down writes by bytes written 2015-06-11 20:42:18 -07:00
version_set.h Allowing L0 -> L1 trivial move on sorted data 2015-06-04 16:51:25 -07:00
version_set_test.cc Fix level size overflow for options_.level_compaction_dynamic_level_bytes=true 2015-04-03 09:04:35 -07:00
wal_manager.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
wal_manager.h Fix -Wnon-virtual-dtor errors 2014-11-10 17:39:38 -05:00
wal_manager_test.cc Fix flakiness of WalManagerTest 2015-04-13 16:15:05 -07:00
write_batch.cc WriteBatch.Merge w/ SliceParts support 2015-05-29 04:30:03 -07:00
write_batch_base.cc WriteBatch.Merge w/ SliceParts support 2015-05-29 04:30:03 -07:00
write_batch_internal.h WriteBatch.Merge w/ SliceParts support 2015-05-29 04:30:03 -07:00
write_batch_test.cc Optimistic Transactions 2015-05-29 14:36:35 -07:00
write_callback.h Optimistic Transactions 2015-05-29 14:36:35 -07:00
write_callback_test.cc Optimistic Transactions 2015-05-29 14:36:35 -07:00
write_controller.cc Slow down writes by bytes written 2015-06-11 20:42:18 -07:00
write_controller.h Slow down writes by bytes written 2015-06-11 20:42:18 -07:00
write_controller_test.cc Slow down writes by bytes written 2015-06-11 20:42:18 -07:00
write_thread.cc Slow down writes by bytes written 2015-06-11 20:42:18 -07:00
write_thread.h Slow down writes by bytes written 2015-06-11 20:42:18 -07:00
writebuffer.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00