mirror of https://github.com/facebook/rocksdb.git
27f3af5966
Summary: TL;DR: due to a recent change, if you drop a column family, often that DB will no longer fsync after writing new SST files to remaining or new column families, which could lead to data loss on power loss. More bug detail: The intent of https://github.com/facebook/rocksdb/issues/10049 was to Close FSDirectory objects at DB::Close time rather than waiting for DB object destruction. Unfortunately, it also closes shared FSDirectory objects on DropColumnFamily (& destroy remaining handles), which can lead to use-after-Close on FSDirectory shared with remaining column families. Those "uses" are only Fsyncs (or redundant Closes). In the default Posix filesystem, an Fsync on a closed FSDirectory is a quiet no-op. Consequently (under most configurations), if you drop a column family, that DB will no longer fsync after writing new SST files to column families sharing the same directory (true under most configurations). More fix detail: Basically, this removes unnecessary Close ops on destroying ColumnFamilyData. We let `shared_ptr` take care of calling the destructor at the right time. If the intent was to require Close be called before destroying FSDirectory, that was not made clear by the author of FileSystem and was not at all enforced by https://github.com/facebook/rocksdb/issues/10049, which could have added `assert(fd_ == -1)` to `~PosixDirectory()` but did not. To keep this fix simple, we relax the unit test for https://github.com/facebook/rocksdb/issues/10049 to allow timely destruction of FSDirectory to suffice as Close (in CountedFileSystem). Added a TODO to revisit that. Also in this PR: * Added a TODO to share FSDirectory instances between DB and its column families. (Already shared among column families.) * Made DB::Close attempt to close all its open FSDirectory objects even if there is a failure in closing one. Also code clean-up around this logic. Pull Request resolved: https://github.com/facebook/rocksdb/pull/10460 Test Plan: add an assert to check for use-after-Close. With that existing tests can detect the misuse. With fix, tests pass (except noted relaxing of unit test for https://github.com/facebook/rocksdb/issues/10049) Reviewed By: ajkr Differential Revision: D38357922 Pulled By: pdillinger fbshipit-source-id: d42079cadbedf0a969f03389bf586b3b4e1f9137 |
||
|---|---|---|
| .. | ||
| composite_env.cc | ||
| composite_env_wrapper.h | ||
| emulated_clock.h | ||
| env.cc | ||
| env_basic_test.cc | ||
| env_chroot.cc | ||
| env_chroot.h | ||
| env_encryption.cc | ||
| env_encryption_ctr.h | ||
| env_posix.cc | ||
| env_test.cc | ||
| file_system.cc | ||
| file_system_tracer.cc | ||
| file_system_tracer.h | ||
| fs_posix.cc | ||
| fs_readonly.h | ||
| fs_remap.cc | ||
| fs_remap.h | ||
| io_posix.cc | ||
| io_posix.h | ||
| io_posix_test.cc | ||
| mock_env.cc | ||
| mock_env.h | ||
| mock_env_test.cc | ||
| unique_id_gen.cc | ||
| unique_id_gen.h | ||