rocksdb/utilities
mrambacher f35f7f2704 Fix many tests to run with MEM_ENV and ENCRYPTED_ENV; Introduce a MemoryFileSystem class (#7566)
Summary:
This PR does a few things:

1.  The MockFileSystem class was split out from the MockEnv.  This change would theoretically allow a MockFileSystem to be used by other Environments as well (if we created a means of constructing one).  The MockFileSystem implements a FileSystem in its entirety and does not rely on any Wrapper implementation.

2.  Make the RocksDB test suite work when MOCK_ENV=1 and ENCRYPTED_ENV=1 are set.  To accomplish this, a few things were needed:
- The tests that tried to use the "wrong" environment (Env::Default() instead of env_) were updated
- The MockFileSystem was changed to support the features it was missing or mishandled (such as recursively deleting files in a directory or supporting renaming of a directory).

3.  Updated the test framework to have a ROCKSDB_GTEST_SKIP macro.  This can be used to flag tests that are skipped.  Currently, this defaults to doing nothing (marks the test as SUCCESS) but will mark the tests as SKIPPED when RocksDB is upgraded to a version of gtest that supports this (gtest-1.10).

I have run a full "make check" with MEM_ENV, ENCRYPTED_ENV,  both, and neither under both MacOS and RedHat.  A few tests were disabled/skipped for the MEM/ENCRYPTED cases.  The error_handler_fs_test fails/hangs for MEM_ENV (presumably a timing problem) and I will introduce another PR/issue to track that problem.  (I will also push a change to disable those tests soon).  There is one more test in DBTest2 that also fails which I need to investigate or skip before this PR is merged.

Theoretically, this PR should also allow the test suite to run against an Env loaded from the registry, though I do not have one to try it with currently.

Finally, once this is accepted, it would be nice if there was a CircleCI job to run these tests on a checkin so this effort does not become stale.  I do not know how to do that, so if someone could write that job, it would be appreciated :)

Pull Request resolved: https://github.com/facebook/rocksdb/pull/7566

Reviewed By: zhichao-cao

Differential Revision: D24408980

Pulled By: jay-zhuang

fbshipit-source-id: 911b1554a4d0da06fd51feca0c090a4abdcb4a5f
2020-10-27 10:33:09 -07:00
..
backupable Make FileType Public and Replace kLogFile with kWalFile (#7580) 2020-10-22 17:06:20 -07:00
blob_db Introduce BlobFileCache and add support for blob files to Get() (#7540) 2020-10-15 13:04:47 -07:00
cassandra Replace reinterpret_cast with static_cast_with_check (#7067) 2020-07-02 19:25:41 -07:00
checkpoint Make FileType Public and Replace kLogFile with kWalFile (#7580) 2020-10-22 17:06:20 -07:00
compaction_filters Compaction filter support for BlobDB (#6850) 2020-06-29 17:32:14 -07:00
convenience Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
leveldb_options Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
memory Add few unit test cases in ASSERT_STATUS_CHECKED (#7500) 2020-10-08 11:22:44 -07:00
merge_operators Make StringAppendOperatorTest a parameterized test (#6930) 2020-06-04 14:17:11 -07:00
option_change_migration Whole DBTest to skip fsync (#7274) 2020-08-17 18:42:25 -07:00
options Revert Statuses returned from pre-Configurable options functions (#7563) 2020-10-20 11:53:28 -07:00
persistent_cache Fix many tests to run with MEM_ENV and ENCRYPTED_ENV; Introduce a MemoryFileSystem class (#7566) 2020-10-27 10:33:09 -07:00
simulator_cache Bring the Configurable options together (#5753) 2020-09-14 17:01:01 -07:00
table_properties_collectors Trigger compaction in CompactOnDeletionCollector based on deletion ratio (#6806) 2020-05-18 08:42:05 -07:00
trace Add some simulator cache and block tracer tests to ASSERT_STATUS_CHECKED (#7305) 2020-08-24 16:43:31 -07:00
transactions Fix unchecked statuses for transaction_test (#7572) 2020-10-21 14:03:59 -07:00
ttl DBWithTTL::Open() param ttls: vector<int32_t> to const vector<int32_t>& (#7196) 2020-08-24 16:24:16 -07:00
write_batch_with_index Fix write_batch_test when ASSERT_STATUS_CHECKED=1 (#7575) 2020-10-20 13:18:41 -07:00
debug.cc Change ParseInternalKey() to return Status instead of bool (#7457) 2020-09-30 19:16:47 -07:00
env_librados.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
env_librados.md Add EnvLibrados - RocksDB Env of RADOS (#1222) 2016-07-21 11:16:34 -07:00
env_librados_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
env_mirror.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
env_mirror_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
env_timed.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
env_timed_test.cc Make env*_test work with ASSERT_STATUS_CHECKED (#7176) 2020-07-28 22:59:48 -07:00
fault_injection_env.cc Support injecting read errors for RandomAccessFile when using FaultInjectionTestEnv (#7447) 2020-09-28 17:32:06 -07:00
fault_injection_env.h Status check enforcement for error_handler_fs_test (#7342) 2020-10-02 16:41:13 -07:00
fault_injection_fs.cc Add EnvTestWithParam::OptionsTest to the ASSERT_STATUS_CHECKED passes (#7283) 2020-08-20 19:18:35 -07:00
fault_injection_fs.h Status check enforcement for error_handler_fs_test (#7342) 2020-10-02 16:41:13 -07:00
merge_operators.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
object_registry.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
object_registry_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
util_merge_operators_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00