dolysis
/
rocksdb
mirror of https://github.com/facebook/rocksdb.git
3
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
rocksdb/TARGETS

541 lines
21 KiB
Plaintext
Raw Normal View History

Make TARGETS file portable Summary: Instead of hard coding the path of the internal repo. Make TARGETS file work anywhere in fbcode Closes https://github.com/facebook/rocksdb/pull/2586 Differential Revision: D5428122 Pulled By: IslamAbdelRahman fbshipit-source-id: 21adec82bfbff14ea93532bee789b5f5bbee5b01
2017-07-14 22:36:20 +00:00
import os
TARGETS_PATH = os.path.dirname(__file__)
REPO_PATH = TARGETS_PATH[(TARGETS_PATH.find('fbcode/') + len('fbcode/')):] + "/"
Add buckifier script to github repo Summary: Add buckifier script and TARGETS file to github repo Closes https://github.com/facebook/rocksdb/pull/2083 Differential Revision: D4825822 Pulled By: IslamAbdelRahman fbshipit-source-id: 205f4a7
2017-04-04 23:09:31 +00:00
BUCK_BINS = "buck-out/gen/" + REPO_PATH
TEST_RUNNER = REPO_PATH + "buckifier/rocks_test_runner.sh"
rocksdb_compiler_flags = [
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
"-fno-builtin-memcmp",
"-DROCKSDB_PLATFORM_POSIX",
"-DROCKSDB_LIB_IO_POSIX",
"-DROCKSDB_FALLOCATE_PRESENT",
"-DROCKSDB_MALLOC_USABLE_SIZE",
"-DROCKSDB_RANGESYNC_PRESENT",
"-DROCKSDB_SCHED_GETCPU_PRESENT",
Fix undefined behavior in Hash Summary: Instead of ignoring UBSan checks, fix the negative shifts in Hash(). Also add test to make sure the hash values are stable over time. The values were computed before this change, so the test also verifies the correctness of the change. Closes https://github.com/facebook/rocksdb/pull/2546 Differential Revision: D5386369 Pulled By: yiwu-arbug fbshipit-source-id: 6de4b44461a544d6222cc5d72d8cda2c0373d17e
2017-07-10 19:22:26 +00:00
"-DROCKSDB_SUPPORT_THREAD_LOCAL",
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
"-DOS_LINUX",
# Flags to enable libs we include
"-DSNAPPY",
"-DZLIB",
"-DBZIP2",
"-DLZ4",
"-DZSTD",
"-DGFLAGS=gflags",
"-DNUMA",
"-DTBB",
# Needed to compile in fbcode
"-Wno-expansion-to-defined",
Add buckifier script to github repo Summary: Add buckifier script and TARGETS file to github repo Closes https://github.com/facebook/rocksdb/pull/2083 Differential Revision: D4825822 Pulled By: IslamAbdelRahman fbshipit-source-id: 205f4a7
2017-04-04 23:09:31 +00:00
]
rocksdb_external_deps = [
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
('bzip2', None, 'bz2'),
('snappy', None, "snappy"),
('zlib', None, 'z'),
('gflags', None, 'gflags'),
('lz4', None, 'lz4'),
('zstd', None),
('tbb', None),
allow numa >= 2.0.8 Summary: Allow numa >= 2.0.8 in buck TARGET file. Closes https://github.com/facebook/rocksdb/pull/2504 Differential Revision: D5330550 Pulled By: yiwu-arbug fbshipit-source-id: 8ffb6167b4ad913877eac16a20a91023b31f8d41
2017-06-27 18:25:00 +00:00
("numa", None, "numa"),
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
("googletest", None, "gtest"),
Add buckifier script to github repo Summary: Add buckifier script and TARGETS file to github repo Closes https://github.com/facebook/rocksdb/pull/2083 Differential Revision: D4825822 Pulled By: IslamAbdelRahman fbshipit-source-id: 205f4a7
2017-04-04 23:09:31 +00:00
]
rocksdb_preprocessor_flags = [
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
# Directories with files for #include
"-I" + REPO_PATH + "include/",
"-I" + REPO_PATH,
Add buckifier script to github repo Summary: Add buckifier script and TARGETS file to github repo Closes https://github.com/facebook/rocksdb/pull/2083 Differential Revision: D4825822 Pulled By: IslamAbdelRahman fbshipit-source-id: 205f4a7
2017-04-04 23:09:31 +00:00
]
rocksdb: make buildable on aarch64 Summary: - Remove default arch-specified flags. - Move non-default arch-specific flags to arch-specific param. Reviewed By: yiwu-arbug Differential Revision: D5597499 fbshipit-source-id: c53108ac39c73ac36893d3fd9aaf3b5e3080f1ae
2017-08-14 00:04:19 +00:00
rocksdb_arch_preprocessor_flags = {
"x86_64": ["-DHAVE_SSE42"],
}
Add buckifier script to github repo Summary: Add buckifier script and TARGETS file to github repo Closes https://github.com/facebook/rocksdb/pull/2083 Differential Revision: D4825822 Pulled By: IslamAbdelRahman fbshipit-source-id: 205f4a7
2017-04-04 23:09:31 +00:00
cpp_library(
name = "rocksdb_lib",
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
headers = AutoHeaders.RECURSIVE_GLOB,
Add buckifier script to github repo Summary: Add buckifier script and TARGETS file to github repo Closes https://github.com/facebook/rocksdb/pull/2083 Differential Revision: D4825822 Pulled By: IslamAbdelRahman fbshipit-source-id: 205f4a7
2017-04-04 23:09:31 +00:00
srcs = [
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
"cache/clock_cache.cc",
"cache/lru_cache.cc",
"cache/sharded_cache.cc",
"db/builder.cc",
"db/c.cc",
"db/column_family.cc",
"db/compacted_db_impl.cc",
"db/compaction.cc",
"db/compaction_iterator.cc",
"db/compaction_job.cc",
"db/compaction_picker.cc",
"db/compaction_picker_universal.cc",
"db/convenience.cc",
"db/db_filesnapshot.cc",
"db/db_impl.cc",
"db/db_impl_write.cc",
"db/db_impl_compaction_flush.cc",
"db/db_impl_files.cc",
"db/db_impl_open.cc",
"db/db_impl_debug.cc",
"db/db_impl_experimental.cc",
"db/db_impl_readonly.cc",
"db/db_info_dumper.cc",
"db/db_iter.cc",
"db/dbformat.cc",
"db/event_helpers.cc",
"db/experimental.cc",
"db/external_sst_file_ingestion_job.cc",
"db/file_indexer.cc",
"db/flush_job.cc",
"db/flush_scheduler.cc",
"db/forward_iterator.cc",
"db/internal_stats.cc",
"db/log_reader.cc",
"db/log_writer.cc",
"db/malloc_stats.cc",
"db/managed_iterator.cc",
"db/memtable.cc",
"db/memtable_list.cc",
"db/merge_helper.cc",
"db/merge_operator.cc",
"db/range_del_aggregator.cc",
"db/repair.cc",
"db/snapshot_impl.cc",
"db/table_cache.cc",
"db/table_properties_collector.cc",
"db/transaction_log_impl.cc",
"db/version_builder.cc",
"db/version_edit.cc",
"db/version_set.cc",
"db/wal_manager.cc",
"db/write_batch.cc",
"db/write_batch_base.cc",
"db/write_controller.cc",
"db/write_thread.cc",
"env/env.cc",
"env/env_chroot.cc",
Encryption at rest support Summary: This PR adds support for encrypting data stored by RocksDB when written to disk. It adds an `EncryptedEnv` override of the `Env` class with matching overrides for sequential&random access files. The encryption itself is done through a configurable `EncryptionProvider`. This class creates is asked to create `BlockAccessCipherStream` for a file. This is where the actual encryption/decryption is being done. Currently there is a Counter mode implementation of `BlockAccessCipherStream` with a `ROT13` block cipher (NOTE the `ROT13` is for demo purposes only!!). The Counter operation mode uses an initial counter & random initialization vector (IV). Both are created randomly for each file and stored in a 4K (default size) block that is prefixed to that file. The `EncryptedEnv` implementation is such that clients of the `Env` class do not see this prefix (nor data, nor in filesize). The largest part of the prefix block is also encrypted, and there is room left for implementation specific settings/values/keys in there. To test the encryption, the `DBTestBase` class has been extended to consider a new environment variable called `ENCRYPTED_ENV`. If set, the test will setup a encrypted instance of the `Env` class to use for all tests. Typically you would run it like this: ``` ENCRYPTED_ENV=1 make check_some ``` There is also an added test that checks that some data inserted into the database is or is not "visible" on disk. With `ENCRYPTED_ENV` active it must not find plain text strings, with `ENCRYPTED_ENV` unset, it must find the plain text strings. Closes https://github.com/facebook/rocksdb/pull/2424 Differential Revision: D5322178 Pulled By: sdwilsh fbshipit-source-id: 253b0a9c2c498cc98f580df7f2623cbf7678a27f
2017-06-26 23:52:06 +00:00
"env/env_encryption.cc",
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
"env/env_hdfs.cc",
"env/env_posix.cc",
"env/io_posix.cc",
"env/mock_env.cc",
"memtable/alloc_tracker.cc",
"memtable/hash_cuckoo_rep.cc",
"memtable/hash_linklist_rep.cc",
"memtable/hash_skiplist_rep.cc",
"memtable/skiplistrep.cc",
"memtable/vectorrep.cc",
"memtable/write_buffer_manager.cc",
"monitoring/histogram.cc",
"monitoring/histogram_windowing.cc",
"monitoring/instrumented_mutex.cc",
"monitoring/iostats_context.cc",
"monitoring/perf_context.cc",
"monitoring/perf_level.cc",
"monitoring/statistics.cc",
"monitoring/thread_status_impl.cc",
"monitoring/thread_status_updater.cc",
"monitoring/thread_status_updater_debug.cc",
"monitoring/thread_status_util.cc",
"monitoring/thread_status_util_debug.cc",
"options/cf_options.cc",
"options/db_options.cc",
"options/options.cc",
"options/options_helper.cc",
"options/options_parser.cc",
"options/options_sanity_check.cc",
"port/port_posix.cc",
"port/stack_trace.cc",
"table/adaptive_table_factory.cc",
"table/block.cc",
"table/block_based_filter_block.cc",
"table/block_based_table_builder.cc",
"table/block_based_table_factory.cc",
"table/block_based_table_reader.cc",
"table/block_builder.cc",
"table/block_prefix_index.cc",
"table/bloom_block.cc",
"table/cuckoo_table_builder.cc",
"table/cuckoo_table_factory.cc",
"table/cuckoo_table_reader.cc",
"table/flush_block_policy.cc",
"table/format.cc",
"table/full_filter_block.cc",
"table/get_context.cc",
"table/index_builder.cc",
"table/iterator.cc",
"table/merging_iterator.cc",
"table/meta_blocks.cc",
"table/partitioned_filter_block.cc",
"table/persistent_cache_helper.cc",
"table/plain_table_builder.cc",
"table/plain_table_factory.cc",
"table/plain_table_index.cc",
"table/plain_table_key_coding.cc",
"table/plain_table_reader.cc",
"table/sst_file_writer.cc",
"table/table_properties.cc",
"table/two_level_iterator.cc",
"tools/dump/db_dump_tool.cc",
"util/arena.cc",
"util/auto_roll_logger.cc",
"util/bloom.cc",
"util/build_version.cc",
"util/coding.cc",
"util/compaction_job_stats_impl.cc",
"util/comparator.cc",
"util/concurrent_arena.cc",
"util/crc32c.cc",
"util/delete_scheduler.cc",
"util/dynamic_bloom.cc",
"util/event_logger.cc",
"util/file_reader_writer.cc",
"util/file_util.cc",
"util/filename.cc",
"util/filter_policy.cc",
"util/hash.cc",
"util/log_buffer.cc",
"util/murmurhash.cc",
"util/random.cc",
"util/rate_limiter.cc",
"util/slice.cc",
"util/sst_file_manager_impl.cc",
"util/status.cc",
"util/status_message.cc",
"util/string_util.cc",
"util/sync_point.cc",
"util/thread_local.cc",
"util/threadpool_imp.cc",
"util/transaction_test_util.cc",
"util/xxhash.cc",
"utilities/backupable/backupable_db.cc",
"utilities/blob_db/blob_db.cc",
"utilities/blob_db/blob_db_impl.cc",
"utilities/blob_db/blob_file.cc",
"utilities/blob_db/blob_log_reader.cc",
"utilities/blob_db/blob_log_writer.cc",
"utilities/blob_db/blob_log_format.cc",
Blob DB TTL extractor Summary: Introducing blob_db::TTLExtractor to replace extract_ttl_fn. The TTL extractor can be use to extract TTL from keys insert with Put or WriteBatch. Change over existing extract_ttl_fn are: * If value is changed, it will be return via std::string* (rather than Slice*). With Slice* the new value has to be part of the existing value. With std::string* the limitation is removed. * It can optionally return TTL or expiration. Other changes in this PR: * replace `std::chrono::system_clock` with `Env::NowMicros` so that I can mock time in tests. * add several TTL tests. * other minor naming change. Closes https://github.com/facebook/rocksdb/pull/2659 Differential Revision: D5512627 Pulled By: yiwu-arbug fbshipit-source-id: 0dfcb00d74d060b8534c6130c808e4d5d0a54440
2017-07-28 06:16:18 +00:00
"utilities/blob_db/ttl_extractor.cc",
Cassandra compaction filter for purge expired columns and rows Summary: Major changes in this PR: * Implement CassandraCompactionFilter to remove expired columns and rows (if all column expired) * Move cassandra related code from utilities/merge_operators/cassandra to utilities/cassandra/* * Switch to use shared_ptr<> from uniqu_ptr for Column membership management in RowValue. Since columns do have multiple owners in Merge and GC process, use shared_ptr helps make RowValue immutable. * Rename cassandra_merge_test to cassandra_functional_test and add two TTL compaction related tests there. Closes https://github.com/facebook/rocksdb/pull/2588 Differential Revision: D5430010 Pulled By: wpc fbshipit-source-id: 9566c21e06de17491d486a68c70f52d501f27687
2017-07-21 21:42:32 +00:00
"utilities/cassandra/cassandra_compaction_filter.cc",
"utilities/cassandra/format.cc",
"utilities/cassandra/merge_operator.cc",
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
"utilities/checkpoint/checkpoint_impl.cc",
"utilities/compaction_filters/remove_emptyvalue_compactionfilter.cc",
"utilities/convenience/info_log_finder.cc",
"utilities/date_tiered/date_tiered_db_impl.cc",
"utilities/debug.cc",
"utilities/document/document_db.cc",
"utilities/document/json_document.cc",
"utilities/document/json_document_builder.cc",
"utilities/env_mirror.cc",
"utilities/env_timed.cc",
"utilities/geodb/geodb_impl.cc",
"utilities/leveldb_options/leveldb_options.cc",
"utilities/lua/rocks_lua_compaction_filter.cc",
"utilities/memory/memory_util.cc",
allow numa >= 2.0.8 Summary: Allow numa >= 2.0.8 in buck TARGET file. Closes https://github.com/facebook/rocksdb/pull/2504 Differential Revision: D5330550 Pulled By: yiwu-arbug fbshipit-source-id: 8ffb6167b4ad913877eac16a20a91023b31f8d41
2017-06-27 18:25:00 +00:00
"utilities/merge_operators/max.cc",
"utilities/merge_operators/put.cc",
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
"utilities/merge_operators/string_append/stringappend.cc",
"utilities/merge_operators/string_append/stringappend2.cc",
"utilities/merge_operators/uint64add.cc",
"utilities/option_change_migration/option_change_migration.cc",
"utilities/options/options_util.cc",
"utilities/persistent_cache/block_cache_tier.cc",
"utilities/persistent_cache/block_cache_tier_file.cc",
"utilities/persistent_cache/block_cache_tier_metadata.cc",
"utilities/persistent_cache/persistent_cache_tier.cc",
"utilities/persistent_cache/volatile_tier_impl.cc",
"utilities/redis/redis_lists.cc",
"utilities/simulator_cache/sim_cache.cc",
"utilities/spatialdb/spatial_db.cc",
"utilities/table_properties_collectors/compact_on_deletion_collector.cc",
"utilities/transactions/optimistic_transaction_db_impl.cc",
Refactor PessimisticTransaction Summary: This patch splits Commit and Prepare into lock-related logic and db-write-related logic. It moves lock-related logic to PessimisticTransaction to be reused by all children classes and movies the existing impl of db-write-related to PrepareInternal, CommitSingleInternal, and CommitInternal in WriteCommittedTxnImpl. Closes https://github.com/facebook/rocksdb/pull/2691 Differential Revision: D5569464 Pulled By: maysamyabandeh fbshipit-source-id: d1b8698e69801a4126c7bc211745d05c636f5325
2017-08-07 23:07:40 +00:00
"utilities/transactions/optimistic_transaction.cc",
WritePrepared Txn: Compaction/Flush Summary: Update Compaction/Flush to support WritePreparedTxnDB: Add SnapshotChecker which is a proxy to query WritePreparedTxnDB::IsInSnapshot. Pass SnapshotChecker to DBImpl on WritePreparedTxnDB open. CompactionIterator use it to check if a key has been committed and if it is visible to a snapshot. In CompactionIterator: * check if key has been committed. If not, output uncommitted keys AS-IS. * use SnapshotChecker to check if key is visible to a snapshot when in need. * do not output key with seq = 0 if the key is not committed. Closes https://github.com/facebook/rocksdb/pull/2926 Differential Revision: D5902907 Pulled By: yiwu-arbug fbshipit-source-id: 945e037fdf0aa652dc5ba0ad879461040baa0320
2017-10-06 17:26:38 +00:00
"utilities/transactions/pessimistic_transaction.cc",
Refactor TransactionDBImpl Summary: This opens space for the new implementations of TransactionDBImpl such as WritePreparedTxnDBImpl that has a different policy of how to write to DB. Closes https://github.com/facebook/rocksdb/pull/2689 Differential Revision: D5568918 Pulled By: maysamyabandeh fbshipit-source-id: f7eac866e175daf3793ae79da108f65cc7dc7b25
2017-08-06 00:17:48 +00:00
"utilities/transactions/pessimistic_transaction_db.cc",
WritePrepared Txn: Compaction/Flush Summary: Update Compaction/Flush to support WritePreparedTxnDB: Add SnapshotChecker which is a proxy to query WritePreparedTxnDB::IsInSnapshot. Pass SnapshotChecker to DBImpl on WritePreparedTxnDB open. CompactionIterator use it to check if a key has been committed and if it is visible to a snapshot. In CompactionIterator: * check if key has been committed. If not, output uncommitted keys AS-IS. * use SnapshotChecker to check if key is visible to a snapshot when in need. * do not output key with seq = 0 if the key is not committed. Closes https://github.com/facebook/rocksdb/pull/2926 Differential Revision: D5902907 Pulled By: yiwu-arbug fbshipit-source-id: 945e037fdf0aa652dc5ba0ad879461040baa0320
2017-10-06 17:26:38 +00:00
"utilities/transactions/snapshot_checker.cc",
"utilities/transactions/transaction_base.cc",
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
"utilities/transactions/transaction_db_mutex_impl.cc",
"utilities/transactions/transaction_lock_mgr.cc",
"utilities/transactions/transaction_util.cc",
Refactor PessimisticTransaction Summary: This patch splits Commit and Prepare into lock-related logic and db-write-related logic. It moves lock-related logic to PessimisticTransaction to be reused by all children classes and movies the existing impl of db-write-related to PrepareInternal, CommitSingleInternal, and CommitInternal in WriteCommittedTxnImpl. Closes https://github.com/facebook/rocksdb/pull/2691 Differential Revision: D5569464 Pulled By: maysamyabandeh fbshipit-source-id: d1b8698e69801a4126c7bc211745d05c636f5325
2017-08-07 23:07:40 +00:00
"utilities/transactions/write_prepared_txn.cc",
WritePrepared Txn: Move DB class to its own file Summary: Move WritePreparedTxnDB from pessimistic_transaction_db.h to its own header, write_prepared_txn_db.h Closes https://github.com/facebook/rocksdb/pull/3114 Differential Revision: D6220987 Pulled By: maysamyabandeh fbshipit-source-id: 18893fb4fdc6b809fe117dabb544080f9b4a301b
2017-11-02 18:05:55 +00:00
"utilities/transactions/write_prepared_txn_db.cc",
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
"utilities/ttl/db_ttl_impl.cc",
"utilities/write_batch_with_index/write_batch_with_index.cc",
"utilities/write_batch_with_index/write_batch_with_index_internal.cc",
"tools/ldb_cmd.cc",
"tools/ldb_tool.cc",
"tools/sst_dump_tool.cc",
"utilities/blob_db/blob_dump_tool.cc",
Add buckifier script to github repo Summary: Add buckifier script and TARGETS file to github repo Closes https://github.com/facebook/rocksdb/pull/2083 Differential Revision: D4825822 Pulled By: IslamAbdelRahman fbshipit-source-id: 205f4a7
2017-04-04 23:09:31 +00:00
],
codemod: format TARGETS with buildifier [5/5] (D5092623) Reviewed By: igorsugak fbshipit-source-id: 906b744c179eb932f5a388b39f93209cecd50a80
2017-06-02 00:48:17 +00:00
deps = [],
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
preprocessor_flags = rocksdb_preprocessor_flags,
rocksdb: make buildable on aarch64 Summary: - Remove default arch-specified flags. - Move non-default arch-specific flags to arch-specific param. Reviewed By: yiwu-arbug Differential Revision: D5597499 fbshipit-source-id: c53108ac39c73ac36893d3fd9aaf3b5e3080f1ae
2017-08-14 00:04:19 +00:00
arch_preprocessor_flags = rocksdb_arch_preprocessor_flags,
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
compiler_flags = rocksdb_compiler_flags,
Add buckifier script to github repo Summary: Add buckifier script and TARGETS file to github repo Closes https://github.com/facebook/rocksdb/pull/2083 Differential Revision: D4825822 Pulled By: IslamAbdelRahman fbshipit-source-id: 205f4a7
2017-04-04 23:09:31 +00:00
external_deps = rocksdb_external_deps,
)
cpp_library(
name = "rocksdb_test_lib",
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
headers = AutoHeaders.RECURSIVE_GLOB,
Add buckifier script to github repo Summary: Add buckifier script and TARGETS file to github repo Closes https://github.com/facebook/rocksdb/pull/2083 Differential Revision: D4825822 Pulled By: IslamAbdelRahman fbshipit-source-id: 205f4a7
2017-04-04 23:09:31 +00:00
srcs = [
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
"table/mock_table.cc",
"util/fault_injection_test_env.cc",
"util/testharness.cc",
"util/testutil.cc",
"db/db_test_util.cc",
Cassandra compaction filter for purge expired columns and rows Summary: Major changes in this PR: * Implement CassandraCompactionFilter to remove expired columns and rows (if all column expired) * Move cassandra related code from utilities/merge_operators/cassandra to utilities/cassandra/* * Switch to use shared_ptr<> from uniqu_ptr for Column membership management in RowValue. Since columns do have multiple owners in Merge and GC process, use shared_ptr helps make RowValue immutable. * Rename cassandra_merge_test to cassandra_functional_test and add two TTL compaction related tests there. Closes https://github.com/facebook/rocksdb/pull/2588 Differential Revision: D5430010 Pulled By: wpc fbshipit-source-id: 9566c21e06de17491d486a68c70f52d501f27687
2017-07-21 21:42:32 +00:00
"utilities/cassandra/test_utils.cc",
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
"utilities/col_buf_encoder.cc",
"utilities/col_buf_decoder.cc",
"utilities/column_aware_encoding_util.cc",
Add buckifier script to github repo Summary: Add buckifier script and TARGETS file to github repo Closes https://github.com/facebook/rocksdb/pull/2083 Differential Revision: D4825822 Pulled By: IslamAbdelRahman fbshipit-source-id: 205f4a7
2017-04-04 23:09:31 +00:00
],
codemod: format TARGETS with buildifier [5/5] (D5092623) Reviewed By: igorsugak fbshipit-source-id: 906b744c179eb932f5a388b39f93209cecd50a80
2017-06-02 00:48:17 +00:00
deps = [":rocksdb_lib"],
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
preprocessor_flags = rocksdb_preprocessor_flags,
rocksdb: make buildable on aarch64 Summary: - Remove default arch-specified flags. - Move non-default arch-specific flags to arch-specific param. Reviewed By: yiwu-arbug Differential Revision: D5597499 fbshipit-source-id: c53108ac39c73ac36893d3fd9aaf3b5e3080f1ae
2017-08-14 00:04:19 +00:00
arch_preprocessor_flags = rocksdb_arch_preprocessor_flags,
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
compiler_flags = rocksdb_compiler_flags,
Add buckifier script to github repo Summary: Add buckifier script and TARGETS file to github repo Closes https://github.com/facebook/rocksdb/pull/2083 Differential Revision: D4825822 Pulled By: IslamAbdelRahman fbshipit-source-id: 205f4a7
2017-04-04 23:09:31 +00:00
external_deps = rocksdb_external_deps,
)
cpp_library(
name = "rocksdb_tools_lib",
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
headers = AutoHeaders.RECURSIVE_GLOB,
Add buckifier script to github repo Summary: Add buckifier script and TARGETS file to github repo Closes https://github.com/facebook/rocksdb/pull/2083 Differential Revision: D4825822 Pulled By: IslamAbdelRahman fbshipit-source-id: 205f4a7
2017-04-04 23:09:31 +00:00
srcs = [
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
"tools/db_bench_tool.cc",
"util/testutil.cc",
Add buckifier script to github repo Summary: Add buckifier script and TARGETS file to github repo Closes https://github.com/facebook/rocksdb/pull/2083 Differential Revision: D4825822 Pulled By: IslamAbdelRahman fbshipit-source-id: 205f4a7
2017-04-04 23:09:31 +00:00
],
codemod: format TARGETS with buildifier [5/5] (D5092623) Reviewed By: igorsugak fbshipit-source-id: 906b744c179eb932f5a388b39f93209cecd50a80
2017-06-02 00:48:17 +00:00
deps = [":rocksdb_lib"],
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
preprocessor_flags = rocksdb_preprocessor_flags,
rocksdb: make buildable on aarch64 Summary: - Remove default arch-specified flags. - Move non-default arch-specific flags to arch-specific param. Reviewed By: yiwu-arbug Differential Revision: D5597499 fbshipit-source-id: c53108ac39c73ac36893d3fd9aaf3b5e3080f1ae
2017-08-14 00:04:19 +00:00
arch_preprocessor_flags = rocksdb_arch_preprocessor_flags,
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
compiler_flags = rocksdb_compiler_flags,
Add buckifier script to github repo Summary: Add buckifier script and TARGETS file to github repo Closes https://github.com/facebook/rocksdb/pull/2083 Differential Revision: D4825822 Pulled By: IslamAbdelRahman fbshipit-source-id: 205f4a7
2017-04-04 23:09:31 +00:00
external_deps = rocksdb_external_deps,
)
cpp_library(
name = "env_basic_test_lib",
codemod: format TARGETS with buildifier [5/5] (D5092623) Reviewed By: igorsugak fbshipit-source-id: 906b744c179eb932f5a388b39f93209cecd50a80
2017-06-02 00:48:17 +00:00
headers = AutoHeaders.RECURSIVE_GLOB,
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
srcs = ["env/env_basic_test.cc"],
codemod: format TARGETS with buildifier [5/5] (D5092623) Reviewed By: igorsugak fbshipit-source-id: 906b744c179eb932f5a388b39f93209cecd50a80
2017-06-02 00:48:17 +00:00
deps = [":rocksdb_test_lib"],
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
preprocessor_flags = rocksdb_preprocessor_flags,
rocksdb: make buildable on aarch64 Summary: - Remove default arch-specified flags. - Move non-default arch-specific flags to arch-specific param. Reviewed By: yiwu-arbug Differential Revision: D5597499 fbshipit-source-id: c53108ac39c73ac36893d3fd9aaf3b5e3080f1ae
2017-08-14 00:04:19 +00:00
arch_preprocessor_flags = rocksdb_arch_preprocessor_flags,
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
compiler_flags = rocksdb_compiler_flags,
Add buckifier script to github repo Summary: Add buckifier script and TARGETS file to github repo Closes https://github.com/facebook/rocksdb/pull/2083 Differential Revision: D4825822 Pulled By: IslamAbdelRahman fbshipit-source-id: 205f4a7
2017-04-04 23:09:31 +00:00
external_deps = rocksdb_external_deps,
)
# [test_name, test_src, test_type]
Fix TARGETS file tests list Summary: 1. The buckifier script assume each test "foo" comes with a .cc file of the same name (i.e. foo.cc). Update cassandra tests to follow this pattern so that the buckifier script can recognize them. 2. add blob_db_test Closes https://github.com/facebook/rocksdb/pull/2506 Differential Revision: D5331517 Pulled By: yiwu-arbug fbshipit-source-id: 86f3eba471fc621186ab44cbd073b6162cde8e57
2017-06-27 21:01:14 +00:00
ROCKS_TESTS = [['arena_test', 'util/arena_test.cc', 'serial'],
['auto_roll_logger_test', 'util/auto_roll_logger_test.cc', 'serial'],
['autovector_test', 'util/autovector_test.cc', 'serial'],
['backupable_db_test',
'utilities/backupable/backupable_db_test.cc',
'parallel'],
['blob_db_test', 'utilities/blob_db/blob_db_test.cc', 'serial'],
['block_based_filter_block_test',
'table/block_based_filter_block_test.cc',
'serial'],
['block_test', 'table/block_test.cc', 'serial'],
['bloom_test', 'util/bloom_test.cc', 'serial'],
['c_test', 'db/c_test.c', 'serial'],
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
['cache_test', 'cache/cache_test.cc', 'serial'],
Fix TARGETS file tests list Summary: 1. The buckifier script assume each test "foo" comes with a .cc file of the same name (i.e. foo.cc). Update cassandra tests to follow this pattern so that the buckifier script can recognize them. 2. add blob_db_test Closes https://github.com/facebook/rocksdb/pull/2506 Differential Revision: D5331517 Pulled By: yiwu-arbug fbshipit-source-id: 86f3eba471fc621186ab44cbd073b6162cde8e57
2017-06-27 21:01:14 +00:00
['cassandra_format_test',
Cassandra compaction filter for purge expired columns and rows Summary: Major changes in this PR: * Implement CassandraCompactionFilter to remove expired columns and rows (if all column expired) * Move cassandra related code from utilities/merge_operators/cassandra to utilities/cassandra/* * Switch to use shared_ptr<> from uniqu_ptr for Column membership management in RowValue. Since columns do have multiple owners in Merge and GC process, use shared_ptr helps make RowValue immutable. * Rename cassandra_merge_test to cassandra_functional_test and add two TTL compaction related tests there. Closes https://github.com/facebook/rocksdb/pull/2588 Differential Revision: D5430010 Pulled By: wpc fbshipit-source-id: 9566c21e06de17491d486a68c70f52d501f27687
2017-07-21 21:42:32 +00:00
'utilities/cassandra/cassandra_format_test.cc',
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
'serial'],
Cassandra compaction filter for purge expired columns and rows Summary: Major changes in this PR: * Implement CassandraCompactionFilter to remove expired columns and rows (if all column expired) * Move cassandra related code from utilities/merge_operators/cassandra to utilities/cassandra/* * Switch to use shared_ptr<> from uniqu_ptr for Column membership management in RowValue. Since columns do have multiple owners in Merge and GC process, use shared_ptr helps make RowValue immutable. * Rename cassandra_merge_test to cassandra_functional_test and add two TTL compaction related tests there. Closes https://github.com/facebook/rocksdb/pull/2588 Differential Revision: D5430010 Pulled By: wpc fbshipit-source-id: 9566c21e06de17491d486a68c70f52d501f27687
2017-07-21 21:42:32 +00:00
['cassandra_functional_test',
'utilities/cassandra/cassandra_functional_test.cc',
allow numa >= 2.0.8 Summary: Allow numa >= 2.0.8 in buck TARGET file. Closes https://github.com/facebook/rocksdb/pull/2504 Differential Revision: D5330550 Pulled By: yiwu-arbug fbshipit-source-id: 8ffb6167b4ad913877eac16a20a91023b31f8d41
2017-06-27 18:25:00 +00:00
'serial'],
Fix TARGETS file tests list Summary: 1. The buckifier script assume each test "foo" comes with a .cc file of the same name (i.e. foo.cc). Update cassandra tests to follow this pattern so that the buckifier script can recognize them. 2. add blob_db_test Closes https://github.com/facebook/rocksdb/pull/2506 Differential Revision: D5331517 Pulled By: yiwu-arbug fbshipit-source-id: 86f3eba471fc621186ab44cbd073b6162cde8e57
2017-06-27 21:01:14 +00:00
['cassandra_row_merge_test',
Cassandra compaction filter for purge expired columns and rows Summary: Major changes in this PR: * Implement CassandraCompactionFilter to remove expired columns and rows (if all column expired) * Move cassandra related code from utilities/merge_operators/cassandra to utilities/cassandra/* * Switch to use shared_ptr<> from uniqu_ptr for Column membership management in RowValue. Since columns do have multiple owners in Merge and GC process, use shared_ptr helps make RowValue immutable. * Rename cassandra_merge_test to cassandra_functional_test and add two TTL compaction related tests there. Closes https://github.com/facebook/rocksdb/pull/2588 Differential Revision: D5430010 Pulled By: wpc fbshipit-source-id: 9566c21e06de17491d486a68c70f52d501f27687
2017-07-21 21:42:32 +00:00
'utilities/cassandra/cassandra_row_merge_test.cc',
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
'serial'],
Fix TARGETS file tests list Summary: 1. The buckifier script assume each test "foo" comes with a .cc file of the same name (i.e. foo.cc). Update cassandra tests to follow this pattern so that the buckifier script can recognize them. 2. add blob_db_test Closes https://github.com/facebook/rocksdb/pull/2506 Differential Revision: D5331517 Pulled By: yiwu-arbug fbshipit-source-id: 86f3eba471fc621186ab44cbd073b6162cde8e57
2017-06-27 21:01:14 +00:00
['cassandra_serialize_test',
Cassandra compaction filter for purge expired columns and rows Summary: Major changes in this PR: * Implement CassandraCompactionFilter to remove expired columns and rows (if all column expired) * Move cassandra related code from utilities/merge_operators/cassandra to utilities/cassandra/* * Switch to use shared_ptr<> from uniqu_ptr for Column membership management in RowValue. Since columns do have multiple owners in Merge and GC process, use shared_ptr helps make RowValue immutable. * Rename cassandra_merge_test to cassandra_functional_test and add two TTL compaction related tests there. Closes https://github.com/facebook/rocksdb/pull/2588 Differential Revision: D5430010 Pulled By: wpc fbshipit-source-id: 9566c21e06de17491d486a68c70f52d501f27687
2017-07-21 21:42:32 +00:00
'utilities/cassandra/cassandra_serialize_test.cc',
allow numa >= 2.0.8 Summary: Allow numa >= 2.0.8 in buck TARGET file. Closes https://github.com/facebook/rocksdb/pull/2504 Differential Revision: D5330550 Pulled By: yiwu-arbug fbshipit-source-id: 8ffb6167b4ad913877eac16a20a91023b31f8d41
2017-06-27 18:25:00 +00:00
'serial'],
Fix TARGETS file tests list Summary: 1. The buckifier script assume each test "foo" comes with a .cc file of the same name (i.e. foo.cc). Update cassandra tests to follow this pattern so that the buckifier script can recognize them. 2. add blob_db_test Closes https://github.com/facebook/rocksdb/pull/2506 Differential Revision: D5331517 Pulled By: yiwu-arbug fbshipit-source-id: 86f3eba471fc621186ab44cbd073b6162cde8e57
2017-06-27 21:01:14 +00:00
['checkpoint_test', 'utilities/checkpoint/checkpoint_test.cc', 'serial'],
['cleanable_test', 'table/cleanable_test.cc', 'serial'],
['coding_test', 'util/coding_test.cc', 'serial'],
['column_aware_encoding_test',
'utilities/column_aware_encoding_test.cc',
allow numa >= 2.0.8 Summary: Allow numa >= 2.0.8 in buck TARGET file. Closes https://github.com/facebook/rocksdb/pull/2504 Differential Revision: D5330550 Pulled By: yiwu-arbug fbshipit-source-id: 8ffb6167b4ad913877eac16a20a91023b31f8d41
2017-06-27 18:25:00 +00:00
'serial'],
Fix TARGETS file tests list Summary: 1. The buckifier script assume each test "foo" comes with a .cc file of the same name (i.e. foo.cc). Update cassandra tests to follow this pattern so that the buckifier script can recognize them. 2. add blob_db_test Closes https://github.com/facebook/rocksdb/pull/2506 Differential Revision: D5331517 Pulled By: yiwu-arbug fbshipit-source-id: 86f3eba471fc621186ab44cbd073b6162cde8e57
2017-06-27 21:01:14 +00:00
['column_family_test', 'db/column_family_test.cc', 'serial'],
['compact_files_test', 'db/compact_files_test.cc', 'serial'],
['compact_on_deletion_collector_test',
'utilities/table_properties_collectors/compact_on_deletion_collector_test.cc',
'serial'],
['compaction_iterator_test', 'db/compaction_iterator_test.cc', 'serial'],
['compaction_job_stats_test', 'db/compaction_job_stats_test.cc', 'serial'],
['compaction_job_test', 'db/compaction_job_test.cc', 'serial'],
['compaction_picker_test', 'db/compaction_picker_test.cc', 'serial'],
['comparator_db_test', 'db/comparator_db_test.cc', 'serial'],
['corruption_test', 'db/corruption_test.cc', 'serial'],
['crc32c_test', 'util/crc32c_test.cc', 'serial'],
['cuckoo_table_builder_test', 'table/cuckoo_table_builder_test.cc', 'serial'],
['cuckoo_table_db_test', 'db/cuckoo_table_db_test.cc', 'serial'],
['cuckoo_table_reader_test', 'table/cuckoo_table_reader_test.cc', 'serial'],
['date_tiered_test', 'utilities/date_tiered/date_tiered_test.cc', 'serial'],
['db_basic_test', 'db/db_basic_test.cc', 'serial'],
Add ValueType::kTypeBlobIndex Summary: Add kTypeBlobIndex value type, which will be used by blob db only, to insert a (key, blob_offset) KV pair. The purpose is to 1. Make it possible to open existing rocksdb instance as blob db. Existing value will be of kTypeIndex type, while value inserted by blob db will be of kTypeBlobIndex. 2. Make rocksdb able to detect if the db contains value written by blob db, if so return error. 3. Make it possible to have blob db optionally store value in SST file (with kTypeValue type) or as a blob value (with kTypeBlobIndex type). The root db (DBImpl) basically pretended kTypeBlobIndex are normal value on write. On Get if is_blob is provided, return whether the value read is of kTypeBlobIndex type, or return Status::NotSupported() status if is_blob is not provided. On scan allow_blob flag is pass and if the flag is true, return wether the value is of kTypeBlobIndex type via iter->IsBlob(). Changes on blob db side will be in a separate patch. Closes https://github.com/facebook/rocksdb/pull/2886 Differential Revision: D5838431 Pulled By: yiwu-arbug fbshipit-source-id: 3c5306c62bc13bb11abc03422ec5cbcea1203cca
2017-10-03 16:08:07 +00:00
['db_blob_index_test', 'db/db_blob_index_test.cc', 'serial'],
Fix TARGETS file tests list Summary: 1. The buckifier script assume each test "foo" comes with a .cc file of the same name (i.e. foo.cc). Update cassandra tests to follow this pattern so that the buckifier script can recognize them. 2. add blob_db_test Closes https://github.com/facebook/rocksdb/pull/2506 Differential Revision: D5331517 Pulled By: yiwu-arbug fbshipit-source-id: 86f3eba471fc621186ab44cbd073b6162cde8e57
2017-06-27 21:01:14 +00:00
['db_block_cache_test', 'db/db_block_cache_test.cc', 'serial'],
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
['db_bloom_filter_test', 'db/db_bloom_filter_test.cc', 'serial'],
Fix TARGETS file tests list Summary: 1. The buckifier script assume each test "foo" comes with a .cc file of the same name (i.e. foo.cc). Update cassandra tests to follow this pattern so that the buckifier script can recognize them. 2. add blob_db_test Closes https://github.com/facebook/rocksdb/pull/2506 Differential Revision: D5331517 Pulled By: yiwu-arbug fbshipit-source-id: 86f3eba471fc621186ab44cbd073b6162cde8e57
2017-06-27 21:01:14 +00:00
['db_compaction_filter_test', 'db/db_compaction_filter_test.cc', 'parallel'],
['db_compaction_test', 'db/db_compaction_test.cc', 'parallel'],
['db_dynamic_level_test', 'db/db_dynamic_level_test.cc', 'serial'],
['db_encryption_test', 'db/db_encryption_test.cc', 'serial'],
['db_flush_test', 'db/db_flush_test.cc', 'serial'],
['db_inplace_update_test', 'db/db_inplace_update_test.cc', 'serial'],
['db_io_failure_test', 'db/db_io_failure_test.cc', 'serial'],
['db_iter_test', 'db/db_iter_test.cc', 'serial'],
['db_iterator_test', 'db/db_iterator_test.cc', 'serial'],
['db_log_iter_test', 'db/db_log_iter_test.cc', 'serial'],
['db_memtable_test', 'db/db_memtable_test.cc', 'serial'],
['db_merge_operator_test', 'db/db_merge_operator_test.cc', 'serial'],
['db_options_test', 'db/db_options_test.cc', 'serial'],
['db_properties_test', 'db/db_properties_test.cc', 'serial'],
['db_range_del_test', 'db/db_range_del_test.cc', 'serial'],
['db_sst_test', 'db/db_sst_test.cc', 'parallel'],
['db_statistics_test', 'db/db_statistics_test.cc', 'serial'],
['db_table_properties_test', 'db/db_table_properties_test.cc', 'serial'],
['db_tailing_iter_test', 'db/db_tailing_iter_test.cc', 'serial'],
['db_test', 'db/db_test.cc', 'parallel'],
['db_test2', 'db/db_test2.cc', 'serial'],
['db_universal_compaction_test',
'db/db_universal_compaction_test.cc',
'parallel'],
['db_wal_test', 'db/db_wal_test.cc', 'parallel'],
['db_write_test', 'db/db_write_test.cc', 'serial'],
['dbformat_test', 'db/dbformat_test.cc', 'serial'],
['delete_scheduler_test', 'util/delete_scheduler_test.cc', 'serial'],
['deletefile_test', 'db/deletefile_test.cc', 'serial'],
['document_db_test', 'utilities/document/document_db_test.cc', 'serial'],
['dynamic_bloom_test', 'util/dynamic_bloom_test.cc', 'serial'],
['env_basic_test', 'env/env_basic_test.cc', 'serial'],
['env_test', 'env/env_test.cc', 'serial'],
['env_timed_test', 'utilities/env_timed_test.cc', 'serial'],
['event_logger_test', 'util/event_logger_test.cc', 'serial'],
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
['external_sst_file_basic_test',
'db/external_sst_file_basic_test.cc',
'serial'],
Fix TARGETS file tests list Summary: 1. The buckifier script assume each test "foo" comes with a .cc file of the same name (i.e. foo.cc). Update cassandra tests to follow this pattern so that the buckifier script can recognize them. 2. add blob_db_test Closes https://github.com/facebook/rocksdb/pull/2506 Differential Revision: D5331517 Pulled By: yiwu-arbug fbshipit-source-id: 86f3eba471fc621186ab44cbd073b6162cde8e57
2017-06-27 21:01:14 +00:00
['external_sst_file_test', 'db/external_sst_file_test.cc', 'parallel'],
['fault_injection_test', 'db/fault_injection_test.cc', 'parallel'],
['file_indexer_test', 'db/file_indexer_test.cc', 'serial'],
['file_reader_writer_test', 'util/file_reader_writer_test.cc', 'serial'],
['filelock_test', 'util/filelock_test.cc', 'serial'],
['filename_test', 'db/filename_test.cc', 'serial'],
['flush_job_test', 'db/flush_job_test.cc', 'serial'],
['full_filter_block_test', 'table/full_filter_block_test.cc', 'serial'],
['geodb_test', 'utilities/geodb/geodb_test.cc', 'serial'],
['hash_table_test',
'utilities/persistent_cache/hash_table_test.cc',
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
'serial'],
Fix undefined behavior in Hash Summary: Instead of ignoring UBSan checks, fix the negative shifts in Hash(). Also add test to make sure the hash values are stable over time. The values were computed before this change, so the test also verifies the correctness of the change. Closes https://github.com/facebook/rocksdb/pull/2546 Differential Revision: D5386369 Pulled By: yiwu-arbug fbshipit-source-id: 6de4b44461a544d6222cc5d72d8cda2c0373d17e
2017-07-10 19:22:26 +00:00
['hash_test', 'util/hash_test.cc', 'serial'],
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
['heap_test', 'util/heap_test.cc', 'serial'],
Fix TARGETS file tests list Summary: 1. The buckifier script assume each test "foo" comes with a .cc file of the same name (i.e. foo.cc). Update cassandra tests to follow this pattern so that the buckifier script can recognize them. 2. add blob_db_test Closes https://github.com/facebook/rocksdb/pull/2506 Differential Revision: D5331517 Pulled By: yiwu-arbug fbshipit-source-id: 86f3eba471fc621186ab44cbd073b6162cde8e57
2017-06-27 21:01:14 +00:00
['histogram_test', 'monitoring/histogram_test.cc', 'serial'],
['inlineskiplist_test', 'memtable/inlineskiplist_test.cc', 'parallel'],
['iostats_context_test', 'monitoring/iostats_context_test.cc', 'serial'],
['json_document_test', 'utilities/document/json_document_test.cc', 'serial'],
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
['ldb_cmd_test', 'tools/ldb_cmd_test.cc', 'serial'],
Fix TARGETS file tests list Summary: 1. The buckifier script assume each test "foo" comes with a .cc file of the same name (i.e. foo.cc). Update cassandra tests to follow this pattern so that the buckifier script can recognize them. 2. add blob_db_test Closes https://github.com/facebook/rocksdb/pull/2506 Differential Revision: D5331517 Pulled By: yiwu-arbug fbshipit-source-id: 86f3eba471fc621186ab44cbd073b6162cde8e57
2017-06-27 21:01:14 +00:00
['listener_test', 'db/listener_test.cc', 'serial'],
['log_test', 'db/log_test.cc', 'serial'],
['lru_cache_test', 'cache/lru_cache_test.cc', 'serial'],
['manual_compaction_test', 'db/manual_compaction_test.cc', 'parallel'],
['memory_test', 'utilities/memory/memory_test.cc', 'serial'],
['memtable_list_test', 'db/memtable_list_test.cc', 'serial'],
['merge_helper_test', 'db/merge_helper_test.cc', 'serial'],
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
['merge_test', 'db/merge_test.cc', 'serial'],
Fix TARGETS file tests list Summary: 1. The buckifier script assume each test "foo" comes with a .cc file of the same name (i.e. foo.cc). Update cassandra tests to follow this pattern so that the buckifier script can recognize them. 2. add blob_db_test Closes https://github.com/facebook/rocksdb/pull/2506 Differential Revision: D5331517 Pulled By: yiwu-arbug fbshipit-source-id: 86f3eba471fc621186ab44cbd073b6162cde8e57
2017-06-27 21:01:14 +00:00
['merger_test', 'table/merger_test.cc', 'serial'],
['mock_env_test', 'env/mock_env_test.cc', 'serial'],
['object_registry_test', 'utilities/object_registry_test.cc', 'serial'],
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
['optimistic_transaction_test',
'utilities/transactions/optimistic_transaction_test.cc',
'serial'],
['option_change_migration_test',
'utilities/option_change_migration/option_change_migration_test.cc',
'serial'],
Fix TARGETS file tests list Summary: 1. The buckifier script assume each test "foo" comes with a .cc file of the same name (i.e. foo.cc). Update cassandra tests to follow this pattern so that the buckifier script can recognize them. 2. add blob_db_test Closes https://github.com/facebook/rocksdb/pull/2506 Differential Revision: D5331517 Pulled By: yiwu-arbug fbshipit-source-id: 86f3eba471fc621186ab44cbd073b6162cde8e57
2017-06-27 21:01:14 +00:00
['options_file_test', 'db/options_file_test.cc', 'serial'],
['options_settable_test', 'options/options_settable_test.cc', 'serial'],
['options_test', 'options/options_test.cc', 'serial'],
['options_util_test', 'utilities/options/options_util_test.cc', 'serial'],
['partitioned_filter_block_test',
'table/partitioned_filter_block_test.cc',
'serial'],
['perf_context_test', 'db/perf_context_test.cc', 'serial'],
['persistent_cache_test',
'utilities/persistent_cache/persistent_cache_test.cc',
'parallel'],
['plain_table_db_test', 'db/plain_table_db_test.cc', 'serial'],
['prefix_test', 'db/prefix_test.cc', 'serial'],
['range_del_aggregator_test', 'db/range_del_aggregator_test.cc', 'serial'],
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
['rate_limiter_test', 'util/rate_limiter_test.cc', 'serial'],
Fix TARGETS file tests list Summary: 1. The buckifier script assume each test "foo" comes with a .cc file of the same name (i.e. foo.cc). Update cassandra tests to follow this pattern so that the buckifier script can recognize them. 2. add blob_db_test Closes https://github.com/facebook/rocksdb/pull/2506 Differential Revision: D5331517 Pulled By: yiwu-arbug fbshipit-source-id: 86f3eba471fc621186ab44cbd073b6162cde8e57
2017-06-27 21:01:14 +00:00
['reduce_levels_test', 'tools/reduce_levels_test.cc', 'serial'],
['repair_test', 'db/repair_test.cc', 'serial'],
['sim_cache_test', 'utilities/simulator_cache/sim_cache_test.cc', 'serial'],
['skiplist_test', 'memtable/skiplist_test.cc', 'serial'],
['slice_transform_test', 'util/slice_transform_test.cc', 'serial'],
['spatial_db_test', 'utilities/spatialdb/spatial_db_test.cc', 'serial'],
['sst_dump_test', 'tools/sst_dump_test.cc', 'serial'],
['statistics_test', 'monitoring/statistics_test.cc', 'serial'],
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
['stringappend_test',
'utilities/merge_operators/string_append/stringappend_test.cc',
'serial'],
Fix TARGETS file tests list Summary: 1. The buckifier script assume each test "foo" comes with a .cc file of the same name (i.e. foo.cc). Update cassandra tests to follow this pattern so that the buckifier script can recognize them. 2. add blob_db_test Closes https://github.com/facebook/rocksdb/pull/2506 Differential Revision: D5331517 Pulled By: yiwu-arbug fbshipit-source-id: 86f3eba471fc621186ab44cbd073b6162cde8e57
2017-06-27 21:01:14 +00:00
['table_properties_collector_test',
'db/table_properties_collector_test.cc',
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
'serial'],
['table_test', 'table/table_test.cc', 'parallel'],
Fix TARGETS file tests list Summary: 1. The buckifier script assume each test "foo" comes with a .cc file of the same name (i.e. foo.cc). Update cassandra tests to follow this pattern so that the buckifier script can recognize them. 2. add blob_db_test Closes https://github.com/facebook/rocksdb/pull/2506 Differential Revision: D5331517 Pulled By: yiwu-arbug fbshipit-source-id: 86f3eba471fc621186ab44cbd073b6162cde8e57
2017-06-27 21:01:14 +00:00
['thread_list_test', 'util/thread_list_test.cc', 'serial'],
['thread_local_test', 'util/thread_local_test.cc', 'serial'],
['timer_queue_test', 'util/timer_queue_test.cc', 'serial'],
Add more unit test to write_prepared txns Summary: Closes https://github.com/facebook/rocksdb/pull/2798 Differential Revision: D5724173 Pulled By: maysamyabandeh fbshipit-source-id: fb6b782d933fb4be315b1a231a6a67a66fdc9c96
2017-08-31 16:27:14 +00:00
['transaction_test',
'utilities/transactions/transaction_test.cc',
'parallel'],
Fix TARGETS file tests list Summary: 1. The buckifier script assume each test "foo" comes with a .cc file of the same name (i.e. foo.cc). Update cassandra tests to follow this pattern so that the buckifier script can recognize them. 2. add blob_db_test Closes https://github.com/facebook/rocksdb/pull/2506 Differential Revision: D5331517 Pulled By: yiwu-arbug fbshipit-source-id: 86f3eba471fc621186ab44cbd073b6162cde8e57
2017-06-27 21:01:14 +00:00
['ttl_test', 'utilities/ttl/ttl_test.cc', 'serial'],
['util_merge_operators_test',
'utilities/util_merge_operators_test.cc',
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
'serial'],
Fix TARGETS file tests list Summary: 1. The buckifier script assume each test "foo" comes with a .cc file of the same name (i.e. foo.cc). Update cassandra tests to follow this pattern so that the buckifier script can recognize them. 2. add blob_db_test Closes https://github.com/facebook/rocksdb/pull/2506 Differential Revision: D5331517 Pulled By: yiwu-arbug fbshipit-source-id: 86f3eba471fc621186ab44cbd073b6162cde8e57
2017-06-27 21:01:14 +00:00
['version_builder_test', 'db/version_builder_test.cc', 'serial'],
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
['version_edit_test', 'db/version_edit_test.cc', 'serial'],
Fix TARGETS file tests list Summary: 1. The buckifier script assume each test "foo" comes with a .cc file of the same name (i.e. foo.cc). Update cassandra tests to follow this pattern so that the buckifier script can recognize them. 2. add blob_db_test Closes https://github.com/facebook/rocksdb/pull/2506 Differential Revision: D5331517 Pulled By: yiwu-arbug fbshipit-source-id: 86f3eba471fc621186ab44cbd073b6162cde8e57
2017-06-27 21:01:14 +00:00
['version_set_test', 'db/version_set_test.cc', 'serial'],
['wal_manager_test', 'db/wal_manager_test.cc', 'serial'],
['write_batch_test', 'db/write_batch_test.cc', 'serial'],
['write_batch_with_index_test',
'utilities/write_batch_with_index/write_batch_with_index_test.cc',
'serial'],
['write_buffer_manager_test',
'memtable/write_buffer_manager_test.cc',
'serial'],
['write_callback_test', 'db/write_callback_test.cc', 'serial'],
Add more unit test to write_prepared txns Summary: Closes https://github.com/facebook/rocksdb/pull/2798 Differential Revision: D5724173 Pulled By: maysamyabandeh fbshipit-source-id: fb6b782d933fb4be315b1a231a6a67a66fdc9c96
2017-08-31 16:27:14 +00:00
['write_controller_test', 'db/write_controller_test.cc', 'serial'],
['write_prepared_transaction_test',
'utilities/transactions/write_prepared_transaction_test.cc',
'serial']]
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
Add buckifier script to github repo Summary: Add buckifier script and TARGETS file to github repo Closes https://github.com/facebook/rocksdb/pull/2083 Differential Revision: D4825822 Pulled By: IslamAbdelRahman fbshipit-source-id: 205f4a7
2017-04-04 23:09:31 +00:00
# Generate a test rule for each entry in ROCKS_TESTS
for test_cfg in ROCKS_TESTS:
test_name = test_cfg[0]
test_cc = test_cfg[1]
ttype = "gtest" if test_cfg[2] == "parallel" else "simple"
test_bin = test_name + "_bin"
cpp_binary (
name = test_bin,
srcs = [test_cc],
deps = [":rocksdb_test_lib"],
preprocessor_flags = rocksdb_preprocessor_flags,
rocksdb: make buildable on aarch64 Summary: - Remove default arch-specified flags. - Move non-default arch-specific flags to arch-specific param. Reviewed By: yiwu-arbug Differential Revision: D5597499 fbshipit-source-id: c53108ac39c73ac36893d3fd9aaf3b5e3080f1ae
2017-08-14 00:04:19 +00:00
arch_preprocessor_flags = rocksdb_arch_preprocessor_flags,
Add buckifier script to github repo Summary: Add buckifier script and TARGETS file to github repo Closes https://github.com/facebook/rocksdb/pull/2083 Differential Revision: D4825822 Pulled By: IslamAbdelRahman fbshipit-source-id: 205f4a7
2017-04-04 23:09:31 +00:00
compiler_flags = rocksdb_compiler_flags,
external_deps = rocksdb_external_deps,
)
custom_unittest(
name = test_name,
type = ttype,
deps = [":" + test_bin],
command = [TEST_RUNNER, BUCK_BINS + test_bin]
)
sync internal/external TARGETS
2017-06-01 19:31:13 +00:00
custom_unittest(
name = "make_rocksdbjavastatic",
codemod: format TARGETS with buildifier [5/5] (D5092623) Reviewed By: igorsugak fbshipit-source-id: 906b744c179eb932f5a388b39f93209cecd50a80
2017-06-02 00:48:17 +00:00
type = "simple",
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
command = ["internal_repo_rocksdb/make_rocksdbjavastatic.sh"],
sync internal/external TARGETS
2017-06-01 19:31:13 +00:00
)
custom_unittest(
name = "make_rocksdb_lite_release",
codemod: format TARGETS with buildifier [5/5] (D5092623) Reviewed By: igorsugak fbshipit-source-id: 906b744c179eb932f5a388b39f93209cecd50a80
2017-06-02 00:48:17 +00:00
type = "simple",
Improve write buffer manager (and allow the size to be tracked in block cache) Summary: Improve write buffer manager in several ways: 1. Size is tracked when arena block is allocated, rather than every allocation, so that it can better track actual memory usage and the tracking overhead is slightly lower. 2. We start to trigger memtable flush when 7/8 of the memory cap hits, instead of 100%, and make 100% much harder to hit. 3. Allow a cache object to be passed into buffer manager and the size allocated by memtable can be costed there. This can help users have one single memory cap across block cache and memtable. Closes https://github.com/facebook/rocksdb/pull/2350 Differential Revision: D5110648 Pulled By: siying fbshipit-source-id: b4238113094bf22574001e446b5d88523ba00017
2017-06-02 21:13:59 +00:00
command = ["internal_repo_rocksdb/make_rocksdb_lite_release.sh"],
sync internal/external TARGETS
2017-06-01 19:31:13 +00:00
)