dolysis
/
rocksdb
mirror of https://github.com/facebook/rocksdb.git
3
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
rocksdb/util
History
Maysam Yabandeh caf0f53a74 Index value delta encoding (#3983) 
Summary:
Given that index value is a BlockHandle, which is basically an <offset, size> pair we can apply delta encoding on the values. The first value at each index restart interval encoded the full BlockHandle but the rest encode only the size. Refer to IndexBlockIter::DecodeCurrentValue for the detail of the encoding. This reduces the index size which helps using the  block cache more efficiently. The feature is enabled with using format_version 4.

The feature comes with a bit of cpu overhead which should be paid back by the higher cache hits due to smaller index block size.
Results with sysbench read-only using 4k blocks and using 16 index restart interval:
Format 2:
19585   rocksdb read-only range=100
Format 3:
19569   rocksdb read-only range=100
Format 4:
19352   rocksdb read-only range=100
Pull Request resolved: https://github.com/facebook/rocksdb/pull/3983

Differential Revision: D8361343

Pulled By: maysamyabandeh

fbshipit-source-id: f882ee082322acac32b0072e2bdbb0b5f854e651
 		2018-08-09 16:58:40 -07:00
..
aligned_buffer.h Fix Copying of data between buffers in FilePrefetchBuffer (#4100) 2018-07-11 12:28:13 -07:00
allocator.h Change RocksDB License 2017-07-15 16:11:23 -07:00
arena.cc comment unused parameters to turn on -Wunused-parameter flag 2018-04-12 17:59:16 -07:00
arena.h arena: derive alignment unit from std::max_align_t 2017-10-17 11:13:19 -07:00
arena_test.cc arena: derive alignment unit from std::max_align_t 2017-10-17 11:13:19 -07:00
auto_roll_logger.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
auto_roll_logger.h Fix the Logger::Close() and DBImpl::Close() design pattern 2018-02-23 13:57:26 -08:00
auto_roll_logger_test.cc Per-thread unique test db names (#4135) 2018-07-13 17:27:39 -07:00
autovector.h Change RocksDB License 2017-07-15 16:11:23 -07:00
autovector_test.cc fix memory leak in two_level_iterator 2018-04-15 17:26:26 -07:00
bloom.cc Improve FullFilterBitsReader::HashMayMatch's doc (#4202) 2018-08-06 11:13:18 -07:00
bloom_test.cc fix gflags namespace 2017-12-01 10:42:05 -08:00
build_version.cc.in Makefile: generate util/build_version.cc from .in file (#1384) 2016-10-25 11:31:39 -07:00
build_version.h Change RocksDB License 2017-07-15 16:11:23 -07:00
cast_util.h Add a missing "once" in .h 2017-07-31 12:12:03 -07:00
channel.h Change RocksDB License 2017-07-15 16:11:23 -07:00
coding.cc Enable MSVC W4 with a few exceptions. Fix warnings and bugs 2017-10-19 10:57:12 -07:00
coding.h Index value delta encoding (#3983) 2018-08-09 16:58:40 -07:00
coding_test.cc Coding.h: Added Fixed16 support (#4142) 2018-07-16 23:43:41 -07:00
compaction_job_stats_impl.cc comment unused parameters to turn on -Wunused-parameter flag 2018-04-12 17:59:16 -07:00
comparator.cc use user_key and iterate_upper_bound to determine compatibility of bloom filters (#3899) 2018-06-26 15:57:26 -07:00
compression.h Converted db/merge_test.cc to use gtest (#4114) 2018-07-13 14:13:07 -07:00
compression_context_cache.cc run make format for PR 3838 (#3954) 2018-06-05 12:58:02 -07:00
compression_context_cache.h run make format for PR 3838 (#3954) 2018-06-05 12:58:02 -07:00
concurrent_arena.cc Cap concurrent arena's shard block size to 128KB (#4147) 2018-07-18 10:43:54 -07:00
concurrent_arena.h util: Fix coverity issues 2017-11-03 14:42:08 -07:00
core_local.h Change RocksDB License 2017-07-15 16:11:23 -07:00
crc32c.cc Add GCC 8 to Travis (#3433) 2018-07-13 10:58:06 -07:00
crc32c.h Updated CRC32 Power Optimization Changes 2017-08-31 14:16:30 -07:00
crc32c_ppc.c Updated CRC32 Power Optimization Changes 2017-08-31 14:16:30 -07:00
crc32c_ppc.h Updated CRC32 Power Optimization Changes 2017-08-31 14:16:30 -07:00
crc32c_ppc_asm.S Updated CRC32 Power Optimization Changes 2017-08-31 14:16:30 -07:00
crc32c_ppc_constants.h Make it explicit blob db doesn't support CF 2017-09-08 11:11:04 -07:00
crc32c_test.cc Build and tests fixes for Solaris Sparc (#4000) 2018-06-15 12:42:53 -07:00
delete_scheduler.cc In delete scheduler, before ftruncate file for slow delete, check whether there is other hard links (#4093) 2018-07-09 15:28:12 -07:00
delete_scheduler.h In delete scheduler, before ftruncate file for slow delete, check whether there is other hard links (#4093) 2018-07-09 15:28:12 -07:00
delete_scheduler_test.cc Per-thread unique test db names (#4135) 2018-07-13 17:27:39 -07:00
duplicate_detector.h Improve db_stress with transactions 2018-04-18 16:32:35 -07:00
dynamic_bloom.cc Use nullptr instead of NULL / 0 more consistently. 2018-03-07 12:42:12 -08:00
dynamic_bloom.h Enable MSVC W4 with a few exceptions. Fix warnings and bugs 2017-10-19 10:57:12 -07:00
dynamic_bloom_test.cc fix gflags namespace 2017-12-01 10:42:05 -08:00
event_logger.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
event_logger.h Change RocksDB License 2017-07-15 16:11:23 -07:00
event_logger_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
fault_injection_test_env.cc Allow DB resume after background errors (#3997) 2018-06-28 12:34:40 -07:00
fault_injection_test_env.h Allow DB resume after background errors (#3997) 2018-06-28 12:34:40 -07:00
file_reader_writer.cc BlockBasedTableReader: automatically adjust tail prefetch size (#4156) 2018-07-20 14:43:37 -07:00
file_reader_writer.h BlockBasedTableReader: automatically adjust tail prefetch size (#4156) 2018-07-20 14:43:37 -07:00
file_reader_writer_test.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
file_util.cc Add file name info to SequentialFileReader. (#4026) 2018-06-21 08:42:24 -07:00
file_util.h Sync parent directory after deleting a file in delete scheduler 2018-04-26 13:58:20 -07:00
filelock_test.cc Per-thread unique test db names (#4135) 2018-07-13 17:27:39 -07:00
filename.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
filename.h Change RocksDB License 2017-07-15 16:11:23 -07:00
filter_policy.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
gflags_compat.h fix gflags namespace 2017-12-01 10:42:05 -08:00
hash.cc Add GCC 8 to Travis (#3433) 2018-07-13 10:58:06 -07:00
hash.h Change RocksDB License 2017-07-15 16:11:23 -07:00
hash_map.h Change RocksDB License 2017-07-15 16:11:23 -07:00
hash_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
heap.h Change RocksDB License 2017-07-15 16:11:23 -07:00
heap_test.cc fix gflags namespace 2017-12-01 10:42:05 -08:00
kv_map.h Change RocksDB License 2017-07-15 16:11:23 -07:00
log_buffer.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
log_buffer.h Change RocksDB License 2017-07-15 16:11:23 -07:00
log_write_bench.cc Per-thread unique test db names (#4135) 2018-07-13 17:27:39 -07:00
logging.h Change RocksDB License 2017-07-15 16:11:23 -07:00
memory_usage.h Change RocksDB License 2017-07-15 16:11:23 -07:00
murmurhash.cc Add GCC 8 to Travis (#3433) 2018-07-13 10:58:06 -07:00
murmurhash.h Change RocksDB License 2017-07-15 16:11:23 -07:00
mutexlock.h Change RocksDB License 2017-07-15 16:11:23 -07:00
ppc-opcode.h Updated CRC32 Power Optimization Changes 2017-08-31 14:16:30 -07:00
random.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
random.h Change RocksDB License 2017-07-15 16:11:23 -07:00
rate_limiter.cc rate limit auto-tuning 2017-10-04 19:15:01 -07:00
rate_limiter.h rate limit auto-tuning 2017-10-04 19:15:01 -07:00
rate_limiter_test.cc comment unused parameters to turn on -Wunused-parameter flag 2018-04-12 17:59:16 -07:00
set_comparator.h WritePrepared Txn: Move DuplicateDetector to util 2018-03-05 23:57:12 -08:00
slice.cc use user_key and iterate_upper_bound to determine compatibility of bloom filters (#3899) 2018-06-26 15:57:26 -07:00
slice_transform_test.cc Per-thread unique test db names (#4135) 2018-07-13 17:27:39 -07:00
sst_file_manager_impl.cc Expose GetTotalTrashSize in SstFileManager interface (#4206) 2018-08-04 17:57:48 -07:00
sst_file_manager_impl.h Expose GetTotalTrashSize in SstFileManager interface (#4206) 2018-08-04 17:57:48 -07:00
status.cc move static msgs out of Status class (#4144) 2018-07-23 15:44:16 -07:00
stderr_logger.h Change RocksDB License 2017-07-15 16:11:23 -07:00
stop_watch.h Exclude time waiting for rate limiter from rocksdb.sst.read.micros (#4102) 2018-07-13 18:44:14 -07:00
string_util.cc Several small "fixes" 2018-02-15 16:57:37 -08:00
string_util.h Change RocksDB License 2017-07-15 16:11:23 -07:00
sync_point.cc Refactor sync_point to make implementation either customizable or replaceable 2018-03-23 12:56:52 -07:00
sync_point.h Fix singleton destruction order of PosixEnv and SyncPoint (#3951) 2018-06-04 15:58:46 -07:00
sync_point_impl.cc Avoid acquiring SyncPoint mutex when it is disabled (#3991) 2018-06-13 13:13:18 -07:00
sync_point_impl.h Avoid acquiring SyncPoint mutex when it is disabled (#3991) 2018-06-13 13:13:18 -07:00
testharness.cc Per-thread unique test db names (#4135) 2018-07-13 17:27:39 -07:00
testharness.h Per-thread unique test db names (#4135) 2018-07-13 17:27:39 -07:00
testutil.cc Index value delta encoding (#3983) 2018-08-09 16:58:40 -07:00
testutil.h Add file name info to SequentialFileReader. (#4026) 2018-06-21 08:42:24 -07:00
thread_list_test.cc Use nullptr instead of NULL / 0 more consistently. 2018-03-07 12:42:12 -08:00
thread_local.cc Reformatting some recent changes (#4161) 2018-07-20 14:43:38 -07:00
thread_local.h Provide a way to override windows memory allocator with jemalloc for ZSTD 2018-06-04 12:12:48 -07:00
thread_local_test.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
thread_operation.h Change RocksDB License 2017-07-15 16:11:23 -07:00
threadpool_imp.cc Support lowering CPU priority of background threads 2018-04-24 08:41:51 -07:00
threadpool_imp.h Support lowering CPU priority of background threads 2018-04-24 08:41:51 -07:00
timer_queue.h Change RocksDB License 2017-07-15 16:11:23 -07:00
timer_queue_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
trace_replay.cc Trace and Replay for RocksDB (#3837) 2018-08-01 00:27:08 -07:00
trace_replay.h Trace and Replay for RocksDB (#3837) 2018-08-01 00:27:08 -07:00
transaction_test_util.cc WriteUnPrepared: Implement unprepared batches for transactions (#4104) 2018-07-24 00:13:18 -07:00
transaction_test_util.h WritePrepared Txn: stress test 2017-12-06 09:42:28 -08:00
util.h Add GCC 8 to Travis (#3433) 2018-07-13 10:58:06 -07:00
xxhash.cc fixed typo 2017-06-05 11:27:34 -07:00
xxhash.h Prevent xxhash symbols from polluting global namespace 2015-03-12 12:07:10 -07:00