dolysis/rocksdb
3
0
Fork 0
mirror of https://github.com/facebook/rocksdb.git synced 2024-11-29 18:33:58 +00:00
Code Issues Packages Projects Releases Wiki Activity
rocksdb/db
History
Feng Zhu f697cad15c create compressed_levels_ in Version, allocate its space using arena. Make Version::Get, Version::FindFile faster 
Summary:
    Define CompressedFileMetaData that just contains fd, smallest_slice, largest_slice. Create compressed_levels_ in Version, the space is allocated using arena
    Thus increase the file meta data locality, speed up "Get" and "FindFile"

    benchmark with in-memory tmpfs, could have 4% improvement under "random read" and 2% improvement under "read while writing"

benchmark command:
./db_bench --db=/mnt/db/rocksdb --num_levels=6 --key_size=20 --prefix_size=20 --keys_per_prefix=0 --value_size=100 --block_size=4096 --cache_size=17179869184 --cache_numshardbits=6 --compression_type=none --compression_ratio=1 --min_level_to_compress=-1 --disable_seek_compaction=1 --hard_rate_limit=2 --write_buffer_size=134217728 --max_write_buffer_number=2 --level0_file_num_compaction_trigger=8 --target_file_size_base=33554432 --max_bytes_for_level_base=1073741824 --disable_wal=0 --sync=0 --disable_data_sync=1 --verify_checksum=1 --delete_obsolete_files_period_micros=314572800 --max_grandparent_overlap_factor=10 --max_background_compactions=4 --max_background_flushes=0 --level0_slowdown_writes_trigger=16 --level0_stop_writes_trigger=24 --statistics=0 --stats_per_interval=0 --stats_interval=1048576 --histogram=0 --use_plain_table=1 --open_files=-1 --mmap_read=1 --mmap_write=0 --memtablerep=prefix_hash --bloom_bits=10 --bloom_locality=1 --perf_level=0 --benchmarks=readwhilewriting,readwhilewriting,readwhilewriting --use_existing_db=1 --num=52428800 --threads=1 —writes_per_second=81920

Read Random:
From 1.8363 ms/op, improve to 1.7587 ms/op.
Read while writing:
From 2.985 ms/op, improve to 2.924 ms/op.

Test Plan:
    make all check

Reviewers: ljin, haobo, yhchiang, sdong

Reviewed By: sdong

Subscribers: dhruba, igor

Differential Revision: https://reviews.facebook.net/D19419
2014-07-09 22:14:39 -07:00
..
builder.cc integrate rate limiter into rocksdb 2014-07-08 12:31:49 -07:00
builder.h integrate rate limiter into rocksdb 2014-07-08 12:31:49 -07:00
c.cc Merge pull request #197 from rdallman/update-options 2014-07-08 11:40:11 -07:00
c_test.c Fix valgrind error in c_test 2014-07-07 14:41:54 -07:00
column_family.cc Support Multiple DB paths (without having an interface to expose to users) 2014-07-02 21:14:44 -07:00
column_family.h Cache some conditions for DBImpl::MakeRoomForWrite 2014-06-26 16:45:27 -07:00
column_family_test.cc Remove seek compaction 2014-06-20 10:23:02 +02:00
compaction.cc Support Multiple DB paths (without having an interface to expose to users) 2014-07-02 21:14:44 -07:00
compaction.h create compressed_levels_ in Version, allocate its space using arena. Make Version::Get, Version::FindFile faster 2014-07-09 22:14:39 -07:00
compaction_picker.cc Some fixes on size compensation logic for deletion entry in compaction 2014-07-09 12:46:08 -07:00
compaction_picker.h FIFO compaction style 2014-05-21 11:43:35 -07:00
corruption_test.cc Fix corruption test 2014-04-24 14:56:41 -04:00
db_bench.cc Adding NUMA support to db_bench tests 2014-07-07 10:53:31 -07:00
db_filesnapshot.cc Support Multiple DB paths (without having an interface to expose to users) 2014-07-02 21:14:44 -07:00
db_impl.cc integrate rate limiter into rocksdb 2014-07-08 12:31:49 -07:00
db_impl.h Finer report I/O stats about Flush and Compaction. 2014-07-03 16:28:03 -07:00
db_impl_debug.cc FIFO compaction style 2014-05-21 11:43:35 -07:00
db_impl_readonly.cc Remove seek compaction 2014-06-20 10:23:02 +02:00
db_impl_readonly.h [CF] Rethinking ColumnFamilyHandle and fix to dropping column families 2014-02-12 13:47:09 -08:00
db_iter.cc Fix compile errors on Mac 2014-06-03 12:28:58 -07:00
db_iter.h In DB::NewIterator(), try to allocate the whole iterator tree in an arena 2014-06-02 17:44:57 -07:00
db_stats_logger.cc RocksDBLite 2014-04-15 13:39:26 -07:00
db_test.cc make rate limiter test more reliable 2014-07-08 15:15:00 -07:00
dbformat.cc macros for perf_context 2014-04-08 10:58:07 -07:00
dbformat.h HashLinkList memtable switches a bucket to a skip list to reduce performance outliers 2014-07-01 17:14:15 -07:00
dbformat_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
deletefile_test.cc Start DeleteFileTest with clean plate 2013-11-15 16:30:23 -08:00
file_indexer.cc hints for narrowing down FindFile range and avoiding checking unrelevant L0 files 2014-04-21 09:10:12 -07:00
file_indexer.h Remove TransactionLogIteratorRace when -DNDEBUG 2014-04-21 11:08:30 -07:00
file_indexer_test.cc Some fixes as preparation for release 2014-04-30 09:03:24 -07:00
filename.cc Support Multiple DB paths (without having an interface to expose to users) 2014-07-02 21:14:44 -07:00
filename.h Support Multiple DB paths (without having an interface to expose to users) 2014-07-02 21:14:44 -07:00
filename_test.cc Support Multiple DB paths (without having an interface to expose to users) 2014-07-02 21:14:44 -07:00
forward_iterator.cc Apply fractional cascading in ForwardIterator::Seek() 2014-07-08 11:40:42 -07:00
forward_iterator.h Fix compile errors on Mac 2014-06-03 12:28:58 -07:00
internal_stats.cc report correct interval amplification 2014-07-08 12:48:10 -07:00
internal_stats.h fix one more internal_stats issue 2014-07-08 15:29:13 -07:00
log_and_apply_bench.cc Support Multiple DB paths (without having an interface to expose to users) 2014-07-02 21:14:44 -07:00
log_format.h Some minor refactoring on the code 2014-01-02 16:32:31 -08:00
log_reader.cc Make Log::Reader more robust 2014-02-28 13:19:47 -08:00
log_reader.h Fix UnmarkEOF for partial blocks 2014-01-27 14:49:10 -08:00
log_test.cc Make it compile on Debian/GCC 4.7 2014-03-14 22:44:35 +00:00
log_writer.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
log_writer.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
memtable.cc use arena to allocate memtable's bloomfilter and hashskiplist's buckets_ 2014-06-30 15:54:31 -07:00
memtable.h In DB::NewIterator(), try to allocate the whole iterator tree in an arena 2014-06-02 17:44:57 -07:00
memtable_list.cc Support Multiple DB paths (without having an interface to expose to users) 2014-07-02 21:14:44 -07:00
memtable_list.h Support Multiple DB paths (without having an interface to expose to users) 2014-07-02 21:14:44 -07:00
merge_context.h Enhance partial merge to support multiple arguments 2014-03-24 17:57:13 -07:00
merge_helper.cc Fix the bug in MergeUtil which causes mixing values of different keys. 2014-03-27 16:15:25 -07:00
merge_helper.h [rocksdb] new CompactionFilterV2 API 2014-03-24 20:47:53 -07:00
merge_operator.cc Some small cleaning up to make some compiling environment happy 2014-03-26 18:11:41 -07:00
merge_test.cc Use new DBWithTTL API in tests 2014-04-28 23:46:24 -04:00
perf_context_test.cc Missing includes 2014-03-14 13:02:20 -07:00
plain_table_db_test.cc HashLinkList memtable switches a bucket to a skip list to reduce performance outliers 2014-07-01 17:14:15 -07:00
prefix_test.cc HashLinkList memtable switches a bucket to a skip list to reduce performance outliers 2014-07-01 17:14:15 -07:00
repair.cc Support Multiple DB paths (without having an interface to expose to users) 2014-07-02 21:14:44 -07:00
simple_table_db_test.cc Remove seek compaction 2014-06-20 10:23:02 +02:00
skiplist.h Consolidate SliceTransform object ownership 2014-03-10 12:56:46 -07:00
skiplist_test.cc Clean up arena API 2014-01-30 22:10:10 -08:00
snapshot.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
table_cache.cc Support Multiple DB paths (without having an interface to expose to users) 2014-07-02 21:14:44 -07:00
table_cache.h Support Multiple DB paths (without having an interface to expose to users) 2014-07-02 21:14:44 -07:00
table_properties_collector.cc Extract metaindex block from block-based table 2013-12-05 16:34:16 -08:00
table_properties_collector.h TablePropertiesCollectorFactory 2014-05-13 12:30:55 -07:00
table_properties_collector_test.cc TablePropertiesCollectorFactory 2014-05-13 12:30:55 -07:00
tailing_iter.cc kill ReadOptions.prefix and .prefix_seek 2014-04-25 12:21:34 -07:00
tailing_iter.h kill ReadOptions.prefix and .prefix_seek 2014-04-25 12:21:34 -07:00
transaction_log_impl.cc Fixed a file-not-found issue when a log file is moved to archive. 2014-05-12 17:50:21 -07:00
transaction_log_impl.h RocksDBLite 2014-04-15 13:39:26 -07:00
version_edit.cc Support Multiple DB paths (without having an interface to expose to users) 2014-07-02 21:14:44 -07:00
version_edit.h create compressed_levels_ in Version, allocate its space using arena. Make Version::Get, Version::FindFile faster 2014-07-09 22:14:39 -07:00
version_edit_test.cc Support Multiple DB paths (without having an interface to expose to users) 2014-07-02 21:14:44 -07:00
version_set.cc create compressed_levels_ in Version, allocate its space using arena. Make Version::Get, Version::FindFile faster 2014-07-09 22:14:39 -07:00
version_set.h create compressed_levels_ in Version, allocate its space using arena. Make Version::Get, Version::FindFile faster 2014-07-09 22:14:39 -07:00
version_set_test.cc create compressed_levels_ in Version, allocate its space using arena. Make Version::Get, Version::FindFile faster 2014-07-09 22:14:39 -07:00
write_batch.cc Support for column families in TTL DB 2014-04-22 11:27:33 -07:00
write_batch_internal.h Support for column families in TTL DB 2014-04-22 11:27:33 -07:00
write_batch_test.cc kill ReadOptions.prefix and .prefix_seek 2014-04-25 12:21:34 -07:00