mirror of https://github.com/facebook/rocksdb.git
42ea795209
Summary:
I am currently working on a project that uses RocksDB. While debugging some perf issues, I came up across interesting compaction concurrency issue. Namely, I had 15 idle threads and a good comapction to do, but CompactionPicker returned "Compaction nothing to do". Here's how Internal stats looked:
2014/08/22-08:08:04.551982 7fc7fc3f5700 ------- DUMPING STATS -------
2014/08/22-08:08:04.552000 7fc7fc3f5700
** Compaction Stats [default] **
Level Files Size(MB) Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) RW-Amp W-Amp Rd(MB/s) Wr(MB/s) Rn(cnt) Rnp1(cnt) Wnp1(cnt) Wnew(cnt) Comp(sec) Comp(cnt) Avg(sec) Stall(sec) Stall(cnt) Avg(ms)
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
L0 7/5 353 1.0 0.0 0.0 0.0 2.3 2.3 0.0 0.0 0.0 9.4 0 0 0 0 247 46 5.359 8.53 1 8526.25
L1 2/2 86 1.3 2.6 1.9 0.7 2.6 1.9 2.7 1.3 24.3 24.0 39 19 71 52 109 11 9.938 0.00 0 0.00
L2 26/0 833 1.3 5.7 1.7 4.0 5.2 1.2 6.3 3.0 15.6 14.2 47 112 147 35 373 44 8.468 0.00 0 0.00
L3 12/0 505 0.1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 0 0 0 0 0.000 0.00 0 0.00
Sum 47/7 1778 0.0 8.3 3.6 4.6 10.0 5.4 8.1 4.4 11.6 14.1 86 131 218 87 728 101 7.212 8.53 1 8526.25
Int 0/0 0 0.0 2.4 0.8 1.6 2.7 1.2 11.5 6.1 12.0 13.6 20 43 63 20 203 23 8.845 0.00 0 0.00
Flush(GB): accumulative 2.266, interval 0.444
Stalls(secs): 0.000 level0_slowdown, 0.000 level0_numfiles, 8.526 memtable_compaction, 0.000 leveln_slowdown_soft, 0.000 leveln_slowdown_hard
Stalls(count): 0 level0_slowdown, 0 level0_numfiles, 1 memtable_compaction, 0 leveln_slowdown_soft, 0 leveln_slowdown_hard
** DB Stats **
Uptime(secs): 336.8 total, 60.4 interval
Cumulative writes: 61584000 writes, 6480589 batches, 9.5 writes per batch, 1.39 GB user ingest
Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, 0.00 GB written
Interval writes: 11235257 writes, 1175050 batches, 9.6 writes per batch, 259.9 MB user ingest
Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, 0.00 MB written
To see what happened, go here:
|
||
|---|---|---|
| .. | ||
| builder.cc | ||
| builder.h | ||
| c.cc | ||
| c_test.c | ||
| column_family.cc | ||
| column_family.h | ||
| column_family_test.cc | ||
| compaction.cc | ||
| compaction.h | ||
| compaction_picker.cc | ||
| compaction_picker.h | ||
| corruption_test.cc | ||
| cuckoo_table_db_test.cc | ||
| db_bench.cc | ||
| db_filesnapshot.cc | ||
| db_impl.cc | ||
| db_impl.h | ||
| db_impl_debug.cc | ||
| db_impl_readonly.cc | ||
| db_impl_readonly.h | ||
| db_iter.cc | ||
| db_iter.h | ||
| db_iter_test.cc | ||
| db_test.cc | ||
| dbformat.cc | ||
| dbformat.h | ||
| dbformat_test.cc | ||
| deletefile_test.cc | ||
| file_indexer.cc | ||
| file_indexer.h | ||
| file_indexer_test.cc | ||
| filename.cc | ||
| filename.h | ||
| filename_test.cc | ||
| forward_iterator.cc | ||
| forward_iterator.h | ||
| internal_stats.cc | ||
| internal_stats.h | ||
| log_and_apply_bench.cc | ||
| log_format.h | ||
| log_reader.cc | ||
| log_reader.h | ||
| log_test.cc | ||
| log_writer.cc | ||
| log_writer.h | ||
| memtable.cc | ||
| memtable.h | ||
| memtable_list.cc | ||
| memtable_list.h | ||
| merge_context.h | ||
| merge_helper.cc | ||
| merge_helper.h | ||
| merge_operator.cc | ||
| merge_test.cc | ||
| perf_context_test.cc | ||
| plain_table_db_test.cc | ||
| prefix_test.cc | ||
| repair.cc | ||
| simple_table_db_test.cc | ||
| skiplist.h | ||
| skiplist_test.cc | ||
| snapshot.h | ||
| table_cache.cc | ||
| table_cache.h | ||
| table_properties_collector.cc | ||
| table_properties_collector.h | ||
| table_properties_collector_test.cc | ||
| transaction_log_impl.cc | ||
| transaction_log_impl.h | ||
| version_edit.cc | ||
| version_edit.h | ||
| version_edit_test.cc | ||
| version_set.cc | ||
| version_set.h | ||
| version_set_test.cc | ||
| write_batch.cc | ||
| write_batch_internal.h | ||
| write_batch_test.cc | ||