dolysis
/
rocksdb
mirror of https://github.com/facebook/rocksdb.git
3
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
rocksdb/db
History
Hui Xiao 06e593376c Group SST write in flush, compaction and db open with new stats (#11910) 
Summary:
## Context/Summary
Similar to https://github.com/facebook/rocksdb/pull/11288, https://github.com/facebook/rocksdb/pull/11444, categorizing SST/blob file write according to different io activities allows more insight into the activity.

For that, this PR does the following:
- Tag different write IOs by passing down and converting WriteOptions to IOOptions
- Add new SST_WRITE_MICROS histogram in WritableFileWriter::Append() and breakdown FILE_WRITE_{FLUSH|COMPACTION|DB_OPEN}_MICROS

Some related code refactory to make implementation cleaner:
- Blob stats
   - Replace high-level write measurement with low-level WritableFileWriter::Append() measurement for BLOB_DB_BLOB_FILE_WRITE_MICROS. This is to make FILE_WRITE_{FLUSH|COMPACTION|DB_OPEN}_MICROS  include blob file. As a consequence, this introduces some behavioral changes on it, see HISTORY and db bench test plan below for more info.
   - Fix bugs where BLOB_DB_BLOB_FILE_SYNCED/BLOB_DB_BLOB_FILE_BYTES_WRITTEN include file failed to sync and bytes failed to write.
- Refactor WriteOptions constructor for easier construction with io_activity and rate_limiter_priority
- Refactor DBImpl::~DBImpl()/BlobDBImpl::Close() to bypass thread op verification
- Build table
   - TableBuilderOptions now includes Read/WriteOpitons so BuildTable() do not need to take these two variables
   - Replace the io_priority passed into BuildTable() with TableBuilderOptions::WriteOpitons::rate_limiter_priority. Similar for BlobFileBuilder.
This parameter is used for dynamically changing file io priority for flush, see  https://github.com/facebook/rocksdb/pull/9988?fbclid=IwAR1DtKel6c-bRJAdesGo0jsbztRtciByNlvokbxkV6h_L-AE9MACzqRTT5s for more
   - Update ThreadStatus::FLUSH_BYTES_WRITTEN to use io_activity to track flush IO in flush job and db open instead of io_priority

## Test
### db bench

Flush
```
./db_bench --statistics=1 --benchmarks=fillseq --num=100000 --write_buffer_size=100

rocksdb.sst.write.micros P50 : 1.830863 P95 : 4.094720 P99 : 6.578947 P100 : 26.000000 COUNT : 7875 SUM : 20377
rocksdb.file.write.flush.micros P50 : 1.830863 P95 : 4.094720 P99 : 6.578947 P100 : 26.000000 COUNT : 7875 SUM : 20377
rocksdb.file.write.compaction.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0
rocksdb.file.write.db.open.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0
```

compaction, db oopen
```
Setup: ./db_bench --statistics=1 --benchmarks=fillseq --num=10000 --disable_auto_compactions=1 -write_buffer_size=100 --db=../db_bench

Run:./db_bench --statistics=1 --benchmarks=compact  --db=../db_bench --use_existing_db=1

rocksdb.sst.write.micros P50 : 2.675325 P95 : 9.578788 P99 : 18.780000 P100 : 314.000000 COUNT : 638 SUM : 3279
rocksdb.file.write.flush.micros P50 : 0.000000 P95 : 0.000000 P99 : 0.000000 P100 : 0.000000 COUNT : 0 SUM : 0
rocksdb.file.write.compaction.micros P50 : 2.757353 P95 : 9.610687 P99 : 19.316667 P100 : 314.000000 COUNT : 615 SUM : 3213
rocksdb.file.write.db.open.micros P50 : 2.055556 P95 : 3.925000 P99 : 9.000000 P100 : 9.000000 COUNT : 23 SUM : 66
```

blob stats - just to make sure they aren't broken by this PR
```
Integrated Blob DB

Setup: ./db_bench --enable_blob_files=1 --statistics=1 --benchmarks=fillseq --num=10000 --disable_auto_compactions=1 -write_buffer_size=100 --db=../db_bench

Run:./db_bench --enable_blob_files=1 --statistics=1 --benchmarks=compact  --db=../db_bench --use_existing_db=1

pre-PR:
rocksdb.blobdb.blob.file.write.micros P50 : 7.298246 P95 : 9.771930 P99 : 9.991813 P100 : 16.000000 COUNT : 235 SUM : 1600
rocksdb.blobdb.blob.file.synced COUNT : 1
rocksdb.blobdb.blob.file.bytes.written COUNT : 34842

post-PR:
rocksdb.blobdb.blob.file.write.micros P50 : 2.000000 P95 : 2.829360 P99 : 2.993779 P100 : 9.000000 COUNT : 707 SUM : 1614
- COUNT is higher and values are smaller as it includes header and footer write
- COUNT is 3X higher due to each Append() count as one post-PR, while in pre-PR, 3 Append()s counts as one. See https://github.com/facebook/rocksdb/pull/11910/files#diff-32b811c0a1c000768cfb2532052b44dc0b3bf82253f3eab078e15ff201a0dabfL157-L164

rocksdb.blobdb.blob.file.synced COUNT : 1 (stay the same)
rocksdb.blobdb.blob.file.bytes.written COUNT : 34842 (stay the same)
```

```
Stacked Blob DB

Run: ./db_bench --use_blob_db=1 --statistics=1 --benchmarks=fillseq --num=10000 --disable_auto_compactions=1 -write_buffer_size=100 --db=../db_bench

pre-PR:
rocksdb.blobdb.blob.file.write.micros P50 : 12.808042 P95 : 19.674497 P99 : 28.539683 P100 : 51.000000 COUNT : 10000 SUM : 140876
rocksdb.blobdb.blob.file.synced COUNT : 8
rocksdb.blobdb.blob.file.bytes.written COUNT : 1043445

post-PR:
rocksdb.blobdb.blob.file.write.micros P50 : 1.657370 P95 : 2.952175 P99 : 3.877519 P100 : 24.000000 COUNT : 30001 SUM : 67924
- COUNT is higher and values are smaller as it includes header and footer write
- COUNT is 3X higher due to each Append() count as one post-PR, while in pre-PR, 3 Append()s counts as one. See https://github.com/facebook/rocksdb/pull/11910/files#diff-32b811c0a1c000768cfb2532052b44dc0b3bf82253f3eab078e15ff201a0dabfL157-L164

rocksdb.blobdb.blob.file.synced COUNT : 8 (stay the same)
rocksdb.blobdb.blob.file.bytes.written COUNT : 1043445 (stay the same)
```

###  Rehearsal CI stress test
Trigger 3 full runs of all our CI stress tests

###  Performance

Flush
```
TEST_TMPDIR=/dev/shm ./db_basic_bench_pre_pr --benchmark_filter=ManualFlush/key_num:524288/per_key_size:256 --benchmark_repetitions=1000
-- default: 1 thread is used to run benchmark; enable_statistics = true

Pre-pr: avg 507515519.3 ns
497686074,499444327,500862543,501389862,502994471,503744435,504142123,504224056,505724198,506610393,506837742,506955122,507695561,507929036,508307733,508312691,508999120,509963561,510142147,510698091,510743096,510769317,510957074,511053311,511371367,511409911,511432960,511642385,511691964,511730908,

Post-pr: avg 511971266.5 ns, regressed 0.88%
502744835,506502498,507735420,507929724,508313335,509548582,509994942,510107257,510715603,511046955,511352639,511458478,512117521,512317380,512766303,512972652,513059586,513804934,513808980,514059409,514187369,514389494,514447762,514616464,514622882,514641763,514666265,514716377,514990179,515502408,
```

Compaction
```
TEST_TMPDIR=/dev/shm ./db_basic_bench_{pre|post}_pr --benchmark_filter=ManualCompaction/comp_style:0/max_data:134217728/per_key_size:256/enable_statistics:1  --benchmark_repetitions=1000
-- default: 1 thread is used to run benchmark

Pre-pr: avg 495346098.30 ns
492118301,493203526,494201411,494336607,495269217,495404950,496402598,497012157,497358370,498153846

Post-pr: avg 504528077.20, regressed 1.85%. "ManualCompaction" include flush so the isolated regression for compaction should be around 1.85-0.88 = 0.97%
502465338,502485945,502541789,502909283,503438601,504143885,506113087,506629423,507160414,507393007
```

Put with WAL (in case passing WriteOptions slows down this path even without collecting SST write stats)
```
TEST_TMPDIR=/dev/shm ./db_basic_bench_pre_pr --benchmark_filter=DBPut/comp_style:0/max_data:107374182400/per_key_size:256/enable_statistics:1/wal:1  --benchmark_repetitions=1000
-- default: 1 thread is used to run benchmark

Pre-pr: avg 3848.10 ns
3814,3838,3839,3848,3854,3854,3854,3860,3860,3860

Post-pr: avg 3874.20 ns, regressed 0.68%
3863,3867,3871,3874,3875,3877,3877,3877,3880,3881
```

Pull Request resolved: https://github.com/facebook/rocksdb/pull/11910

Reviewed By: ajkr

Differential Revision: D49788060

Pulled By: hx235

fbshipit-source-id: 79e73699cda5be3b66461687e5147c2484fc5eff
 		2023-12-29 15:29:23 -08:00
..
blob
…
compaction
…
db_impl
…
wide
…
arena_wrapped_db_iter.cc
…
arena_wrapped_db_iter.h
…
builder.cc
…
builder.h
…
c.cc
…
c_test.c
…
column_family.cc
…
column_family.h
…
column_family_test.cc
…
compact_files_test.cc
…
comparator_db_test.cc
…
convenience.cc
…
convenience_impl.h
…
corruption_test.cc
…
cuckoo_table_db_test.cc
…
db_basic_test.cc
…
db_block_cache_test.cc
…
db_bloom_filter_test.cc
…
db_clip_test.cc
…
db_compaction_filter_test.cc
…
db_compaction_test.cc
…
db_dynamic_level_test.cc
…
db_encryption_test.cc
…
db_filesnapshot.cc
…
db_flush_test.cc
…
db_info_dumper.cc
…
db_info_dumper.h
…
db_inplace_update_test.cc
…
db_io_failure_test.cc
…
db_iter.cc
…
db_iter.h
…
db_iter_stress_test.cc
…
db_iter_test.cc
…
db_iterator_test.cc
…
db_kv_checksum_test.cc
…
db_log_iter_test.cc
…
db_logical_block_size_cache_test.cc
…
db_memtable_test.cc
…
db_merge_operand_test.cc
…
db_merge_operator_test.cc
…
db_options_test.cc
…
db_properties_test.cc
…
db_range_del_test.cc
…
db_rate_limiter_test.cc
…
db_readonly_with_timestamp_test.cc
…
db_secondary_test.cc
…
db_sst_test.cc
…
db_statistics_test.cc
…
db_table_properties_test.cc
…
db_tailing_iter_test.cc
…
db_test.cc
…
db_test2.cc
…
db_test_util.cc
…
db_test_util.h
…
db_universal_compaction_test.cc
…
db_wal_test.cc
…
db_with_timestamp_basic_test.cc
…
db_with_timestamp_compaction_test.cc
…
db_with_timestamp_test_util.cc
…
db_with_timestamp_test_util.h
…
db_write_buffer_manager_test.cc
…
db_write_test.cc
…
dbformat.cc
…
dbformat.h
…
dbformat_test.cc
…
deletefile_test.cc
…
error_handler.cc
…
error_handler.h
…
error_handler_fs_test.cc
…
event_helpers.cc
…
event_helpers.h
…
experimental.cc
…
external_sst_file_basic_test.cc
…
external_sst_file_ingestion_job.cc
…
external_sst_file_ingestion_job.h
…
external_sst_file_test.cc
…
fault_injection_test.cc
…
file_indexer.cc
…
file_indexer.h
…
file_indexer_test.cc
…
filename_test.cc
…
flush_job.cc
…
flush_job.h
…
flush_job_test.cc
…
flush_scheduler.cc
…
flush_scheduler.h
…
forward_iterator.cc
…
forward_iterator.h
…
forward_iterator_bench.cc
…
history_trimming_iterator.h
…
import_column_family_job.cc
…
import_column_family_job.h
…
import_column_family_test.cc
…
internal_stats.cc
…
internal_stats.h
…
job_context.h
…
kv_checksum.h
…
listener_test.cc
…
log_format.h
…
log_reader.cc
…
log_reader.h
…
log_test.cc
…
log_writer.cc
…
log_writer.h
…
logs_with_prep_tracker.cc
…
logs_with_prep_tracker.h
…
lookup_key.h
…
malloc_stats.cc
…
malloc_stats.h
…
manual_compaction_test.cc
…
memtable.cc
…
memtable.h
…
memtable_list.cc
…
memtable_list.h
…
memtable_list_test.cc
…
merge_context.h
…
merge_helper.cc
…
merge_helper.h
…
merge_helper_test.cc
…
merge_operator.cc
…
merge_test.cc
…
obsolete_files_test.cc
…
options_file_test.cc
…
output_validator.cc
…
output_validator.h
…
perf_context_test.cc
…
periodic_task_scheduler.cc
…
periodic_task_scheduler.h
…
periodic_task_scheduler_test.cc
…
pinned_iterators_manager.h
…
plain_table_db_test.cc
…
post_memtable_callback.h
…
pre_release_callback.h
…
prefix_test.cc
…
range_del_aggregator.cc
…
range_del_aggregator.h
…
range_del_aggregator_bench.cc
…
range_del_aggregator_test.cc
…
range_tombstone_fragmenter.cc
…
range_tombstone_fragmenter.h
…
range_tombstone_fragmenter_test.cc
…
read_callback.h
…
repair.cc
…
repair_test.cc
…
seqno_time_test.cc
…
seqno_to_time_mapping.cc
…
seqno_to_time_mapping.h
…
snapshot_checker.h
…
snapshot_impl.cc
…
snapshot_impl.h
…
table_cache.cc
…
table_cache.h
…
table_cache_sync_and_async.h
…
table_properties_collector.cc
…
table_properties_collector.h
…
table_properties_collector_test.cc
…
transaction_log_impl.cc
…
transaction_log_impl.h
…
trim_history_scheduler.cc
…
trim_history_scheduler.h
…
version_builder.cc
…
version_builder.h
…
version_builder_test.cc
…
version_edit.cc
…
version_edit.h
…
version_edit_handler.cc
…
version_edit_handler.h
…
version_edit_test.cc
…
version_set.cc
…
version_set.h
…
version_set_sync_and_async.h
…
version_set_test.cc
…
version_util.h
…
wal_edit.cc
…
wal_edit.h
…
wal_edit_test.cc
…
wal_manager.cc
…
wal_manager.h
…
wal_manager_test.cc
…
write_batch.cc
…
write_batch_base.cc
…
write_batch_internal.h
…
write_batch_test.cc
…
write_callback.h
…
write_callback_test.cc
…
write_controller.cc
…
write_controller.h
…
write_controller_test.cc
…
write_stall_stats.cc
…
write_stall_stats.h
…
write_thread.cc
…
write_thread.h
…