dolysis
/
rocksdb
mirror of https://github.com/facebook/rocksdb.git
3
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

reduce memory usage in CircleCI mini crashtest (#10639) 

Summary:
Example flake where CircleCI reports memory at 99% and process gets killed with signal 9 (likely OOM): https://app.circleci.com/pipelines/github/facebook/rocksdb/18085/workflows/bdadbfe6-c40f-4ccb-a5db-fc8c4036f20a/jobs/475628

The previous settings of max_key=25000000, column_families=10, and log2_keys_per_lock=2 resulted in 3GB memory usage just for SharedState. The locks alone consume at least (25000000 keys per CF) * (10 CFs) / (2^2 keys per lock) * (40 bytes per lock) = 2.3GB. This PR reduces it 10x by reducing max_key by that factor.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/10639

Reviewed By: cbi42

Differential Revision: D39263804

Pulled By: ajkr

fbshipit-source-id: 9b5565bbafcb21a2f5b487c8364808dea2f0bc0c
This commit is contained in:
Andrew Kryczka 2022-09-05 16:22:37 -07:00 committed by Facebook GitHub Bot
parent 36dec11bc6
commit 76de3c85cc
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.circleci/config.yml
View File

@ -444,7 +444,7 @@ jobs:
resource_class: large resource_class: large
steps: steps:
- pre-steps - pre-steps
- run: ulimit -S -n `ulimit -H -n` && make V=1 -j8 CRASH_TEST_EXT_ARGS=--duration=960 blackbox_crash_test_with_atomic_flush - run: ulimit -S -n `ulimit -H -n` && make V=1 -j8 CRASH_TEST_EXT_ARGS='--duration=960 --max_key=2500000' blackbox_crash_test_with_atomic_flush
- post-steps - post-steps
build-linux-crashtest-tiered-storage-bb: build-linux-crashtest-tiered-storage-bb: