Update main branch for 9.6 release (#12945)

Summary: Main branch cut at defd97bc9. Updated HISTORY.md, version and format compatibility test. Pull Request resolved: https://github.com/facebook/rocksdb/pull/12945 Test Plan: CI Reviewed By: jowlyzhang Differential Revision: D61482149 Pulled By: jaykorean fbshipit-source-id: 4edf7c0a8c6e4df8fcc938bc778dfd02981d0c55
2024-08-19 17:36:23 -07:00 · 2024-08-19 17:36:23 -07:00 · 5b8f5cbcf4
parent defd97bc9d
commit 5b8f5cbcf4
16 changed files with 29 additions and 16 deletions
--- a/HISTORY.md
+++ b/HISTORY.md
@ -1,6 +1,33 @@
 # Rocksdb Change Log
 > NOTE: Entries for next release do not go here. Follow instructions in `unreleased_history/README.txt`

+## 9.7.0 (08/19/2024)
+### New Features
+* Introduce a new mutable CF option `paranoid_memory_checks`. It enables additional validation on data integrity during reads/scanning. Currently, skip list based memtable will validate key ordering during look up and scans.
+
+### Public API Changes
+* Adds optional installation callback function for remote compaction
+
+## 9.6.0 (08/16/2024)
+### New Features
+* *Best efforts recovery supports recovering to incomplete Version with a clean seqno cut that presents a valid point in time view from the user's perspective, if versioning history doesn't include atomic flush.
+* New option `BlockBasedTableOptions::decouple_partitioned_filters` should improve efficiency in serving read queries because filter and index partitions can consistently target the configured `metadata_block_size`. This option is currently opt-in.
+
+### Public API Changes
+* Add ticker stats to count file read retries due to checksum mismatch
+
+### Behavior Changes
+* There may be less intra-L0 compaction triggered by total L0 size being too small. We now use compensated file size (tombstones are assigned some value size) when calculating L0 size and reduce the threshold for L0 size limit. This is to avoid accumulating too much data/tombstones in L0.
+
+### Bug Fixes
+* *Make DestroyDB supports slow deletion when it's configured in `SstFileManager`. The slow deletion is subject to the configured `rate_bytes_per_sec`, but not subject to the `max_trash_db_ratio`.
+* Fixed a bug where we set unprep_seqs_ even when WriteImpl() fails. This was caught by stress test write fault injection in WriteImpl(). This may have incorrectly caused iteration creation failure for unvalidated writes or returned wrong result for WriteUnpreparedTxn::GetUnpreparedSequenceNumbers().
+* Fixed a bug where successful write right after error recovery for last failed write finishes causes duplicate WAL entries
+* Fixed a data race involving the background error status in `unordered_write` mode.
+* *Fix a bug where file snapshot functions like backup, checkpoint may attempt to copy a non-existing manifest file. #12882
+* Fix a bug where per kv checksum corruption may be ignored in MultiGet().
+* Fix a race condition in pessimistic transactions that could allow multiple transactions with the same name to be registered simultaneously, resulting in a crash or other unpredictable behavior.
+
 ## 9.5.0 (07/19/2024)
 ### Public API Changes
 * Introduced new C API function rocksdb_writebatch_iterate_cf for column family-aware iteration over the contents of a WriteBatch
--- a/include/rocksdb/version.h
+++ b/include/rocksdb/version.h
@ -12,7 +12,7 @@
 // NOTE: in 'main' development branch, this should be the *next*
 // minor or major version number planned for release.
 #define ROCKSDB_MAJOR 9
-#define ROCKSDB_MINOR 6
+#define ROCKSDB_MINOR 7
 #define ROCKSDB_PATCH 0

 // Do not use these. We made the mistake of declaring macros starting with
--- a/tools/check_format_compatible.sh
+++ b/tools/check_format_compatible.sh
@ -125,7 +125,7 @@ EOF

 # To check for DB forward compatibility with loading options (old version
 # reading data from new), as well as backward compatibility
-declare -a db_forward_with_options_refs=("8.6.fb" "8.7.fb" "8.8.fb" "8.9.fb" "8.10.fb" "8.11.fb" "9.0.fb" "9.1.fb" "9.2.fb" "9.3.fb" "9.4.fb" "9.5.fb")
+declare -a db_forward_with_options_refs=("8.6.fb" "8.7.fb" "8.8.fb" "8.9.fb" "8.10.fb" "8.11.fb" "9.0.fb" "9.1.fb" "9.2.fb" "9.3.fb" "9.4.fb" "9.5.fb", "9.6.fb")
 # To check for DB forward compatibility without loading options (in addition
 # to the "with loading options" set), as well as backward compatibility
 declare -a db_forward_no_options_refs=() # N/A at the moment
--- a/unreleased_history/behavior_changes/intra-l0.md
+++ b/unreleased_history/behavior_changes/intra-l0.md
@ -1 +0,0 @@
-* There may be less intra-L0 compaction triggered by total L0 size being too small. We now use compensated file size (tombstones are assigned some value size) when calculating L0 size and reduce the threshold for L0 size limit. This is to avoid accumulating too much data/tombstones in L0.
--- a/unreleased_history/bug_fixes/destroy_db_supports_slow_deletion.md
+++ b/unreleased_history/bug_fixes/destroy_db_supports_slow_deletion.md
@ -1 +0,0 @@
-*Make DestroyDB supports slow deletion when it's configured in `SstFileManager`. The slow deletion is subject to the configured `rate_bytes_per_sec`, but not subject to the `max_trash_db_ratio`.
--- a/unreleased_history/bug_fixes/do_not_add_unprep_seqs_when_write_impl_fails.md
+++ b/unreleased_history/bug_fixes/do_not_add_unprep_seqs_when_write_impl_fails.md
@ -1 +0,0 @@
-Fixed a bug where we set unprep_seqs_ even when WriteImpl() fails. This was caught by stress test write fault injection in WriteImpl(). This may have incorrectly caused iteration creation failure for unvalidated writes or returned wrong result for WriteUnpreparedTxn::GetUnpreparedSequenceNumbers().
--- a/unreleased_history/bug_fixes/duplicate_wal_entries.md
+++ b/unreleased_history/bug_fixes/duplicate_wal_entries.md
@ -1 +0,0 @@
-Fixed a bug where successful write right after error recovery for last failed write finishes causes duplicate WAL entries
--- a/unreleased_history/bug_fixes/get_bg_error_race.md
+++ b/unreleased_history/bug_fixes/get_bg_error_race.md
@ -1 +0,0 @@
-Fixed a data race involving the background error status in `unordered_write` mode.
--- a/unreleased_history/bug_fixes/invalid_manifest_number.md
+++ b/unreleased_history/bug_fixes/invalid_manifest_number.md
@ -1 +0,0 @@
-*Fix a bug where file snapshot functions like backup, checkpoint may attempt to copy a non-existing manifest file. #12882
--- a/unreleased_history/bug_fixes/multiget-kv-checksum.md
+++ b/unreleased_history/bug_fixes/multiget-kv-checksum.md
@ -1 +0,0 @@
-* Fix a bug where per kv checksum corruption may be ignored in MultiGet().
--- a/unreleased_history/bug_fixes/prevent_duplicate_txn_name.md
+++ b/unreleased_history/bug_fixes/prevent_duplicate_txn_name.md
@ -1 +0,0 @@
-Fix a race condition in pessimistic transactions that could allow multiple transactions with the same name to be registered simultaneously, resulting in a crash or other unpredictable behavior.
--- a/unreleased_history/new_features/best_efforts_recovery_seqno_prefix.md
+++ b/unreleased_history/new_features/best_efforts_recovery_seqno_prefix.md
@ -1 +0,0 @@
-*Best efforts recovery supports recovering to incomplete Version with a clean seqno cut that presents a valid point in time view from the user's perspective, if versioning history doesn't include atomic flush.
--- a/unreleased_history/new_features/decouple.md
+++ b/unreleased_history/new_features/decouple.md
@ -1 +0,0 @@
-* New option `BlockBasedTableOptions::decouple_partitioned_filters` should improve efficiency in serving read queries because filter and index partitions can consistently target the configured `metadata_block_size`. This option is currently opt-in.
--- a/unreleased_history/new_features/memtable-paranoid-checks.md
+++ b/unreleased_history/new_features/memtable-paranoid-checks.md
@ -1 +0,0 @@
-* Introduce a new mutable CF option `paranoid_memory_checks`. It enables additional validation on data integrity during reads/scanning. Currently, skip list based memtable will validate key ordering during look up and scans.
--- a/unreleased_history/public_api_changes/callback_fn_remote_compaction.md
+++ b/unreleased_history/public_api_changes/callback_fn_remote_compaction.md
@ -1,2 +0,0 @@
-Adds optional installation callback function for remote compaction
-
--- a/unreleased_history/public_api_changes/corruption_retry_stats.md
+++ b/unreleased_history/public_api_changes/corruption_retry_stats.md
@ -1 +0,0 @@
-Add ticker stats to count file read retries due to checksum mismatch
				`@ -1 +0,0 @@`
				`* There may be less intra-L0 compaction triggered by total L0 size being too small. We now use compensated file size (tombstones are assigned some value size) when calculating L0 size and reduce the threshold for L0 size limit. This is to avoid accumulating too much data/tombstones in L0.`
				`@ -1 +0,0 @@`
				*Make DestroyDB supports slow deletion when it's configured in `SstFileManager`. The slow deletion is subject to the configured `rate_bytes_per_sec`, but not subject to the `max_trash_db_ratio`.
				`@ -1 +0,0 @@`
				`Fixed a bug where we set unprep_seqs_ even when WriteImpl() fails. This was caught by stress test write fault injection in WriteImpl(). This may have incorrectly caused iteration creation failure for unvalidated writes or returned wrong result for WriteUnpreparedTxn::GetUnpreparedSequenceNumbers().`
				`@ -1 +0,0 @@`
				`Fixed a bug where successful write right after error recovery for last failed write finishes causes duplicate WAL entries`
				`@ -1 +0,0 @@`
				Fixed a data race involving the background error status in `unordered_write` mode.
				`@ -1 +0,0 @@`
				`*Fix a bug where file snapshot functions like backup, checkpoint may attempt to copy a non-existing manifest file. #12882`
				`@ -1 +0,0 @@`
				`* Fix a bug where per kv checksum corruption may be ignored in MultiGet().`
				`@ -1 +0,0 @@`
				`Fix a race condition in pessimistic transactions that could allow multiple transactions with the same name to be registered simultaneously, resulting in a crash or other unpredictable behavior.`
				`@ -1 +0,0 @@`
				`*Best efforts recovery supports recovering to incomplete Version with a clean seqno cut that presents a valid point in time view from the user's perspective, if versioning history doesn't include atomic flush.`
				`@ -1 +0,0 @@`
				* New option `BlockBasedTableOptions::decouple_partitioned_filters` should improve efficiency in serving read queries because filter and index partitions can consistently target the configured `metadata_block_size`. This option is currently opt-in.