From bee8d5560eb9e65fdbbb52f2734bc6bf8f8b1d5f Mon Sep 17 00:00:00 2001 From: Yu Zhang Date: Mon, 18 Nov 2024 21:49:47 -0800 Subject: [PATCH] Start version 9.10.0 (#13146) Summary: Pull in HISTORY for 9.9.0, update version.h for next version, update check_format_compatible.sh, update git hash for folly Pull Request resolved: https://github.com/facebook/rocksdb/pull/13146 Test Plan: CI Reviewed By: ltamasi Differential Revision: D66142259 Pulled By: jowlyzhang fbshipit-source-id: 90216b2d7cff2e0befb4f56567e3bd074f97c484 --- HISTORY.md | 23 +++++++++++++++++++ Makefile | 2 +- include/rocksdb/version.h | 2 +- tools/check_format_compatible.sh | 2 +- ...e_delta_iterator_allow_unprepared_value.md | 1 - .../base_delta_iterator_prepare_value.md | 1 - .../bug_fixes/blob_file_leak.md | 1 - .../bug_fixes/missing_strong_reads.md | 1 - unreleased_history/bug_fixes/old_wal_2pc.md | 1 - .../bug_fixes/open_sst_file_leaks.md | 1 - .../bug_fixes/write_committed_toggle_udt.md | 1 - .../new_features/multi_cf_iterator.md | 1 - .../new_features/tp_newest_key_time.md | 1 - .../cra_regression.md | 1 - .../txn_get_attribute_group_iterator.md | 1 - .../txn_get_coalescing_iterator.md | 1 - 16 files changed, 26 insertions(+), 15 deletions(-) delete mode 100644 unreleased_history/behavior_changes/base_delta_iterator_allow_unprepared_value.md delete mode 100644 unreleased_history/bug_fixes/base_delta_iterator_prepare_value.md delete mode 100644 unreleased_history/bug_fixes/blob_file_leak.md delete mode 100644 unreleased_history/bug_fixes/missing_strong_reads.md delete mode 100644 unreleased_history/bug_fixes/old_wal_2pc.md delete mode 100644 unreleased_history/bug_fixes/open_sst_file_leaks.md delete mode 100644 unreleased_history/bug_fixes/write_committed_toggle_udt.md delete mode 100644 unreleased_history/new_features/multi_cf_iterator.md delete mode 100644 unreleased_history/new_features/tp_newest_key_time.md delete mode 100644 unreleased_history/performance_improvements/cra_regression.md delete mode 100644 unreleased_history/public_api_changes/txn_get_attribute_group_iterator.md delete mode 100644 unreleased_history/public_api_changes/txn_get_coalescing_iterator.md diff --git a/HISTORY.md b/HISTORY.md index 8f3920d4b8..15938e142b 100644 --- a/HISTORY.md +++ b/HISTORY.md @@ -1,6 +1,29 @@ # Rocksdb Change Log > NOTE: Entries for next release do not go here. Follow instructions in `unreleased_history/README.txt` +## 9.9.0 (11/18/2024) +### New Features +* Multi-Column-Family-Iterator (CoalescingIterator/AttributeGroupIterator) is no longer marked as experimental +* Adds a new table property "rocksdb.newest.key.time" which records the unix timestamp of the newest key. Uses this table property for FIFO TTL and temperature change compaction. + +### Public API Changes +* Added a new API `Transaction::GetAttributeGroupIterator` that can be used to create a multi-column-family attribute group iterator over the specified column families, including the data from both the transaction and the underlying database. This API is currently supported for optimistic and write-committed pessimistic transactions. +* Added a new API `Transaction::GetCoalescingIterator` that can be used to create a multi-column-family coalescing iterator over the specified column families, including the data from both the transaction and the underlying database. This API is currently supported for optimistic and write-committed pessimistic transactions. + +### Behavior Changes +* `BaseDeltaIterator` now honors the read option `allow_unprepared_value`. + +### Bug Fixes +* `BaseDeltaIterator` now calls `PrepareValue` on the base iterator in case it has been created with the `allow_unprepared_value` read option set. Earlier, such base iterators could lead to incorrect values being exposed from `BaseDeltaIterator`. +* Fix a leak of obsolete blob files left open until DB::Close(). This bug was introduced in version 9.4.0. +* Fix missing cases of corruption retry during DB open and read API processing. +* Fix a bug for transaction db with 2pc where an old WAL may be retained longer than needed (#13127). +* Fix leaks of some open SST files (until `DB::Close()`) that are written but never become live due to various failures. (We now have a check for such leaks with no outstanding issues.) +* Fix a bug for replaying WALs for WriteCommitted transaction DB when its user-defined timestamps setting is toggled on/off between DB sessions. + +### Performance Improvements +* Fix regression in issue #12038 due to `Options::compaction_readahead_size` greater than `max_sectors_kb` (i.e, largest I/O size that the OS issues to a block device defined in linux) + ## 9.8.0 (10/25/2024) ### New Features * All non-`block_cache` options in `BlockBasedTableOptions` are now mutable with `DB::SetOptions()`. See also Bug Fixes below. diff --git a/Makefile b/Makefile index e6e57655de..0bedd667b3 100644 --- a/Makefile +++ b/Makefile @@ -2489,7 +2489,7 @@ checkout_folly: fi @# Pin to a particular version for public CI, so that PR authors don't @# need to worry about folly breaking our integration. Update periodically - cd third-party/folly && git reset --hard 03041f014b6e6ebb6119ffae8b7a37308f52e913 + cd third-party/folly && git reset --hard 33f5b67fcaeb8705b04fd1b850873a180dc89aaa @# NOTE: this hack is required for clang in some cases perl -pi -e 's/int rv = syscall/int rv = (int)syscall/' third-party/folly/folly/detail/Futex.cpp @# NOTE: this hack is required for gcc in some cases diff --git a/include/rocksdb/version.h b/include/rocksdb/version.h index d414e2fd5b..23e347661e 100644 --- a/include/rocksdb/version.h +++ b/include/rocksdb/version.h @@ -12,7 +12,7 @@ // NOTE: in 'main' development branch, this should be the *next* // minor or major version number planned for release. #define ROCKSDB_MAJOR 9 -#define ROCKSDB_MINOR 9 +#define ROCKSDB_MINOR 10 #define ROCKSDB_PATCH 0 // Do not use these. We made the mistake of declaring macros starting with diff --git a/tools/check_format_compatible.sh b/tools/check_format_compatible.sh index 131e32faaf..c012e473c7 100755 --- a/tools/check_format_compatible.sh +++ b/tools/check_format_compatible.sh @@ -125,7 +125,7 @@ EOF # To check for DB forward compatibility with loading options (old version # reading data from new), as well as backward compatibility -declare -a db_forward_with_options_refs=("8.6.fb" "8.7.fb" "8.8.fb" "8.9.fb" "8.10.fb" "8.11.fb" "9.0.fb" "9.1.fb" "9.2.fb" "9.3.fb" "9.4.fb" "9.5.fb" "9.6.fb" "9.7.fb" "9.8.fb") +declare -a db_forward_with_options_refs=("8.6.fb" "8.7.fb" "8.8.fb" "8.9.fb" "8.10.fb" "8.11.fb" "9.0.fb" "9.1.fb" "9.2.fb" "9.3.fb" "9.4.fb" "9.5.fb" "9.6.fb" "9.7.fb" "9.8.fb" "9.9.fb") # To check for DB forward compatibility without loading options (in addition # to the "with loading options" set), as well as backward compatibility declare -a db_forward_no_options_refs=() # N/A at the moment diff --git a/unreleased_history/behavior_changes/base_delta_iterator_allow_unprepared_value.md b/unreleased_history/behavior_changes/base_delta_iterator_allow_unprepared_value.md deleted file mode 100644 index ab24be58fc..0000000000 --- a/unreleased_history/behavior_changes/base_delta_iterator_allow_unprepared_value.md +++ /dev/null @@ -1 +0,0 @@ -`BaseDeltaIterator` now honors the read option `allow_unprepared_value`. diff --git a/unreleased_history/bug_fixes/base_delta_iterator_prepare_value.md b/unreleased_history/bug_fixes/base_delta_iterator_prepare_value.md deleted file mode 100644 index 3cc28d3c6f..0000000000 --- a/unreleased_history/bug_fixes/base_delta_iterator_prepare_value.md +++ /dev/null @@ -1 +0,0 @@ -`BaseDeltaIterator` now calls `PrepareValue` on the base iterator in case it has been created with the `allow_unprepared_value` read option set. Earlier, such base iterators could lead to incorrect values being exposed from `BaseDeltaIterator`. diff --git a/unreleased_history/bug_fixes/blob_file_leak.md b/unreleased_history/bug_fixes/blob_file_leak.md deleted file mode 100644 index 57f54df767..0000000000 --- a/unreleased_history/bug_fixes/blob_file_leak.md +++ /dev/null @@ -1 +0,0 @@ -* Fix a leak of obsolete blob files left open until DB::Close(). This bug was introduced in version 9.4.0. diff --git a/unreleased_history/bug_fixes/missing_strong_reads.md b/unreleased_history/bug_fixes/missing_strong_reads.md deleted file mode 100644 index b0d10c4b15..0000000000 --- a/unreleased_history/bug_fixes/missing_strong_reads.md +++ /dev/null @@ -1 +0,0 @@ -Fix missing cases of corruption retry during DB open and read API processing. diff --git a/unreleased_history/bug_fixes/old_wal_2pc.md b/unreleased_history/bug_fixes/old_wal_2pc.md deleted file mode 100644 index 4989c80a88..0000000000 --- a/unreleased_history/bug_fixes/old_wal_2pc.md +++ /dev/null @@ -1 +0,0 @@ -* Fix a bug for transaction db with 2pc where an old WAL may be retained longer than needed (#13127). \ No newline at end of file diff --git a/unreleased_history/bug_fixes/open_sst_file_leaks.md b/unreleased_history/bug_fixes/open_sst_file_leaks.md deleted file mode 100644 index a75e3be541..0000000000 --- a/unreleased_history/bug_fixes/open_sst_file_leaks.md +++ /dev/null @@ -1 +0,0 @@ -* Fix leaks of some open SST files (until `DB::Close()`) that are written but never become live due to various failures. (We now have a check for such leaks with no outstanding issues.) diff --git a/unreleased_history/bug_fixes/write_committed_toggle_udt.md b/unreleased_history/bug_fixes/write_committed_toggle_udt.md deleted file mode 100644 index 920d8a8a65..0000000000 --- a/unreleased_history/bug_fixes/write_committed_toggle_udt.md +++ /dev/null @@ -1 +0,0 @@ -*Fix a bug for replaying WALs for WriteCommitted transaction DB when its user-defined timestamps setting is toggled on/off between DB sessions. \ No newline at end of file diff --git a/unreleased_history/new_features/multi_cf_iterator.md b/unreleased_history/new_features/multi_cf_iterator.md deleted file mode 100644 index 78dd32655d..0000000000 --- a/unreleased_history/new_features/multi_cf_iterator.md +++ /dev/null @@ -1 +0,0 @@ -Multi-Column-Family-Iterator (CoalescingIterator/AttributeGroupIterator) is no longer marked as experimental diff --git a/unreleased_history/new_features/tp_newest_key_time.md b/unreleased_history/new_features/tp_newest_key_time.md deleted file mode 100644 index 1aaa773fa2..0000000000 --- a/unreleased_history/new_features/tp_newest_key_time.md +++ /dev/null @@ -1 +0,0 @@ -* Adds a new table property "rocksdb.newest.key.time" which records the unix timestamp of the newest key. Uses this table property for FIFO TTL and temperature change compaction. diff --git a/unreleased_history/performance_improvements/cra_regression.md b/unreleased_history/performance_improvements/cra_regression.md deleted file mode 100644 index 89788b66ca..0000000000 --- a/unreleased_history/performance_improvements/cra_regression.md +++ /dev/null @@ -1 +0,0 @@ -Fix regression in issue #12038 due to `Options::compaction_readahead_size` greater than `max_sectors_kb` (i.e, largest I/O size that the OS issues to a block device defined in linux) diff --git a/unreleased_history/public_api_changes/txn_get_attribute_group_iterator.md b/unreleased_history/public_api_changes/txn_get_attribute_group_iterator.md deleted file mode 100644 index 2dad16d679..0000000000 --- a/unreleased_history/public_api_changes/txn_get_attribute_group_iterator.md +++ /dev/null @@ -1 +0,0 @@ -Added a new API `Transaction::GetAttributeGroupIterator` that can be used to create a multi-column-family attribute group iterator over the specified column families, including the data from both the transaction and the underlying database. This API is currently supported for optimistic and write-committed pessimistic transactions. diff --git a/unreleased_history/public_api_changes/txn_get_coalescing_iterator.md b/unreleased_history/public_api_changes/txn_get_coalescing_iterator.md deleted file mode 100644 index 45dafd3025..0000000000 --- a/unreleased_history/public_api_changes/txn_get_coalescing_iterator.md +++ /dev/null @@ -1 +0,0 @@ -Added a new API `Transaction::GetCoalescingIterator` that can be used to create a multi-column-family coalescing iterator over the specified column families, including the data from both the transaction and the underlying database. This API is currently supported for optimistic and write-committed pessimistic transactions.