Update HISTORY.md and version.h for 8.6 (#11728)

Summary: Pull Request resolved: https://github.com/facebook/rocksdb/pull/11728 Reviewed By: jaykorean, jowlyzhang Differential Revision: D48527100 Pulled By: anand1976 fbshipit-source-id: c48baa44e538fb6bfd3fe7f19046746d3540763f
2024-11-28 15:33:54 +00:00 · 2023-08-21 13:25:04 -07:00 · 2023-08-21 13:25:04 -07:00 · 4b53520709
parent 4fa2c01719
commit 4b53520709
19 changed files with 27 additions and 18 deletions
--- a/HISTORY.md
+++ b/HISTORY.md
@ -1,6 +1,31 @@
 # Rocksdb Change Log
 > NOTE: Entries for next release do not go here. Follow instructions in `unreleased_history/README.txt`

+## 8.6.0 (08/18/2023)
+### New Features
+* Added enhanced data integrity checking on SST files with new format_version=6. Performance impact is very small or negligible. Previously if SST data was misplaced or re-arranged by the storage layer, it could pass block checksum with higher than 1 in 4 billion probability. With format_version=6, block checksums depend on what file they are in and location within the file. This way, misplaced SST data is no more likely to pass checksum verification than randomly corrupted data. Also in format_version=6, SST footers are checksum-protected.
+* Add a new feature to trim readahead_size during scans upto upper_bound when iterate_upper_bound is specified. It's enabled through ReadOptions.auto_readahead_size. Users must also specify ReadOptions.iterate_upper_bound.
+* RocksDB will compare the number of input keys to the number of keys processed after each compaction. Compaction will fail and report Corruption status if the verification fails. Option `compaction_verify_record_count` is introduced for this purpose and is enabled by default.
+* Add a CF option `bottommost_file_compaction_delay` to allow specifying the delay of bottommost level single-file compactions.
+* Add support to allow enabling / disabling user-defined timestamps feature for an existing column family in combination with the in-Memtable only feature.
+* Implement a new admission policy for the compressed secondary cache that admits blocks evicted from the primary cache with the hit bit set. This policy can be specified in TieredVolatileCacheOptions by setting the newly added adm_policy option.
+* Add a column family option `memtable_max_range_deletions` that limits the number of range deletions in a memtable. RocksDB will try to do an automatic flush after the limit is reached. (#11358)
+* Add PutEntity API in sst_file_writer
+* Add `timeout` in microsecond option to `WaitForCompactOptions` to allow timely termination of prolonged waiting in scenarios like recurring recoverable errors, such as out-of-space situations and continuous write streams that sustain ongoing flush and compactions
+* New statistics `rocksdb.file.read.{db.open|get|multiget|db.iterator|verify.checksum|verify.file.checksums}.micros` measure read time of block-based SST tables or blob files during db open, `Get()`, `MultiGet()`, using db iterator, `VerifyFileChecksums()` and `VerifyChecksum()`. They require stats level greater than `StatsLevel::kExceptDetailedTimers`.
+* Add close_db option to `WaitForCompactOptions` to call Close() after waiting is done.
+* Add a new compression option `CompressionOptions::checksum` for enabling ZSTD's checksum feature to detect corruption during decompression.
+
+### Public API Changes
+* Mark `Options::access_hint_on_compaction_start` related APIs as deprecated. See #11631 for alternative behavior.
+
+### Behavior Changes
+* Statistics `rocksdb.sst.read.micros` now includes time spent on multi read and async read into the file
+* For Universal Compaction users, periodic compaction (option `periodic_compaction_seconds`) will be set to 30 days by default if block based table is used.
+
+### Bug Fixes
+* Fix a bug in FileTTLBooster that can cause users with a large number of levels (more than 65) to see errors like "runtime error: shift exponent .. is too large.." (#11673).
+
 ## 8.5.0 (07/21/2023)
 ### Public API Changes
 * Removed recently added APIs `GeneralCache` and `MakeSharedGeneralCache()` as our plan changed to stop exposing a general-purpose cache interface. The old forms of these APIs, `Cache` and `NewLRUCache()`, are still available, although general-purpose caching support will be dropped eventually.
--- a/include/rocksdb/version.h
+++ b/include/rocksdb/version.h
@ -12,7 +12,7 @@
 // NOTE: in 'main' development branch, this should be the *next*
 // minor or major version number planned for release.
 #define ROCKSDB_MAJOR 8
-#define ROCKSDB_MINOR 6
+#define ROCKSDB_MINOR 7
 #define ROCKSDB_PATCH 0

 // Do not use these. We made the mistake of declaring macros starting with
--- a/tools/check_format_compatible.sh
+++ b/tools/check_format_compatible.sh
@ -125,7 +125,7 @@ EOF

 # To check for DB forward compatibility with loading options (old version
 # reading data from new), as well as backward compatibility
-declare -a db_forward_with_options_refs=("6.27.fb" "6.28.fb" "6.29.fb" "7.0.fb" "7.1.fb" "7.2.fb" "7.3.fb" "7.4.fb" "7.5.fb" "7.6.fb" "7.7.fb" "7.8.fb" "7.9.fb" "7.10.fb" "8.0.fb" "8.1.fb" "8.2.fb" "8.3.fb" "8.4.fb" "8.5.fb")
+declare -a db_forward_with_options_refs=("6.27.fb" "6.28.fb" "6.29.fb" "7.0.fb" "7.1.fb" "7.2.fb" "7.3.fb" "7.4.fb" "7.5.fb" "7.6.fb" "7.7.fb" "7.8.fb" "7.9.fb" "7.10.fb" "8.0.fb" "8.1.fb" "8.2.fb" "8.3.fb" "8.4.fb" "8.5.fb" "8.6.fb")
 # To check for DB forward compatibility without loading options (in addition
 # to the "with loading options" set), as well as backward compatibility
 declare -a db_forward_no_options_refs=() # N/A at the moment
--- a/unreleased_history/behavior_changes/more_sst_read_micros.md
+++ b/unreleased_history/behavior_changes/more_sst_read_micros.md
@ -1 +0,0 @@
-Statistics `rocksdb.sst.read.micros` now includes time spent on multi read and async read into the file
--- a/unreleased_history/behavior_changes/universal_ttl_periodic_compaction.md
+++ b/unreleased_history/behavior_changes/universal_ttl_periodic_compaction.md
@ -1 +0,0 @@
-For Universal Compaction users, periodic compaction (option `periodic_compaction_seconds`) will be set to 30 days by default if block based table is used.
--- a/unreleased_history/bug_fixes/shifting_componeng_too_large_file_ttl_booster.md
+++ b/unreleased_history/bug_fixes/shifting_componeng_too_large_file_ttl_booster.md
@ -1 +0,0 @@
-Fix a bug in FileTTLBooster that can cause users with a large number of levels (more than 65) to see errors like "runtime error: shift exponent .. is too large.." (#11673).
--- a/unreleased_history/new_features/1_context_checksum.md
+++ b/unreleased_history/new_features/1_context_checksum.md
@ -1 +0,0 @@
-* Added enhanced data integrity checking on SST files with new format_version=6. Performance impact is very small or negligible. Previously if SST data was misplaced or re-arranged by the storage layer, it could pass block checksum with higher than 1 in 4 billion probability. With format_version=6, block checksums depend on what file they are in and location within the file. This way, misplaced SST data is no more likely to pass checksum verification than randomly corrupted data. Also in format_version=6, SST footers are checksum-protected.
--- a/unreleased_history/new_features/auto_readahead.md
+++ b/unreleased_history/new_features/auto_readahead.md
@ -1 +0,0 @@
-Add a new feature to trim readahead_size during scans upto upper_bound when iterate_upper_bound is specified. It's enabled through ReadOptions.auto_readahead_size. Users must also specify ReadOptions.iterate_upper_bound.
--- a/unreleased_history/new_features/compaction_verify_input_count.md
+++ b/unreleased_history/new_features/compaction_verify_input_count.md
@ -1 +0,0 @@
-* RocksDB will compare the number of input keys to the number of keys processed after each compaction. Compaction will fail and report Corruption status if the verification fails. Option `compaction_verify_record_count` is introduced for this purpose and is enabled by default.
--- a/unreleased_history/new_features/delay_bottommost_single_file_compaction.md
+++ b/unreleased_history/new_features/delay_bottommost_single_file_compaction.md
@ -1 +0,0 @@
-Add a CF option `bottommost_file_compaction_delay` to allow specifying the delay of bottommost level single-file compactions.
--- a/unreleased_history/new_features/enable_disable_udt.md
+++ b/unreleased_history/new_features/enable_disable_udt.md
@ -1 +0,0 @@
-Add support to allow enabling / disabling user-defined timestamps feature for an existing column family in combination with the in-Memtable only feature.
--- a/unreleased_history/new_features/implement_allow_cache_hit_admission_policy.md
+++ b/unreleased_history/new_features/implement_allow_cache_hit_admission_policy.md
@ -1 +0,0 @@
-Implement a new admission policy for the compressed secondary cache that admits blocks evicted from the primary cache with the hit bit set. This policy can be specified in TieredVolatileCacheOptions by setting the newly added adm_policy option.
--- a/unreleased_history/new_features/memetable_range_del_limit.md
+++ b/unreleased_history/new_features/memetable_range_del_limit.md
@ -1 +0,0 @@
-Add a column family option `memtable_max_range_deletions` that limits the number of range deletions in a memtable. RocksDB will try to do an automatic flush after the limit is reached. (#11358)
--- a/unreleased_history/new_features/put_entity_support_in_sst_file_writer.md
+++ b/unreleased_history/new_features/put_entity_support_in_sst_file_writer.md
@ -1 +0,0 @@
-Add PutEntity API in sst_file_writer
--- a/unreleased_history/new_features/timeout_for_wait_for_compact_api.md
+++ b/unreleased_history/new_features/timeout_for_wait_for_compact_api.md
@ -1 +0,0 @@
-Add `timeout` in microsecond option to `WaitForCompactOptions` to allow timely termination of prolonged waiting in scenarios like recurring recoverable errors, such as out-of-space situations and continuous write streams that sustain ongoing flush and compactions
--- a/unreleased_history/new_features/user_read_io_activity_stats.md
+++ b/unreleased_history/new_features/user_read_io_activity_stats.md
@ -1 +0,0 @@
-New statistics `rocksdb.file.read.{db.open|get|multiget|db.iterator|verify.checksum|verify.file.checksums}.micros` measure read time of block-based SST tables or blob files during db open, `Get()`, `MultiGet()`, using db iterator, `VerifyFileChecksums()` and `VerifyChecksum()`. They require stats level greater than `StatsLevel::kExceptDetailedTimers`.
--- a/unreleased_history/new_features/wait_for_compact_close_db_option.md
+++ b/unreleased_history/new_features/wait_for_compact_close_db_option.md
@ -1 +0,0 @@
-Add close_db option to `WaitForCompactOptions` to call Close() after waiting is done.
--- a/unreleased_history/new_features/zstd-checksum.md
+++ b/unreleased_history/new_features/zstd-checksum.md
@ -1 +0,0 @@
-* Add a new compression option `CompressionOptions::checksum` for enabling ZSTD's checksum feature to detect corruption during decompression.
--- a/unreleased_history/public_api_changes/mark_dep_access_hint_on_compaction_start.md
+++ b/unreleased_history/public_api_changes/mark_dep_access_hint_on_compaction_start.md
@ -1 +0,0 @@
-Mark `Options::access_hint_on_compaction_start` related APIs as deprecated. See #11631 for alternative behavior.
				`@ -1 +0,0 @@`
				Statistics `rocksdb.sst.read.micros` now includes time spent on multi read and async read into the file
				`@ -1 +0,0 @@`
				For Universal Compaction users, periodic compaction (option `periodic_compaction_seconds`) will be set to 30 days by default if block based table is used.
				`@ -1 +0,0 @@`
				`Fix a bug in FileTTLBooster that can cause users with a large number of levels (more than 65) to see errors like "runtime error: shift exponent .. is too large.." (#11673).`
				`@ -1 +0,0 @@`
				* Added enhanced data integrity checking on SST files with new format_version=6. Performance impact is very small or negligible. Previously if SST data was misplaced or re-arranged by the storage layer, it could pass block checksum with higher than 1 in 4 billion probability. With format_version=6, block checksums depend on what file they are in and location within the file. This way, misplaced SST data is no more likely to pass checksum verification than randomly corrupted data. Also in format_version=6, SST footers are checksum-protected.
				`@ -1 +0,0 @@`
				`Add a new feature to trim readahead_size during scans upto upper_bound when iterate_upper_bound is specified. It's enabled through ReadOptions.auto_readahead_size. Users must also specify ReadOptions.iterate_upper_bound.`
				`@ -1 +0,0 @@`
				* RocksDB will compare the number of input keys to the number of keys processed after each compaction. Compaction will fail and report Corruption status if the verification fails. Option `compaction_verify_record_count` is introduced for this purpose and is enabled by default.
				`@ -1 +0,0 @@`
				Add a CF option `bottommost_file_compaction_delay` to allow specifying the delay of bottommost level single-file compactions.
				`@ -1 +0,0 @@`
				`Add support to allow enabling / disabling user-defined timestamps feature for an existing column family in combination with the in-Memtable only feature.`
				`@ -1 +0,0 @@`
				`Implement a new admission policy for the compressed secondary cache that admits blocks evicted from the primary cache with the hit bit set. This policy can be specified in TieredVolatileCacheOptions by setting the newly added adm_policy option.`
				`@ -1 +0,0 @@`
				Add a column family option `memtable_max_range_deletions` that limits the number of range deletions in a memtable. RocksDB will try to do an automatic flush after the limit is reached. (#11358)