dolysis/rocksdb
3
0
Fork 0
mirror of https://github.com/facebook/rocksdb.git synced 2024-12-04 02:02:41 +00:00
Code Issues Packages Projects Releases Wiki Activity
rocksdb/java/rocksjni
History
Peter Dillinger a1a102ffce Steps toward deprecating implicit prefix seek, related fixes (#13026) 
Summary:
With some new use cases onboarding to prefix extractors/seek/filters, one of the risks is existing iterator code, e.g. for maintenance tasks, being unintentionally subject to prefix seek semantics. This is a longstanding known design flaw with prefix seek, and `prefix_same_as_start` and `auto_prefix_mode` were steps in the direction of making that obsolete. However, we can't just immediately set `total_order_seek` to true by default, because that would impact so much code instantly.

Here we add a new DB option, `prefix_seek_opt_in_only` that basically allows users to transition to the future behavior when they are ready. When set to true, all iterators will be treated as if `total_order_seek=true` and then the only ways to get prefix seek semantics are with `prefix_same_as_start` or `auto_prefix_mode`.

Related fixes / changes:
* Make sure that `prefix_same_as_start` and `auto_prefix_mode` are compatible with (or override) `total_order_seek` (depending on your interpretation).
* Fix a bug in which a new iterator after dynamically changing the prefix extractor might mix different prefix semantics between memtable and SSTs. Both should use the latest extractor semantics, which means iterators ignoring memtable prefix filters with an old extractor. And that means passing the latest prefix extractor to new memtable iterators that might use prefix seek. (Without the fix, the test added for this fails in many ways.)

Suggested follow-up:
* Investigate a FIXME where a MergeIteratorBuilder is created in db_impl.cc. No unit test detects a change in value that should impact correctness.
* Make memtable prefix bloom compatible with `auto_prefix_mode`, which might require involving the memtablereps because we don't know at iterator creation time (only seek time) whether an auto_prefix_mode seek will be a prefix seek.
* Add `prefix_same_as_start` testing to db_stress

Pull Request resolved: https://github.com/facebook/rocksdb/pull/13026

Test Plan:
tests updated, added. Add combination of `total_order_seek=true` and `auto_prefix_mode=true` to stress test. Ran `make blackbox_crash_test` for a long while.

Manually ran tests with `prefix_seek_opt_in_only=true` as default, looking for unexpected issues. I inspected most of the results and migrated many tests to be ready for such a change (but not all).

Reviewed By: ltamasi

Differential Revision: D63147378

Pulled By: pdillinger

fbshipit-source-id: 1f4477b730683d43b4be7e933338583702d3c25e
2024-09-20 15:54:19 -07:00
..
backup_engine_options.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
backupenginejni.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
cache.cc Put Cache and CacheWrapper in new public header (#11192) 2023-02-09 12:12:02 -08:00
cassandra_compactionfilterjni.cc Fix pointer to jlong conversion in 32 bit OS (#9396) 2022-03-01 09:02:15 -08:00
cassandra_value_operator.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
checkpoint.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
clock_cache.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
columnfamilyhandle.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
compact_range_options.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
compaction_filter.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
compaction_filter_factory.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
compaction_filter_factory_jnicallback.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
compaction_filter_factory_jnicallback.h Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
compaction_job_info.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
compaction_job_stats.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
compaction_options.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
compaction_options_fifo.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
compaction_options_universal.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
comparator.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
comparatorjnicallback.cc Fix Java API ComparatorOptions use after delete error (#11176) 2023-02-17 13:03:41 -08:00
comparatorjnicallback.h Fix Java API ComparatorOptions use after delete error (#11176) 2023-02-17 13:03:41 -08:00
compression_options.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
concurrent_task_limiter.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
config_options.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
cplusplus_to_java_convert.h Fix pointer to jlong conversion in 32 bit OS (#9396) 2022-03-01 09:02:15 -08:00
env.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
env_options.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
event_listener.cc Fix pointer to jlong conversion in 32 bit OS (#9396) 2022-03-01 09:02:15 -08:00
event_listener_jnicallback.cc Add (& fix) some simple source code checks (#8821) 2021-09-07 21:19:27 -07:00
event_listener_jnicallback.h Add (& fix) some simple source code checks (#8821) 2021-09-07 21:19:27 -07:00
export_import_files_metadatajni.cc Add jni Support for API CreateColumnFamilyWithImport (#11646) 2023-11-06 07:38:42 -08:00
filter.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
hyper_clock_cache.cc Add HyperClockCache Java API. (#12065) 2023-11-16 15:46:31 -08:00
import_column_family_options.cc Add jni Support for API CreateColumnFamilyWithImport (#11646) 2023-11-06 07:38:42 -08:00
ingest_external_file_options.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
iterator.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
jni_multiget_helpers.cc JNI get_helper code sharing / multiGet() use efficient batch C++ support (#12344) 2024-03-12 12:42:08 -07:00
jni_multiget_helpers.h JNI get_helper code sharing / multiGet() use efficient batch C++ support (#12344) 2024-03-12 12:42:08 -07:00
jni_perf_context.cc Implement PerfContex#toString for the Java API (#12473) 2024-04-03 14:33:31 -07:00
jnicallback.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
jnicallback.h Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
kv_helper.h Fix deprecated dynamic exception in internal_repo_rocksdb/repo/java/rocksjni/kv_helper.h +1 2024-06-13 12:41:13 -07:00
loggerjnicallback.cc Add native logger support to RocksJava (#12213) 2024-01-17 17:51:36 -08:00
loggerjnicallback.h Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
lru_cache.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
memory_util.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
memtablejni.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
merge_operator.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
native_comparator_wrapper_test.cc Fix pointer to jlong conversion in 32 bit OS (#9396) 2022-03-01 09:02:15 -08:00
optimistic_transaction_db.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
optimistic_transaction_options.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
options.cc use nullptr instead of NULL / 0 in rocksdbjni (#12575) 2024-05-21 12:56:07 -07:00
options_util.cc java API - load block based table config (#10826) 2023-10-12 09:39:01 -07:00
persistent_cache.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
portal.h Add ticker stats for read corruption retries (#12923) 2024-08-12 15:32:07 -07:00
ratelimiterjni.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
remove_emptyvalue_compactionfilterjni.cc Fix pointer to jlong conversion in 32 bit OS (#9396) 2022-03-01 09:02:15 -08:00
restorejni.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
rocks_callback_object.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
rocksdb_exception_test.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
rocksjni.cc JNI get_helper code sharing / multiGet() use efficient batch C++ support (#12344) 2024-03-12 12:42:08 -07:00
slice.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
snapshot.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
sst_file_manager.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
sst_file_reader_iterator.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
sst_file_readerjni.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
sst_file_writerjni.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
sst_partitioner.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
statistics.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
statisticsjni.cc Fix tabs and lint-ignores (#6734) 2020-04-20 11:39:31 -07:00
statisticsjni.h Remove 'virtual' when implied by 'override' (#12319) 2024-01-31 13:14:42 -08:00
stderr_logger.cc Add native logger support to RocksJava (#12213) 2024-01-17 17:51:36 -08:00
table.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
table_filter.cc Fix pointer to jlong conversion in 32 bit OS (#9396) 2022-03-01 09:02:15 -08:00
table_filter_jnicallback.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
table_filter_jnicallback.h Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
table_properties_collector_factory.cc Fix header files to meet Open source requirements (#12164) 2023-12-19 13:43:17 -08:00
table_properties_collector_factory.h Fix header files to meet Open source requirements (#12164) 2023-12-19 13:43:17 -08:00
testable_event_listener.cc Fix build on OpenBSD i386 (#12142) 2024-01-29 16:19:59 -08:00
thread_status.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
trace_writer.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
trace_writer_jnicallback.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
trace_writer_jnicallback.h Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
transaction.cc JNI get_helper code sharing / multiGet() use efficient batch C++ support (#12344) 2024-03-12 12:42:08 -07:00
transaction_db.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
transaction_db_options.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
transaction_log.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
transaction_notifier.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
transaction_notifier_jnicallback.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
transaction_notifier_jnicallback.h Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
transaction_options.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
ttl.cc use nullptr instead of NULL / 0 in rocksdbjni (#12575) 2024-05-21 12:56:07 -07:00
wal_filter.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
wal_filter_jnicallback.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
wal_filter_jnicallback.h Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
write_batch.cc Replace ScopedArenaIterator with ScopedArenaPtr<InternalIterator> (#12470) 2024-03-22 13:40:42 -07:00
write_batch_test.cc Steps toward deprecating implicit prefix seek, related fixes (#13026) 2024-09-20 15:54:19 -07:00
write_batch_with_index.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
write_buffer_manager.cc Change Java native methods to static (#11882) 2024-01-25 12:36:30 -08:00
writebatchhandlerjnicallback.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
writebatchhandlerjnicallback.h Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00