dolysis/rocksdb
3
0
Fork 0
mirror of https://github.com/facebook/rocksdb.git synced 2024-11-27 20:43:57 +00:00
Code Issues Packages Projects Releases Wiki Activity
rocksdb/java/rocksjni
History
Neil Ramaswamy 4835c11cce Add native logger support to RocksJava (#12213) 
Summary:
## Overview

In this PR, we introduce support for setting the RocksDB native logger through Java. As mentioned in the discussion on the [Google Group discussion](https://groups.google.com/g/rocksdb/c/xYmbEs4sqRM/m/e73E4whJAQAJ), this work is primarily motivated by the  JDK 17 [performance regression in JNI thread attach/detach calls](https://bugs.openjdk.org/browse/JDK-8314859): the only existing RocksJava logging configuration call, `setLogger`, invokes the provided logger over the JNI.

## Changes

Specifically, these changes add support for the `devnull` and `stderr` native loggers. For the `stderr` logger, we add the ability to prefix every log with a `logPrefix`, so that it becomes possible know which database a particular log is coming from (if multiple databases are in use). The  API looks like the following:

```java
Options opts = new Options();

NativeLogger stderrNativeLogger = NativeLogger.newStderrLogger(
  InfoLogLevel.DEBUG_LEVEL, "[my prefix here]");
options.setLogger(stderrNativeLogger);

try (final RocksDB db = RocksDB.open(options, ...))  {...}

// Cleanup
stderrNativeLogger.close()
opts.close();
```

Note that the API to set the logger is the same, via `Options::setLogger` (or `DBOptions::setLogger`). However, it will set the RocksDB logger to be native when  the provided logger is an instance of `NativeLogger`.

## Testing

Two tests have been added in `NativeLoggerTest.java`. The first test creates both the `devnull` and `stderr` loggers, and sets them on the associated `Options`. However, to avoid polluting the testing output with logs from `stderr`, only the `devnull` logger is actually used in the test. The second test does the same logic, but for `DBOptions`.

It is possible to manually verify the `stderr` logger by modifying the tests slightly, and observing that the console indeed gets cluttered with logs from `stderr`.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/12213

Reviewed By: cbi42

Differential Revision: D52772306

Pulled By: ajkr

fbshipit-source-id: 4026895f78f9cc250daf6bfa57427957e2d8b053
2024-01-17 17:51:36 -08:00
..
backup_engine_options.cc Fix remaining uses of "backupable" (#9792) 2022-04-05 09:52:33 -07:00
backupenginejni.cc Fix pointer to jlong conversion in 32 bit OS (#9396) 2022-03-01 09:02:15 -08:00
cache.cc Put Cache and CacheWrapper in new public header (#11192) 2023-02-09 12:12:02 -08:00
cassandra_compactionfilterjni.cc Fix pointer to jlong conversion in 32 bit OS (#9396) 2022-03-01 09:02:15 -08:00
cassandra_value_operator.cc Fix pointer to jlong conversion in 32 bit OS (#9396) 2022-03-01 09:02:15 -08:00
checkpoint.cc Add jni Support for API CreateColumnFamilyWithImport (#11646) 2023-11-06 07:38:42 -08:00
clock_cache.cc Revert NewClockCache signature (#10358) 2022-07-13 17:43:39 -07:00
columnfamilyhandle.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
compact_range_options.cc Implement missing compactrangeoptions from Java API (#10880) 2023-05-24 11:04:46 -07:00
compaction_filter.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
compaction_filter_factory.cc Fix pointer to jlong conversion in 32 bit OS (#9396) 2022-03-01 09:02:15 -08:00
compaction_filter_factory_jnicallback.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
compaction_filter_factory_jnicallback.h Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
compaction_job_info.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
compaction_job_stats.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
compaction_options.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
compaction_options_fifo.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
compaction_options_universal.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
comparator.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
comparatorjnicallback.cc Fix Java API ComparatorOptions use after delete error (#11176) 2023-02-17 13:03:41 -08:00
comparatorjnicallback.h Fix Java API ComparatorOptions use after delete error (#11176) 2023-02-17 13:03:41 -08:00
compression_options.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
concurrent_task_limiter.cc Add some missing headers (#10519) 2022-08-11 12:45:50 -07:00
config_options.cc Remove deprecated util functions in options_util.h (#11126) 2023-01-27 11:10:53 -08:00
cplusplus_to_java_convert.h Fix pointer to jlong conversion in 32 bit OS (#9396) 2022-03-01 09:02:15 -08:00
env.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
env_options.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
event_listener.cc Fix pointer to jlong conversion in 32 bit OS (#9396) 2022-03-01 09:02:15 -08:00
event_listener_jnicallback.cc Add (& fix) some simple source code checks (#8821) 2021-09-07 21:19:27 -07:00
event_listener_jnicallback.h Add (& fix) some simple source code checks (#8821) 2021-09-07 21:19:27 -07:00
export_import_files_metadatajni.cc Add jni Support for API CreateColumnFamilyWithImport (#11646) 2023-11-06 07:38:42 -08:00
filter.cc Fix pointer to jlong conversion in 32 bit OS (#9396) 2022-03-01 09:02:15 -08:00
hyper_clock_cache.cc Add HyperClockCache Java API. (#12065) 2023-11-16 15:46:31 -08:00
import_column_family_options.cc Add jni Support for API CreateColumnFamilyWithImport (#11646) 2023-11-06 07:38:42 -08:00
ingest_external_file_options.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
iterator.cc Add Iterator::Refresh(Snapshot*) to RocksJava (#12145) 2023-12-20 18:03:42 -08:00
jni_perf_context.cc Add missing copyright headers to files added in PR 11805 (#11942) 2023-10-11 12:56:39 -07:00
jnicallback.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
jnicallback.h Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
kv_helper.h Java API consistency between RocksDB.put() , .merge() and Transaction.put() , .merge() (#11019) 2023-12-11 11:03:17 -08:00
loggerjnicallback.cc Add native logger support to RocksJava (#12213) 2024-01-17 17:51:36 -08:00
loggerjnicallback.h Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
lru_cache.cc Use the default metadata charge policy when creating an LRU cache via the Java API (#10577) 2022-08-29 09:42:04 -07:00
memory_util.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
memtablejni.cc Fix pointer to jlong conversion in 32 bit OS (#9396) 2022-03-01 09:02:15 -08:00
merge_operator.cc Fix pointer to jlong conversion in 32 bit OS (#9396) 2022-03-01 09:02:15 -08:00
native_comparator_wrapper_test.cc Fix pointer to jlong conversion in 32 bit OS (#9396) 2022-03-01 09:02:15 -08:00
optimistic_transaction_db.cc RocksJava API - fix Transaction.multiGet() size limit, remove bogus EnsureLocalCapacity() calls (#10674) 2022-10-26 17:25:33 -07:00
optimistic_transaction_options.cc Fix pointer to jlong conversion in 32 bit OS (#9396) 2022-03-01 09:02:15 -08:00
options.cc Add native logger support to RocksJava (#12213) 2024-01-17 17:51:36 -08:00
options_util.cc java API - load block based table config (#10826) 2023-10-12 09:39:01 -07:00
persistent_cache.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
portal.h Group SST write in flush, compaction and db open with new stats (#11910) 2023-12-29 15:29:23 -08:00
ratelimiterjni.cc Fix pointer to jlong conversion in 32 bit OS (#9396) 2022-03-01 09:02:15 -08:00
remove_emptyvalue_compactionfilterjni.cc Fix pointer to jlong conversion in 32 bit OS (#9396) 2022-03-01 09:02:15 -08:00
restorejni.cc Fix pointer to jlong conversion in 32 bit OS (#9396) 2022-03-01 09:02:15 -08:00
rocks_callback_object.cc Fix tabs and lint-ignores (#6734) 2020-04-20 11:39:31 -07:00
rocksdb_exception_test.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
rocksjni.cc Java API consistency between RocksDB.put() , .merge() and Transaction.put() , .merge() (#11019) 2023-12-11 11:03:17 -08:00
slice.cc Fix pointer to jlong conversion in 32 bit OS (#9396) 2022-03-01 09:02:15 -08:00
snapshot.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
sst_file_manager.cc Fix pointer to jlong conversion in 32 bit OS (#9396) 2022-03-01 09:02:15 -08:00
sst_file_reader_iterator.cc Add Iterator::Refresh(Snapshot*) to RocksJava (#12145) 2023-12-20 18:03:42 -08:00
sst_file_readerjni.cc Fix pointer to jlong conversion in 32 bit OS (#9396) 2022-03-01 09:02:15 -08:00
sst_file_writerjni.cc Fix pointer to jlong conversion in 32 bit OS (#9396) 2022-03-01 09:02:15 -08:00
sst_partitioner.cc Fix pointer to jlong conversion in 32 bit OS (#9396) 2022-03-01 09:02:15 -08:00
statistics.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
statisticsjni.cc Fix tabs and lint-ignores (#6734) 2020-04-20 11:39:31 -07:00
statisticsjni.h Change internal headers with duplicate names (#11408) 2023-05-17 11:27:09 -07:00
stderr_logger.cc Add native logger support to RocksJava (#12213) 2024-01-17 17:51:36 -08:00
table.cc Remove compressed block cache (#11117) 2023-01-24 17:09:19 -08:00
table_filter.cc Fix pointer to jlong conversion in 32 bit OS (#9396) 2022-03-01 09:02:15 -08:00
table_filter_jnicallback.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
table_filter_jnicallback.h Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
table_properties_collector_factory.cc Fix header files to meet Open source requirements (#12164) 2023-12-19 13:43:17 -08:00
table_properties_collector_factory.h Fix header files to meet Open source requirements (#12164) 2023-12-19 13:43:17 -08:00
testable_event_listener.cc Update build files for java8 build (#9541) 2022-02-17 13:29:21 -08:00
thread_status.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
trace_writer.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
trace_writer_jnicallback.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
trace_writer_jnicallback.h Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
transaction.cc Java API consistency between RocksDB.put() , .merge() and Transaction.put() , .merge() (#11019) 2023-12-11 11:03:17 -08:00
transaction_db.cc RocksJava API - fix Transaction.multiGet() size limit, remove bogus EnsureLocalCapacity() calls (#10674) 2022-10-26 17:25:33 -07:00
transaction_db_options.cc Fix pointer to jlong conversion in 32 bit OS (#9396) 2022-03-01 09:02:15 -08:00
transaction_log.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
transaction_notifier.cc Fix pointer to jlong conversion in 32 bit OS (#9396) 2022-03-01 09:02:15 -08:00
transaction_notifier_jnicallback.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
transaction_notifier_jnicallback.h Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
transaction_options.cc Fix pointer to jlong conversion in 32 bit OS (#9396) 2022-03-01 09:02:15 -08:00
ttl.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
wal_filter.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
wal_filter_jnicallback.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
wal_filter_jnicallback.h Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
write_batch.cc Fix pointer to jlong conversion in 32 bit OS (#9396) 2022-03-01 09:02:15 -08:00
write_batch_test.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
write_batch_with_index.cc Add Iterator::Refresh(Snapshot*) to RocksJava (#12145) 2023-12-20 18:03:42 -08:00
write_buffer_manager.cc Put Cache and CacheWrapper in new public header (#11192) 2023-02-09 12:12:02 -08:00
writebatchhandlerjnicallback.cc Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00
writebatchhandlerjnicallback.h Run format check for *.h and *.cc files under java/ (#10851) 2022-10-25 09:26:51 -07:00