rocksdb/util/stderr_logger.cc

63 lines
2.2 KiB
C++
Raw Normal View History

// Copyright (c) Meta Platforms, Inc. and affiliates.
//
// This source code is licensed under both the GPLv2 (found in the
// COPYING file in the root directory) and Apache 2.0 License
// (found in the LICENSE.Apache file in the root directory).
#include "util/stderr_logger.h"
Add native logger support to RocksJava (#12213) Summary: ## Overview In this PR, we introduce support for setting the RocksDB native logger through Java. As mentioned in the discussion on the [Google Group discussion](https://groups.google.com/g/rocksdb/c/xYmbEs4sqRM/m/e73E4whJAQAJ), this work is primarily motivated by the JDK 17 [performance regression in JNI thread attach/detach calls](https://bugs.openjdk.org/browse/JDK-8314859): the only existing RocksJava logging configuration call, `setLogger`, invokes the provided logger over the JNI. ## Changes Specifically, these changes add support for the `devnull` and `stderr` native loggers. For the `stderr` logger, we add the ability to prefix every log with a `logPrefix`, so that it becomes possible know which database a particular log is coming from (if multiple databases are in use). The API looks like the following: ```java Options opts = new Options(); NativeLogger stderrNativeLogger = NativeLogger.newStderrLogger( InfoLogLevel.DEBUG_LEVEL, "[my prefix here]"); options.setLogger(stderrNativeLogger); try (final RocksDB db = RocksDB.open(options, ...)) {...} // Cleanup stderrNativeLogger.close() opts.close(); ``` Note that the API to set the logger is the same, via `Options::setLogger` (or `DBOptions::setLogger`). However, it will set the RocksDB logger to be native when the provided logger is an instance of `NativeLogger`. ## Testing Two tests have been added in `NativeLoggerTest.java`. The first test creates both the `devnull` and `stderr` loggers, and sets them on the associated `Options`. However, to avoid polluting the testing output with logs from `stderr`, only the `devnull` logger is actually used in the test. The second test does the same logic, but for `DBOptions`. It is possible to manually verify the `stderr` logger by modifying the tests slightly, and observing that the console indeed gets cluttered with logs from `stderr`. Pull Request resolved: https://github.com/facebook/rocksdb/pull/12213 Reviewed By: cbi42 Differential Revision: D52772306 Pulled By: ajkr fbshipit-source-id: 4026895f78f9cc250daf6bfa57427957e2d8b053
2024-01-18 01:51:36 +00:00
#include "port/malloc.h"
#include "port/sys_time.h"
namespace ROCKSDB_NAMESPACE {
Add native logger support to RocksJava (#12213) Summary: ## Overview In this PR, we introduce support for setting the RocksDB native logger through Java. As mentioned in the discussion on the [Google Group discussion](https://groups.google.com/g/rocksdb/c/xYmbEs4sqRM/m/e73E4whJAQAJ), this work is primarily motivated by the JDK 17 [performance regression in JNI thread attach/detach calls](https://bugs.openjdk.org/browse/JDK-8314859): the only existing RocksJava logging configuration call, `setLogger`, invokes the provided logger over the JNI. ## Changes Specifically, these changes add support for the `devnull` and `stderr` native loggers. For the `stderr` logger, we add the ability to prefix every log with a `logPrefix`, so that it becomes possible know which database a particular log is coming from (if multiple databases are in use). The API looks like the following: ```java Options opts = new Options(); NativeLogger stderrNativeLogger = NativeLogger.newStderrLogger( InfoLogLevel.DEBUG_LEVEL, "[my prefix here]"); options.setLogger(stderrNativeLogger); try (final RocksDB db = RocksDB.open(options, ...)) {...} // Cleanup stderrNativeLogger.close() opts.close(); ``` Note that the API to set the logger is the same, via `Options::setLogger` (or `DBOptions::setLogger`). However, it will set the RocksDB logger to be native when the provided logger is an instance of `NativeLogger`. ## Testing Two tests have been added in `NativeLoggerTest.java`. The first test creates both the `devnull` and `stderr` loggers, and sets them on the associated `Options`. However, to avoid polluting the testing output with logs from `stderr`, only the `devnull` logger is actually used in the test. The second test does the same logic, but for `DBOptions`. It is possible to manually verify the `stderr` logger by modifying the tests slightly, and observing that the console indeed gets cluttered with logs from `stderr`. Pull Request resolved: https://github.com/facebook/rocksdb/pull/12213 Reviewed By: cbi42 Differential Revision: D52772306 Pulled By: ajkr fbshipit-source-id: 4026895f78f9cc250daf6bfa57427957e2d8b053
2024-01-18 01:51:36 +00:00
StderrLogger::~StderrLogger() {
if (log_prefix != nullptr) {
free((void*)log_prefix);
}
}
void StderrLogger::Logv(const char* format, va_list ap) {
const uint64_t thread_id = Env::Default()->GetThreadID();
port::TimeVal now_tv;
port::GetTimeOfDay(&now_tv, nullptr);
const time_t seconds = now_tv.tv_sec;
struct tm t;
port::LocalTimeR(&seconds, &t);
Add native logger support to RocksJava (#12213) Summary: ## Overview In this PR, we introduce support for setting the RocksDB native logger through Java. As mentioned in the discussion on the [Google Group discussion](https://groups.google.com/g/rocksdb/c/xYmbEs4sqRM/m/e73E4whJAQAJ), this work is primarily motivated by the JDK 17 [performance regression in JNI thread attach/detach calls](https://bugs.openjdk.org/browse/JDK-8314859): the only existing RocksJava logging configuration call, `setLogger`, invokes the provided logger over the JNI. ## Changes Specifically, these changes add support for the `devnull` and `stderr` native loggers. For the `stderr` logger, we add the ability to prefix every log with a `logPrefix`, so that it becomes possible know which database a particular log is coming from (if multiple databases are in use). The API looks like the following: ```java Options opts = new Options(); NativeLogger stderrNativeLogger = NativeLogger.newStderrLogger( InfoLogLevel.DEBUG_LEVEL, "[my prefix here]"); options.setLogger(stderrNativeLogger); try (final RocksDB db = RocksDB.open(options, ...)) {...} // Cleanup stderrNativeLogger.close() opts.close(); ``` Note that the API to set the logger is the same, via `Options::setLogger` (or `DBOptions::setLogger`). However, it will set the RocksDB logger to be native when the provided logger is an instance of `NativeLogger`. ## Testing Two tests have been added in `NativeLoggerTest.java`. The first test creates both the `devnull` and `stderr` loggers, and sets them on the associated `Options`. However, to avoid polluting the testing output with logs from `stderr`, only the `devnull` logger is actually used in the test. The second test does the same logic, but for `DBOptions`. It is possible to manually verify the `stderr` logger by modifying the tests slightly, and observing that the console indeed gets cluttered with logs from `stderr`. Pull Request resolved: https://github.com/facebook/rocksdb/pull/12213 Reviewed By: cbi42 Differential Revision: D52772306 Pulled By: ajkr fbshipit-source-id: 4026895f78f9cc250daf6bfa57427957e2d8b053
2024-01-18 01:51:36 +00:00
// The string we eventually log has three parts: the context (time, thread),
// optional user-supplied prefix, and the actual log message (the "suffix").
//
// We compute their lengths so that we can allocate a buffer big enough to
// print it. The context string (with the date and thread id) is really only
// 44 bytes, but we allocate 50 to be safe.
//
// ctx_len = 44 = ( 4+ 1+ 2+1+2+ 1+2+ 1+2+ 1+ 2+1+6+ 1+16+1)
const char* ctx_prefix_fmt = "%04d/%02d/%02d-%02d:%02d:%02d.%06d %llx %s";
size_t ctx_len = 50;
va_list ap_copy;
va_copy(ap_copy, ap);
const size_t log_suffix_len = vsnprintf(nullptr, 0, format, ap_copy) + 1;
Add native logger support to RocksJava (#12213) Summary: ## Overview In this PR, we introduce support for setting the RocksDB native logger through Java. As mentioned in the discussion on the [Google Group discussion](https://groups.google.com/g/rocksdb/c/xYmbEs4sqRM/m/e73E4whJAQAJ), this work is primarily motivated by the JDK 17 [performance regression in JNI thread attach/detach calls](https://bugs.openjdk.org/browse/JDK-8314859): the only existing RocksJava logging configuration call, `setLogger`, invokes the provided logger over the JNI. ## Changes Specifically, these changes add support for the `devnull` and `stderr` native loggers. For the `stderr` logger, we add the ability to prefix every log with a `logPrefix`, so that it becomes possible know which database a particular log is coming from (if multiple databases are in use). The API looks like the following: ```java Options opts = new Options(); NativeLogger stderrNativeLogger = NativeLogger.newStderrLogger( InfoLogLevel.DEBUG_LEVEL, "[my prefix here]"); options.setLogger(stderrNativeLogger); try (final RocksDB db = RocksDB.open(options, ...)) {...} // Cleanup stderrNativeLogger.close() opts.close(); ``` Note that the API to set the logger is the same, via `Options::setLogger` (or `DBOptions::setLogger`). However, it will set the RocksDB logger to be native when the provided logger is an instance of `NativeLogger`. ## Testing Two tests have been added in `NativeLoggerTest.java`. The first test creates both the `devnull` and `stderr` loggers, and sets them on the associated `Options`. However, to avoid polluting the testing output with logs from `stderr`, only the `devnull` logger is actually used in the test. The second test does the same logic, but for `DBOptions`. It is possible to manually verify the `stderr` logger by modifying the tests slightly, and observing that the console indeed gets cluttered with logs from `stderr`. Pull Request resolved: https://github.com/facebook/rocksdb/pull/12213 Reviewed By: cbi42 Differential Revision: D52772306 Pulled By: ajkr fbshipit-source-id: 4026895f78f9cc250daf6bfa57427957e2d8b053
2024-01-18 01:51:36 +00:00
va_end(ap_copy);
// Allocate space for the context, log_prefix, and log itself
// Extra byte for null termination
size_t buf_len = ctx_len + log_prefix_len + log_suffix_len;
Add native logger support to RocksJava (#12213) Summary: ## Overview In this PR, we introduce support for setting the RocksDB native logger through Java. As mentioned in the discussion on the [Google Group discussion](https://groups.google.com/g/rocksdb/c/xYmbEs4sqRM/m/e73E4whJAQAJ), this work is primarily motivated by the JDK 17 [performance regression in JNI thread attach/detach calls](https://bugs.openjdk.org/browse/JDK-8314859): the only existing RocksJava logging configuration call, `setLogger`, invokes the provided logger over the JNI. ## Changes Specifically, these changes add support for the `devnull` and `stderr` native loggers. For the `stderr` logger, we add the ability to prefix every log with a `logPrefix`, so that it becomes possible know which database a particular log is coming from (if multiple databases are in use). The API looks like the following: ```java Options opts = new Options(); NativeLogger stderrNativeLogger = NativeLogger.newStderrLogger( InfoLogLevel.DEBUG_LEVEL, "[my prefix here]"); options.setLogger(stderrNativeLogger); try (final RocksDB db = RocksDB.open(options, ...)) {...} // Cleanup stderrNativeLogger.close() opts.close(); ``` Note that the API to set the logger is the same, via `Options::setLogger` (or `DBOptions::setLogger`). However, it will set the RocksDB logger to be native when the provided logger is an instance of `NativeLogger`. ## Testing Two tests have been added in `NativeLoggerTest.java`. The first test creates both the `devnull` and `stderr` loggers, and sets them on the associated `Options`. However, to avoid polluting the testing output with logs from `stderr`, only the `devnull` logger is actually used in the test. The second test does the same logic, but for `DBOptions`. It is possible to manually verify the `stderr` logger by modifying the tests slightly, and observing that the console indeed gets cluttered with logs from `stderr`. Pull Request resolved: https://github.com/facebook/rocksdb/pull/12213 Reviewed By: cbi42 Differential Revision: D52772306 Pulled By: ajkr fbshipit-source-id: 4026895f78f9cc250daf6bfa57427957e2d8b053
2024-01-18 01:51:36 +00:00
std::unique_ptr<char[]> buf(new char[buf_len]);
// If the logger was created without a prefix, the prefix is a nullptr
const char* prefix = log_prefix == nullptr ? "" : log_prefix;
// Write out the context and prefix string
int written =
snprintf(buf.get(), ctx_len + log_prefix_len, ctx_prefix_fmt,
t.tm_year + 1900, t.tm_mon + 1, t.tm_mday, t.tm_hour, t.tm_min,
t.tm_sec, static_cast<int>(now_tv.tv_usec),
static_cast<long long unsigned int>(thread_id), prefix);
vsnprintf(buf.get() + written, log_suffix_len, format, ap);
Add native logger support to RocksJava (#12213) Summary: ## Overview In this PR, we introduce support for setting the RocksDB native logger through Java. As mentioned in the discussion on the [Google Group discussion](https://groups.google.com/g/rocksdb/c/xYmbEs4sqRM/m/e73E4whJAQAJ), this work is primarily motivated by the JDK 17 [performance regression in JNI thread attach/detach calls](https://bugs.openjdk.org/browse/JDK-8314859): the only existing RocksJava logging configuration call, `setLogger`, invokes the provided logger over the JNI. ## Changes Specifically, these changes add support for the `devnull` and `stderr` native loggers. For the `stderr` logger, we add the ability to prefix every log with a `logPrefix`, so that it becomes possible know which database a particular log is coming from (if multiple databases are in use). The API looks like the following: ```java Options opts = new Options(); NativeLogger stderrNativeLogger = NativeLogger.newStderrLogger( InfoLogLevel.DEBUG_LEVEL, "[my prefix here]"); options.setLogger(stderrNativeLogger); try (final RocksDB db = RocksDB.open(options, ...)) {...} // Cleanup stderrNativeLogger.close() opts.close(); ``` Note that the API to set the logger is the same, via `Options::setLogger` (or `DBOptions::setLogger`). However, it will set the RocksDB logger to be native when the provided logger is an instance of `NativeLogger`. ## Testing Two tests have been added in `NativeLoggerTest.java`. The first test creates both the `devnull` and `stderr` loggers, and sets them on the associated `Options`. However, to avoid polluting the testing output with logs from `stderr`, only the `devnull` logger is actually used in the test. The second test does the same logic, but for `DBOptions`. It is possible to manually verify the `stderr` logger by modifying the tests slightly, and observing that the console indeed gets cluttered with logs from `stderr`. Pull Request resolved: https://github.com/facebook/rocksdb/pull/12213 Reviewed By: cbi42 Differential Revision: D52772306 Pulled By: ajkr fbshipit-source-id: 4026895f78f9cc250daf6bfa57427957e2d8b053
2024-01-18 01:51:36 +00:00
fprintf(stderr, "%s%c", buf.get(), '\n');
}
} // namespace ROCKSDB_NAMESPACE