dolysis/rocksdb
3
0
Fork 0
mirror of https://github.com/facebook/rocksdb.git synced 2024-11-29 18:33:58 +00:00
Code Issues Packages Projects Releases Wiki Activity
rocksdb/util
History
Igor Canadi 52d8347a91 EventLogger 
Summary:
Here's my proposal for making our LOGs easier to read by machines.

The idea is to dump all events as JSON objects. JSON is easy to read by humans, but more importantly, it's easy to read by machines. That way, we can parse this, load into SQLite/mongo and then query or visualize.

I started with table_create and table_delete events, but if everybody agrees, I'll continue by adding more events (flush/compaction/etc etc)

Test Plan:
Ran db_bench. Observed:
2015/01/15-14:13:25.788019 1105ef000 EVENT_LOG_v1 {"time_micros": 1421360005788015, "event": "table_file_creation", "file_number": 12, "file_size": 1909699}
2015/01/15-14:13:25.956500 110740000 EVENT_LOG_v1 {"time_micros": 1421360005956498, "event": "table_file_deletion", "file_number": 12}

Reviewers: yhchiang, rven, dhruba, MarkCallaghan, lgalanis, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D31647
2015-03-13 10:15:54 -07:00
..
allocator.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
arena.cc Make arena use hugepage if possible 2014-11-21 14:11:40 -08:00
arena.h Removing unnecessary kInlineSize 2015-03-12 21:13:53 +03:00
arena_test.cc Make arena use hugepage if possible 2014-11-21 14:11:40 -08:00
auto_roll_logger.cc Add Header to logging to capture application level information 2015-02-06 10:37:45 -08:00
auto_roll_logger.h rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
auto_roll_logger_test.cc Fixed a bug in the test case 2015-02-23 11:59:39 -08:00
autovector.h Replace exception by assertion in autovector 2014-12-04 11:41:56 -08:00
autovector_test.cc Add rocksdb::ToString() to address cases where std::to_string is not available. 2014-11-24 20:44:49 -08:00
benchharness.cc Add rocksdb::ToString() to address cases where std::to_string is not available. 2014-11-24 20:44:49 -08:00
benchharness.h mark as unused some variables with cpp-derived names 2015-02-20 11:07:21 -08:00
benchharness_test.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
bloom.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
bloom_test.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
build_version.h build: do not relink every single binary just for a timestamp 2015-02-19 13:11:10 -08:00
cache.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
cache_bench.cc Fix -Wshadow for tools 2014-11-07 15:04:30 -08:00
cache_test.cc Fix Mac compile errors on util/cache_test.cc 2014-12-11 14:15:13 -08:00
coding.cc Removing BitStream* functions 2014-08-19 06:48:21 -07:00
coding.h Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
coding_test.cc Remove BitStream* tests 2014-08-19 09:52:54 -04:00
comparator.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
compression.h New BlockBasedTable version -- better compressed block format 2015-01-14 16:24:24 -08:00
crc32c.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
crc32c.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
crc32c_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
db_info_dumper.cc Fix iOS compile with -Wshorten-64-to-32 2014-11-13 14:39:30 -05:00
db_info_dumper.h Fix iOS compile with -Wshorten-64-to-32 2014-11-13 14:39:30 -05:00
dynamic_bloom.cc Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
dynamic_bloom.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
dynamic_bloom_test.cc rocksdb: Fix 'Division by zero' scan-build warning 2015-02-17 17:54:02 -08:00
env.cc Add Header to logging to capture application level information 2015-02-06 10:37:45 -08:00
env_hdfs.cc Add rocksdb::ToString() to address cases where std::to_string is not available. 2014-11-24 20:44:49 -08:00
env_posix.cc Revert chrono use 2015-03-03 11:29:31 -08:00
env_test.cc Switch logv with loglevel to virtual 2015-02-09 20:59:29 +01:00
event_logger.cc EventLogger 2015-03-13 10:15:54 -07:00
event_logger.h EventLogger 2015-03-13 10:15:54 -07:00
event_logger_test.cc EventLogger 2015-03-13 10:15:54 -07:00
file_util.cc Provide openable snapshots 2014-11-14 11:38:26 -08:00
file_util.h Provide openable snapshots 2014-11-14 11:38:26 -08:00
filelock_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
filter_policy.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
hash.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
hash.h Introduce GetThreadList API 2014-11-20 10:49:32 -08:00
hash_cuckoo_rep.cc Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
hash_cuckoo_rep.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
hash_linklist_rep.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
hash_linklist_rep.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
hash_skiplist_rep.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
hash_skiplist_rep.h Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
histogram.cc Fix iOS compile with -Wshorten-64-to-32 2014-11-13 14:39:30 -05:00
histogram.h Fix iOS compile with -Wshorten-64-to-32 2014-11-13 14:39:30 -05:00
histogram_test.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
instrumented_mutex.cc Perf Context to report DB mutex waiting time 2015-02-09 17:55:12 -08:00
instrumented_mutex.h Add a counter for collecting the wait time on db mutex. 2015-02-04 21:39:45 -08:00
iostats_context.cc Fix ios compile 2014-08-28 12:46:05 -04:00
iostats_context_imp.h Fix ios compile 2014-08-28 12:46:05 -04:00
ldb_cmd.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
ldb_cmd.h rocksdb: Fix uninitialized use error 2015-02-26 14:19:51 -08:00
ldb_cmd_execute_result.h Revert "Fix lint errors and coding style of ldb related codes." 2014-10-31 19:22:49 -07:00
ldb_tool.cc Added 'dump_live_files' command to ldb tool. 2014-12-12 17:50:36 -08:00
log_buffer.cc Enlarge log size cap when printing file summary 2014-09-23 16:56:34 -07:00
log_buffer.h RocksDB on FreeBSD support 2015-02-26 15:19:17 -08:00
log_write_bench.cc Fix more gflag namespace issues 2014-05-09 08:41:02 -07:00
logging.cc Allow GetThreadList() to report the start time of the current operation. 2015-03-10 14:51:28 -07:00
logging.h Fix a bug in stall time counter. Improve its output format. 2015-03-03 12:48:12 -08:00
manual_compaction_test.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
memenv.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
memenv_test.cc Compile MemEnv with standard RocksDB library 2015-01-29 16:33:11 -08:00
mock_env.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
mock_env.h rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
mock_env_test.cc Add fsync / corrupt simulation to env_mem 2014-10-31 15:16:31 -07:00
murmurhash.cc Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
murmurhash.h Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
mutable_cf_options.cc options.level_compaction_dynamic_level_bytes to allow RocksDB to pick size bases of levels dynamically. 2015-03-02 22:40:41 -08:00
mutable_cf_options.h options.level_compaction_dynamic_level_bytes to allow RocksDB to pick size bases of levels dynamically. 2015-03-02 22:40:41 -08:00
mutexlock.h Add separate Read/WriteUnlock methods in MutexRW. 2014-06-16 15:41:46 -07:00
options.cc options.level_compaction_dynamic_level_bytes to allow RocksDB to pick size bases of levels dynamically. 2015-03-02 22:40:41 -08:00
options_builder.cc Remove the compability check on log2 OS_ANDROID as it's already blocked by ROCKSDB_LITE 2014-12-04 13:56:14 -08:00
options_helper.cc Add rate_limiter to string options 2015-03-06 14:21:15 -08:00
options_helper.h Missing header in build on CentOS 2014-11-18 22:21:02 +01:00
options_test.cc Add rate_limiter to string options 2015-03-06 14:21:15 -08:00
perf_context.cc Instrument memtable seeks 2015-02-27 17:06:06 -08:00
perf_context_imp.h Refactor PerfStepTimer to stop on destruct 2014-09-02 12:04:22 -07:00
posix_logger.h rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
random.h Add appropriate LICENSE and Copyright message. 2013-10-16 17:48:41 -07:00
rate_limiter.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
rate_limiter.h expose RateLimiter definition 2014-07-25 15:17:06 -07:00
rate_limiter_test.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
scoped_arena_iterator.h Remove path with arena==nullptr from NewInternalIterator 2014-09-04 17:40:41 -07:00
signal_test.cc rocksdb: Fix scan-build 'Called C++ object pointer is null' and 'Dereference of null pointer' bugs 2015-02-13 15:10:47 -08:00
skiplistrep.cc Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
slice.cc rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
slice_transform_test.cc CappedFixTransform: return fixed length prefix, or full key if key is shorter than the fixed length 2015-01-30 16:04:30 -08:00
sst_dump_test.cc Disable pre-fetching of index and filter blocks for sst_dump_tool. 2015-02-25 16:34:26 -08:00
sst_dump_tool.cc Disable pre-fetching of index and filter blocks for sst_dump_tool. 2015-02-25 16:34:26 -08:00
sst_dump_tool_imp.h Disable pre-fetching of index and filter blocks for sst_dump_tool. 2015-02-25 16:34:26 -08:00
statistics.cc Fix assert in histogramData 2015-01-23 18:10:52 -08:00
statistics.h make statistics forward-able 2014-07-28 12:10:49 -07:00
status.cc Turn on -Wshorten-64-to-32 and fix all the errors 2014-11-11 16:47:22 -05:00
stl_wrappers.h Killing Transform Rep 2013-12-03 12:42:15 -08:00
stop_watch.h Change StopWatch interface 2014-07-28 12:22:37 -07:00
string_util.cc Clean up StringSplit 2014-11-21 11:05:28 -05:00
string_util.h Add rocksdb::ToString() to address cases where std::to_string is not available. 2014-11-24 20:44:49 -08:00
sync_point.cc options.level_compaction_dynamic_level_bytes to allow RocksDB to pick size bases of levels dynamically. 2015-03-02 22:40:41 -08:00
sync_point.h options.level_compaction_dynamic_level_bytes to allow RocksDB to pick size bases of levels dynamically. 2015-03-02 22:40:41 -08:00
testharness.cc introduce TestMemEnv and use it in db_test 2014-10-31 15:08:10 -07:00
testharness.h mark as unused some variables with cpp-derived names 2015-02-20 11:07:21 -08:00
testutil.cc Merger test 2014-09-08 22:24:40 -07:00
testutil.h rocksdb: Add missing override 2015-02-26 11:28:41 -08:00
thread_list_test.cc Add utility functions for interpreting ThreadStatus 2015-02-04 01:47:32 -08:00
thread_local.cc Use ustricter consistency in thread local operations 2015-01-27 13:56:03 -08:00
thread_local.h Improve the comment of util/thread_local.h 2014-10-21 17:28:31 -07:00
thread_local_test.cc Turn on -Wshadow 2014-10-31 11:59:54 -07:00
thread_operation.h Add a missing field for STATE_MUTEX_WAIT to global_state_table 2015-02-06 02:38:14 -08:00
thread_status_impl.cc Fixed a compile error in db_bench in mac. 2015-03-11 13:02:46 -07:00
thread_status_updater.cc Allow GetThreadList() to report the start time of the current operation. 2015-03-10 14:51:28 -07:00
thread_status_updater.h Allow GetThreadList() to report the start time of the current operation. 2015-03-10 14:51:28 -07:00
thread_status_updater_debug.cc Allow GetThreadList() to indicate a thread is doing Compaction. 2015-01-13 00:04:08 -08:00
thread_status_util.cc Allow GetThreadList() to report the start time of the current operation. 2015-03-10 14:51:28 -07:00
thread_status_util.h Add thread_status_util_debug.cc back 2015-02-17 11:29:36 -08:00
thread_status_util_debug.cc Add thread_status_util_debug.cc back 2015-02-17 11:29:36 -08:00
vectorrep.cc Enforce write buffer memory limit across column families 2014-12-02 12:09:20 -08:00
xfunc.cc Managed iterator 2015-02-18 11:49:31 -08:00
xfunc.h RocksDB on FreeBSD support 2015-02-26 15:19:17 -08:00
xxhash.cc Prevent xxhash symbols from polluting global namespace 2015-03-12 12:07:10 -07:00
xxhash.h Prevent xxhash symbols from polluting global namespace 2015-03-12 12:07:10 -07:00