Commit Graph

3055 Commits

Author SHA1 Message Date
Yueh-Hsuan Chiang 97c1940882 Fix compile warning in db_stress.cc on Mac
Summary:
Fix the following compile warning in db_stress.cc on Mac
tools/db_stress.cc:1688:52: error: format specifies type 'unsigned long' but the argument has type '::google::uint64' (aka 'unsigned long long') [-Werror,-Wformat]
    fprintf(stdout, "DB-write-buffer-size: %lu\n", FLAGS_db_write_buffer_size);
                                           ~~~     ^~~~~~~~~~~~~~~~~~~~~~~~~~
                                           %llu

Test Plan:
make
2014-12-04 11:19:12 -08:00
Yueh-Hsuan Chiang 5f719d7202 Replace exception by setting valid_ = false in DBIter::MergeValuesNewToOld()
Summary: Replace exception by setting valid_ = false in DBIter::MergeValuesNewToOld().

Test Plan:
Not sure if I am right at this, but it seems we currently don't have a good
way to test that code path as it requires dynamically set merge_operator = nullptr
at the time while Merge() is calling.

Reviewers: igor, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D29811
2014-12-04 11:11:11 -08:00
Mark Callaghan c0dee851c3 Improve formatting, add missing newlines
Summary:
Improve formatting

Task ID: #

Blame Rev:

Test Plan:
make

Revert Plan:

Database Impact:

Memcache Impact:

Other Notes:

EImportant:

- begin *PUBLIC* platform impact section -
Bugzilla: #
- end platform impact -

Reviewers: igor

Reviewed By: igor

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D29829
2014-12-04 10:34:06 -08:00
Igor Canadi 815f638cd0 Fix java build 2014-12-03 19:06:57 -08:00
Mark Callaghan 32a0a03844 Add Moved(GB) to Compaction IO stats
Summary:
Adds counter for bytes moved (files pushed down a level rather than compacted) to compaction
IO stats as Moved(GB). From the output removed these infrequently used columns: RW-Amp, Rn(cnt), Rnp1(cnt),
Wnp1(cnt), Wnew(cnt).
Example old output:
Level   Files   Size(MB) Score Read(GB)  Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) RW-Amp W-Amp Rd(MB/s) Wr(MB/s)  Rn(cnt) Rnp1(cnt) Wnp1(cnt) Wnew(cnt)  Comp(sec) Comp(cnt) Avg(sec) Stall(sec) Stall(cnt) Avg(ms) RecordIn RecordDrop
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  L0     0/0          0   0.0      0.0     0.0      0.0    2130.8   2130.8    0.0   0.0      0.0    109.1        0         0         0         0      20002     25068    0.798      28.75     182059    0.16       0          0
  L1   142/0        509   1.0   4618.5  2036.5   2582.0    4602.1   2020.2    4.5   2.3     88.5     88.1    24220    701246   1215528    514282      53466      4229   12.643       0.00          0    0.002032745988  300688729

Example new output:
Level   Files   Size(MB) Score Read(GB)  Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) Stall(sec) Stall(cnt) Avg(ms)     RecordIn   RecordDrop
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  L0     7/0         13   1.8      0.0     0.0      0.0       0.6      0.6       0.0   0.0      0.0     14.7        44       353    0.124       0.03        626    0.05            0            0
  L1     9/0         16   1.6      0.0     0.0      0.0       0.0      0.0       0.6   0.0      0.0      0.0         0         0    0.000       0.00          0    0.00            0            0

Task ID: #

Blame Rev:

Test Plan:
make check, run db_bench --fillseq --stats_per_interval --stats_interval and look at output

Revert Plan:

Database Impact:

Memcache Impact:

Other Notes:

EImportant:

- begin *PUBLIC* platform impact section -
Bugzilla: #
- end platform impact -

Reviewers: igor

Reviewed By: igor

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D29787
2014-12-03 18:28:39 -08:00
Jonah Cohen a14b7873ee Enforce write buffer memory limit across column families
Summary:
Introduces a new class for managing write buffer memory across column
families.  We supplement ColumnFamilyOptions::write_buffer_size with
ColumnFamilyOptions::write_buffer, a shared pointer to a WriteBuffer
instance that enforces memory limits before flushing out to disk.

Test Plan: Added SharedWriteBuffer unit test to db_test.cc

Reviewers: sdong, rven, ljin, igor

Reviewed By: igor

Subscribers: tnovak, yhchiang, dhruba, xjin, MarkCallaghan, yoshinorim

Differential Revision: https://reviews.facebook.net/D22581
2014-12-02 12:09:20 -08:00
fyrz 3e684aa685 Integrated changes from D29571 2014-12-02 19:56:45 +01:00
Igor Canadi 37d73d597e Fix linters
Summary:
Two fixes:
1. if cpplint is not present on the system, don't return a confusing error in the linter
2. Add include_alpha, which means our includes should be sorted lexicographically

Test Plan: Tried unsorting our includes, lint complained

Reviewers: rven, ljin, yhchiang, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D28845
2014-12-02 13:53:39 -05:00
fyrz a15169f2e9 Fixed a Lint problem 2014-12-02 09:58:20 +01:00
fyrz b7f9e644cc [RocksJava] Quality improvements
Summary:
- Addressed some FindBugs issues.
- Remove obsolete dbFolder cleanup
- Comparator tests for CF
 - Added AbstractComparatorTest.
 - Fixed a bug in the JNI Part about Java comparators
- Minor test improvements

Test Plan:
make rocksdbjava
make jtest
mvn -f rocksjni.pom package

Reviewers: adamretter, yhchiang, ankgup87

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D29571
2014-12-02 09:58:19 +01:00
fyrz e002a6122f [RocksJava] Comparator tests for CF
- Added AbstractComparatorTest.
- Fixed a bug in the JNI Part about Java comparators
2014-12-02 09:58:19 +01:00
fyrz 335e6ad5cd [RocksJava] Remove obsolete dbFolder cleanup 2014-12-02 09:58:18 +01:00
fyrz b036804ac1 RocksJava - FindBugs issues
Addressed some FindBugs issues.
2014-12-02 09:58:17 +01:00
Igor Canadi 9a632b4a92 Merge pull request #429 from fyrz/RocksJava-MacOSX-strip-fix
[RocksJava] MacOSX strip support
2014-12-01 13:13:02 -05:00
fyrz b426675061 [RocksJava] MacOSX strip support 2014-12-01 19:01:29 +01:00
Igor Canadi e463cb0bcf Merge pull request #424 from eile/master
Tweak Makefile for building on BG/Q
2014-12-01 10:20:12 -05:00
Stefan Eilemann 91d8981639 Tweak Makefile for building on BG/Q 2014-12-01 09:01:54 +01:00
Haneef Mubarak c6f31a2893 minor memory leak in C example 2014-11-29 21:42:42 -08:00
Igor Canadi 703ef66a86 Merge pull request #426 from fyrz/RocksJava-Restore-PrecisionFix
[RocksJava] Fixed MacOS build of RocksJava
2014-11-27 20:39:39 -05:00
Haneef Mubarak ac4ed1e305 fix examples/makefile for C example 2014-11-27 15:20:55 -08:00
Haneef Mubarak d7f5ccb0c2 add c example to makefile and fix "make clean" 2014-11-27 15:06:12 -08:00
Haneef Mubarak 9c34d5e361 fix type in C simple example 2014-11-27 13:53:04 -08:00
Haneef Mubarak 0a9a7e753c added C version of simple_example 2014-11-27 13:49:19 -08:00
Yueh-Hsuan Chiang bcf9086899 Block Universal and FIFO compactions in ROCKSDB_LITE
Summary: Block Universal and FIFO compactions in ROCKSDB_LITE

Test Plan:
make shared_lib -j32
make OPT=-DROCKSDB_LITE shared_lib

Reviewers: ljin, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D29589
2014-11-26 15:45:11 -08:00
fyrz 67cb7ca758 [RocksJava] Fixed MacOS build of RocksJava
There were still some precision loss problems
remainging in RocksJava. This pull request resolve
these.
2014-11-26 20:53:23 +01:00
Yueh-Hsuan Chiang b8136a7d27 Merge pull request #398 from fyrz/RocksJava-CreateCheckPoint
[RocksJava] Support for stored snapshots
2014-11-26 11:40:41 -08:00
Yueh-Hsuan Chiang 533592a27d Merge pull request #401 from fyrz/RocksJava-Sigsegv-MergeOperatorName
[RocksJava] Fixes in MergeOperatorByName
2014-11-26 11:40:13 -08:00
Yueh-Hsuan Chiang 73d72ed5c7 Block ReadOnlyDB in ROCKSDB_LITE
Summary:
db_imp_readonly.o is one of the big obj file.  If it's not a necessary
feature, we should probably block it in ROCKSDB_LITE.

    1322704 Nov 24 16:55 db/db_impl_readonly.o

Test Plan:
make shared_lib -j32
make ROCKSDB_LITE shared_lib -j32

Reviewers: ljin, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D29583
2014-11-26 11:37:59 -08:00
Igor Canadi e47f0fa9ed Merge pull request #425 from adamretter/macosx-clean-fix
Mac OS X jclean fix
2014-11-26 13:40:32 -05:00
Adam Retter ff0cb90d1c Do not delete Java Fatal Error Log, developers may still want these for
reference
2014-11-26 18:18:04 +00:00
Adam Retter 2a792cd300 There will also be a librocksdbjni-osx.jnilib.dSYM folder on MacOSX
builds to be deleted
2014-11-26 18:17:54 +00:00
Yueh-Hsuan Chiang beb74c14ca Fix travis-build error
Summary:
Fix travis-build error
2014-11-26 09:42:13 -08:00
Igor Canadi a486352e07 Merge pull request #423 from zerebubuth/c_iterate_upper_bound
C iterate upper bound
2014-11-26 10:14:16 -05:00
Matt Amos 26109d487a Store upper bound `Slice` with the same lifetime as the `ReadOptions` so that we can provide a pointer to it. 2014-11-26 11:29:13 +00:00
Yueh-Hsuan Chiang a97314219e Fix compile error in ROCKSDB_LITE 2014-11-25 23:39:52 -08:00
Yueh-Hsuan Chiang 9d5019327b Replace log2 by implementing Log2 in options_builder
Summary:
log2 function is only used in options_builder, and this function
is not available under certain platform such as android.
This patch implements Log2 by log(n) / log(2).

Test Plan:
make
2014-11-25 23:28:36 -08:00
Matt Amos 805bac6d25 Add test for upper bounds on iterators using C interface. 2014-11-25 23:07:40 +00:00
fyrz f193deea31 [RocksJava] Addressed comments in D28971 2014-11-25 23:33:42 +01:00
fyrz 94f70a86b9 [RocksJava] Incoroporated changes for D29013 2014-11-25 22:26:03 +01:00
fyrz a280af2a57 [RocksJava] Sigsegv fix for MergerOperatorByName 2014-11-25 22:26:02 +01:00
fyrz fcc2dfd9f9 [RocksJava] Support for stored snapshots
Summary:
RocksDB supports two ways of saving snapshots. In
memory and on disk. The later was added with this
pull request to RocksJava.

Test Plan:

Reviewers:

Subscribers:
2014-11-25 22:25:22 +01:00
Yueh-Hsuan Chiang 274ba62707 Block internal_stats in ROCKSDB_LITE
Summary: Block internal_stats in ROCKSDB_LITE.

Test Plan: make OPT=-DROCKSDB_LITE shared_lib

Reviewers: ljin, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D29541
2014-11-25 12:01:27 -08:00
Igor Canadi 4f2e8bab5f Merge pull request #421 from fyrz/RocksJava-PrecisionFix
[RocksJava] Fix precision problem in rocksjni
2014-11-25 14:32:44 -05:00
fyrz c4765dc10b [RocksJava] Fix precision problem in rocksjni 2014-11-25 20:29:52 +01:00
Igor Canadi 14788e1811 Merge pull request #420 from rdallman/add-wal
c api: add max wal total to opts
2014-11-25 09:34:15 -05:00
Reed Allman 88dd8d889b c api: add max wal total to opts 2014-11-24 22:00:29 -08:00
Yueh-Hsuan Chiang 7e608e2fe3 Block plain_table_index.cc in ROCKSDB_LITE
Summary: Block plain_table_index.cc in ROCKSDB_LITE

Test Plan:
make clean
make OPT=-DROCKSDB_LITE shared_lib -j32
make clean
make shared_lib -j32

Reviewers: ljin, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D29535
2014-11-24 20:47:27 -08:00
Yueh-Hsuan Chiang 13de000f07 Add rocksdb::ToString() to address cases where std::to_string is not available.
Summary:
In some environment such as android, the c++ library does not have
std::to_string.  This path adds rocksdb::ToString(), which wraps std::to_string
when std::to_string is not available, and implements std::to_string
in the other case.

Test Plan:
make dbg -j32
./db_test
make clean
make dbg OPT=-DOS_ANDROID -j32
./db_test

Reviewers: ljin, sdong, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D29181
2014-11-24 20:44:49 -08:00
Yueh-Hsuan Chiang 90ee85f8e1 Improve listener_test to avoid possible false alarm
Summary:
Improve listener_test to avoid possible false alarm

Test Plan:
./listener_test
2014-11-24 18:28:06 -08:00
Lei Jin 2946e37a08 remove unreliable test in db/cuckoo_table_db_test.cc
Summary:
This compaction trigger does not seem to test any thing specific to
cuckoo table. Remove it.

Test Plan: make all check

Reviewers: igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D29523
2014-11-24 15:18:09 -08:00