Require C++17 (#9481)

Summary:
Drop support for some old compilers by requiring C++17 standard
(or higher). See https://github.com/facebook/rocksdb/issues/9388

First modification based on this is to remove some conditional compilation in slice.h (also
better for ODR)

Also in this PR:
* Fix some Makefile formatting that seems to affect ASSERT_STATUS_CHECKED config in
some cases
* Add c_test to NON_PARALLEL_TEST in Makefile
* Fix a clang-analyze reported "potential leak" in lru_cache_test
* Better "compatibility" definition of DEFINE_uint32 for old versions of gflags
* Fix a linking problem with shared libraries in Makefile (`./random_test: error while loading shared libraries: librocksdb.so.6.29: cannot open shared object file: No such file or directory`)
* Always set ROCKSDB_SUPPORT_THREAD_LOCAL and use thread_local (from C++11)
  * TODO in later PR: clean up that obsolete flag
* Fix a cosmetic typo in c.h (https://github.com/facebook/rocksdb/issues/9488)

Pull Request resolved: https://github.com/facebook/rocksdb/pull/9481

Test Plan:
CircleCI config substantially updated.

* Upgrade to latest Ubuntu images for each release
* Generally prefer Ubuntu 20, but keep a couple Ubuntu 16 builds with oldest supported
compilers, to ensure compatibility
* Remove .circleci/cat_ignore_eagain except for Ubuntu 16 builds, because this is to work
around a kernel bug that should not affect anything but Ubuntu 16.
* Remove designated gcc-9 build, because the default linux build now uses GCC 9 from
Ubuntu 20.
* Add some `apt-key add` to fix some apt "couldn't be verified" errors
* Generally drop SKIP_LINK=1; work-around no longer needed
* Generally `add-apt-repository` before `apt-get update` as manual testing indicated the
reverse might not work.

Travis:
* Use gcc-7 by default (remove specific gcc-7 and gcc-4.8 builds)
* TODO in later PR: fix s390x "Assembler messages: Error: invalid switch -march=z14" failure

AppVeyor:
* Completely dropped because we are dropping VS2015 support and CircleCI covers
VS >= 2017

Also local testing with old gflags (out of necessity when using ROCKSDB_NO_FBCODE=1).

Reviewed By: mrambacher

Differential Revision: D33946377

Pulled By: pdillinger

fbshipit-source-id: ae077c823905b45370a26c0103ada119459da6c1
This commit is contained in:
Peter Dillinger 2022-02-04 17:12:03 -08:00 committed by Facebook GitHub Bot
parent 42c8afd85a
commit fd3e0f43b3
17 changed files with 162 additions and 268 deletions

View File

@ -76,8 +76,9 @@ commands:
- run: - run:
name: Install Clang 10 name: Install Clang 10
command: | command: |
echo "deb http://apt.llvm.org/xenial/ llvm-toolchain-xenial-10 main" | sudo tee -a /etc/apt/sources.list wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | sudo apt-key add -
echo "deb-src http://apt.llvm.org/xenial/ llvm-toolchain-xenial-10 main" | sudo tee -a /etc/apt/sources.list echo "deb http://apt.llvm.org/focal/ llvm-toolchain-focal-10 main" | sudo tee -a /etc/apt/sources.list
echo "deb-src http://apt.llvm.org/focal/ llvm-toolchain-focal-10 main" | sudo tee -a /etc/apt/sources.list
echo "APT::Acquire::Retries \"10\";" | sudo tee -a /etc/apt/apt.conf.d/80-retries # llvm.org unreliable echo "APT::Acquire::Retries \"10\";" | sudo tee -a /etc/apt/apt.conf.d/80-retries # llvm.org unreliable
sudo apt-get update -y && sudo apt-get install -y clang-10 sudo apt-get update -y && sudo apt-get install -y clang-10
@ -86,6 +87,7 @@ commands:
- run: - run:
name: Install Clang 13 name: Install Clang 13
command: | command: |
wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | sudo apt-key add -
echo "deb http://apt.llvm.org/focal/ llvm-toolchain-focal-13 main" | sudo tee -a /etc/apt/sources.list echo "deb http://apt.llvm.org/focal/ llvm-toolchain-focal-13 main" | sudo tee -a /etc/apt/sources.list
echo "deb-src http://apt.llvm.org/focal/ llvm-toolchain-focal-13 main" | sudo tee -a /etc/apt/sources.list echo "deb-src http://apt.llvm.org/focal/ llvm-toolchain-focal-13 main" | sudo tee -a /etc/apt/sources.list
echo "APT::Acquire::Retries \"10\";" | sudo tee -a /etc/apt/apt.conf.d/80-retries # llvm.org unreliable echo "APT::Acquire::Retries \"10\";" | sudo tee -a /etc/apt/apt.conf.d/80-retries # llvm.org unreliable
@ -169,7 +171,7 @@ jobs:
- increase-max-open-files-on-macos - increase-max-open-files-on-macos
- install-gflags-on-macos - install-gflags-on-macos
- pre-steps-macos - pre-steps-macos
- run: ulimit -S -n 1048576 && OPT=-DCIRCLECI make V=1 J=32 -j32 check 2>&1 | .circleci/cat_ignore_eagain - run: ulimit -S -n 1048576 && OPT=-DCIRCLECI make V=1 J=32 -j32 check
- post-steps - post-steps
build-macos-cmake: build-macos-cmake:
@ -181,153 +183,153 @@ jobs:
- install-cmake-on-macos - install-cmake-on-macos
- install-gflags-on-macos - install-gflags-on-macos
- pre-steps-macos - pre-steps-macos
- run: ulimit -S -n 1048576 && (mkdir build && cd build && cmake -DWITH_GFLAGS=1 .. && make V=1 -j32 && ctest -j10) 2>&1 | .circleci/cat_ignore_eagain - run: ulimit -S -n 1048576 && (mkdir build && cd build && cmake -DWITH_GFLAGS=1 .. && make V=1 -j32 && ctest -j10)
- post-steps - post-steps
build-linux: build-linux:
machine: machine:
image: ubuntu-1604:202104-01 image: ubuntu-2004:202111-02
resource_class: 2xlarge resource_class: 2xlarge
steps: steps:
- pre-steps - pre-steps
- install-gflags - install-gflags
- run: make V=1 J=32 -j32 check 2>&1 | .circleci/cat_ignore_eagain - run: make V=1 J=32 -j32 check
- post-steps - post-steps
build-linux-encrypted-env: build-linux-encrypted-env:
machine: machine:
image: ubuntu-1604:202104-01 image: ubuntu-2004:202111-02
resource_class: 2xlarge resource_class: 2xlarge
steps: steps:
- pre-steps - pre-steps
- install-gflags - install-gflags
- run: ENCRYPTED_ENV=1 make V=1 J=32 -j32 check 2>&1 | .circleci/cat_ignore_eagain - run: ENCRYPTED_ENV=1 make V=1 J=32 -j32 check
- post-steps - post-steps
build-linux-shared_lib-alt_namespace-status_checked: build-linux-shared_lib-alt_namespace-status_checked:
machine: machine:
image: ubuntu-1604:202104-01 image: ubuntu-2004:202111-02
resource_class: 2xlarge resource_class: 2xlarge
steps: steps:
- pre-steps - pre-steps
- install-gflags - install-gflags
- run: ASSERT_STATUS_CHECKED=1 TEST_UINT128_COMPAT=1 ROCKSDB_MODIFY_NPHASH=1 LIB_MODE=shared OPT="-DROCKSDB_NAMESPACE=alternative_rocksdb_ns" make V=1 -j32 check 2>&1 | .circleci/cat_ignore_eagain - run: ASSERT_STATUS_CHECKED=1 TEST_UINT128_COMPAT=1 ROCKSDB_MODIFY_NPHASH=1 LIB_MODE=shared OPT="-DROCKSDB_NAMESPACE=alternative_rocksdb_ns" make V=1 -j32 check
- post-steps - post-steps
build-linux-release: build-linux-release:
machine: machine:
image: ubuntu-1604:202104-01 image: ubuntu-2004:202111-02
resource_class: large resource_class: large
steps: steps:
- checkout # check out the code in the project directory - checkout # check out the code in the project directory
- run: make V=1 -j8 release 2>&1 | .circleci/cat_ignore_eagain - run: make V=1 -j8 release
- run: if ./db_stress --version; then false; else true; fi # ensure without gflags - run: if ./db_stress --version; then false; else true; fi # ensure without gflags
- install-gflags - install-gflags
- run: make V=1 -j8 release 2>&1 | .circleci/cat_ignore_eagain - run: make V=1 -j8 release
- run: ./db_stress --version # ensure with gflags - run: ./db_stress --version # ensure with gflags
- post-steps - post-steps
build-linux-release-rtti: build-linux-release-rtti:
machine: machine:
image: ubuntu-1604:201903-01 image: ubuntu-2004:202111-02
resource_class: large resource_class: large
steps: steps:
- checkout # check out the code in the project directory - checkout # check out the code in the project directory
- run: make clean - run: make clean
- run: USE_RTTI=1 DEBUG_LEVEL=0 make V=1 -j8 static_lib tools db_bench 2>&1 | .circleci/cat_ignore_eagain - run: USE_RTTI=1 DEBUG_LEVEL=0 make V=1 -j8 static_lib tools db_bench
- run: if ./db_stress --version; then false; else true; fi # ensure without gflags - run: if ./db_stress --version; then false; else true; fi # ensure without gflags
- run: sudo apt-get update -y && sudo apt-get install -y libgflags-dev - run: sudo apt-get update -y && sudo apt-get install -y libgflags-dev
- run: make clean - run: make clean
- run: USE_RTTI=1 DEBUG_LEVEL=0 make V=1 -j8 static_lib tools db_bench 2>&1 | .circleci/cat_ignore_eagain - run: USE_RTTI=1 DEBUG_LEVEL=0 make V=1 -j8 static_lib tools db_bench
- run: ./db_stress --version # ensure with gflags - run: ./db_stress --version # ensure with gflags
build-linux-lite: build-linux-lite:
machine: machine:
image: ubuntu-1604:202104-01 image: ubuntu-2004:202111-02
resource_class: 2xlarge resource_class: 2xlarge
steps: steps:
- pre-steps - pre-steps
- install-gflags - install-gflags
- run: LITE=1 make V=1 J=32 -j32 check 2>&1 | .circleci/cat_ignore_eagain - run: LITE=1 make V=1 J=32 -j32 check
- post-steps - post-steps
build-linux-lite-release: build-linux-lite-release:
machine: machine:
image: ubuntu-1604:202104-01 image: ubuntu-2004:202111-02
resource_class: large resource_class: large
steps: steps:
- checkout # check out the code in the project directory - checkout # check out the code in the project directory
- run: LITE=1 make V=1 -j8 release 2>&1 | .circleci/cat_ignore_eagain - run: LITE=1 make V=1 -j8 release
- run: if ./db_stress --version; then false; else true; fi # ensure without gflags - run: if ./db_stress --version; then false; else true; fi # ensure without gflags
- install-gflags - install-gflags
- run: LITE=1 make V=1 -j8 release 2>&1 | .circleci/cat_ignore_eagain - run: LITE=1 make V=1 -j8 release
- run: ./db_stress --version # ensure with gflags - run: ./db_stress --version # ensure with gflags
- post-steps - post-steps
build-linux-clang-no_test_run: build-linux-clang-no_test_run:
machine: machine:
image: ubuntu-1604:202104-01 image: ubuntu-2004:202111-02
resource_class: xlarge resource_class: xlarge
steps: steps:
- checkout # check out the code in the project directory - checkout # check out the code in the project directory
- run: sudo apt-get update -y && sudo apt-get install -y clang libgflags-dev libtbb-dev - run: sudo apt-get update -y && sudo apt-get install -y clang libgflags-dev libtbb-dev
- run: CC=clang CXX=clang++ USE_CLANG=1 PORTABLE=1 make V=1 -j16 all 2>&1 | .circleci/cat_ignore_eagain - run: CC=clang CXX=clang++ USE_CLANG=1 PORTABLE=1 make V=1 -j16 all
- post-steps - post-steps
build-linux-clang10-asan: build-linux-clang10-asan:
machine: machine:
image: ubuntu-1604:202104-01 image: ubuntu-2004:202111-02
resource_class: 2xlarge resource_class: 2xlarge
steps: steps:
- pre-steps - pre-steps
- install-gflags - install-gflags
- install-clang-10 - install-clang-10
- run: COMPILE_WITH_ASAN=1 CC=clang-10 CXX=clang++-10 ROCKSDB_DISABLE_ALIGNED_NEW=1 USE_CLANG=1 make V=1 -j32 check 2>&1 | .circleci/cat_ignore_eagain # aligned new doesn't work for reason we haven't figured out - run: COMPILE_WITH_ASAN=1 CC=clang-10 CXX=clang++-10 ROCKSDB_DISABLE_ALIGNED_NEW=1 USE_CLANG=1 make V=1 -j32 check # aligned new doesn't work for reason we haven't figured out
- post-steps - post-steps
build-linux-clang10-mini-tsan: build-linux-clang10-mini-tsan:
machine: machine:
image: ubuntu-1604:202104-01 image: ubuntu-2004:202111-02
resource_class: 2xlarge resource_class: 2xlarge
steps: steps:
- pre-steps - pre-steps
- install-gflags - install-gflags
- install-clang-10 - install-clang-10
- run: COMPILE_WITH_TSAN=1 CC=clang-10 CXX=clang++-10 ROCKSDB_DISABLE_ALIGNED_NEW=1 USE_CLANG=1 make V=1 -j32 check 2>&1 | .circleci/cat_ignore_eagain # aligned new doesn't work for reason we haven't figured out. - run: COMPILE_WITH_TSAN=1 CC=clang-10 CXX=clang++-10 ROCKSDB_DISABLE_ALIGNED_NEW=1 USE_CLANG=1 make V=1 -j32 check # aligned new doesn't work for reason we haven't figured out.
- post-steps - post-steps
build-linux-clang10-ubsan: build-linux-clang10-ubsan:
machine: machine:
image: ubuntu-1604:202104-01 image: ubuntu-2004:202111-02
resource_class: 2xlarge resource_class: 2xlarge
steps: steps:
- pre-steps - pre-steps
- install-gflags - install-gflags
- install-clang-10 - install-clang-10
- run: COMPILE_WITH_UBSAN=1 OPT="-fsanitize-blacklist=.circleci/ubsan_suppression_list.txt" CC=clang-10 CXX=clang++-10 ROCKSDB_DISABLE_ALIGNED_NEW=1 USE_CLANG=1 make V=1 -j32 ubsan_check 2>&1 | .circleci/cat_ignore_eagain # aligned new doesn't work for reason we haven't figured out - run: COMPILE_WITH_UBSAN=1 OPT="-fsanitize-blacklist=.circleci/ubsan_suppression_list.txt" CC=clang-10 CXX=clang++-10 ROCKSDB_DISABLE_ALIGNED_NEW=1 USE_CLANG=1 make V=1 -j32 ubsan_check # aligned new doesn't work for reason we haven't figured out
- post-steps - post-steps
build-linux-clang10-clang-analyze: build-linux-clang10-clang-analyze:
machine: machine:
image: ubuntu-1604:202104-01 image: ubuntu-2004:202111-02
resource_class: 2xlarge resource_class: 2xlarge
steps: steps:
- pre-steps - pre-steps
- install-gflags - install-gflags
- install-clang-10 - install-clang-10
- run: sudo apt-get update -y && sudo apt-get install -y clang-tools-10 - run: sudo apt-get update -y && sudo apt-get install -y clang-tools-10
- run: CC=clang-10 CXX=clang++-10 ROCKSDB_DISABLE_ALIGNED_NEW=1 CLANG_ANALYZER="/usr/bin/clang++-10" CLANG_SCAN_BUILD=scan-build-10 USE_CLANG=1 make V=1 -j32 analyze 2>&1 | .circleci/cat_ignore_eagain # aligned new doesn't work for reason we haven't figured out. For unknown, reason passing "clang++-10" as CLANG_ANALYZER doesn't work, and we need a full path. - run: CC=clang-10 CXX=clang++-10 ROCKSDB_DISABLE_ALIGNED_NEW=1 CLANG_ANALYZER="/usr/bin/clang++-10" CLANG_SCAN_BUILD=scan-build-10 USE_CLANG=1 make V=1 -j32 analyze # aligned new doesn't work for reason we haven't figured out. For unknown, reason passing "clang++-10" as CLANG_ANALYZER doesn't work, and we need a full path.
- post-steps - post-steps
build-linux-cmake: build-linux-cmake:
machine: machine:
image: ubuntu-1604:202104-01 image: ubuntu-2004:202111-02
resource_class: 2xlarge resource_class: 2xlarge
steps: steps:
- pre-steps - pre-steps
- install-gflags - install-gflags
- upgrade-cmake - upgrade-cmake
- run: (mkdir build && cd build && cmake -DWITH_GFLAGS=1 .. && make V=1 -j20 && ctest -j20) 2>&1 | .circleci/cat_ignore_eagain - run: (mkdir build && cd build && cmake -DWITH_GFLAGS=1 .. && make V=1 -j20 && ctest -j20)
- post-steps - post-steps
build-linux-cmake-ubuntu-20: build-linux-cmake-ubuntu-20:
@ -338,7 +340,7 @@ jobs:
- pre-steps - pre-steps
- install-gflags - install-gflags
- install-benchmark - install-benchmark
- run: (mkdir build && cd build && cmake -DWITH_GFLAGS=1 -DWITH_BENCHMARK=1 .. && make V=1 -j20 && ctest -j20) 2>&1 | .circleci/cat_ignore_eagain - run: (mkdir build && cd build && cmake -DWITH_GFLAGS=1 -DWITH_BENCHMARK=1 .. && make V=1 -j20 && ctest -j20)
- post-steps - post-steps
build-linux-unity-and-headers: build-linux-unity-and-headers:
@ -348,79 +350,69 @@ jobs:
steps: steps:
- checkout # check out the code in the project directory - checkout # check out the code in the project directory
- run: apt-get update -y && apt-get install -y libgflags-dev - run: apt-get update -y && apt-get install -y libgflags-dev
- run: TEST_TMPDIR=/dev/shm && make V=1 -j8 unity_test 2>&1 | .circleci/cat_ignore_eagain - run: TEST_TMPDIR=/dev/shm && make V=1 -j8 unity_test
- run: make V=1 -j8 -k check-headers 2>&1 | .circleci/cat_ignore_eagain # could be moved to a different build - run: make V=1 -j8 -k check-headers # could be moved to a different build
- post-steps - post-steps
build-linux-gcc-4_8-no_test_run: build-linux-gcc-7:
machine:
image: ubuntu-1604:202104-01
resource_class: 2xlarge
steps:
- pre-steps
- run: sudo add-apt-repository -y ppa:ubuntu-toolchain-r/test && sudo apt-get update -y && sudo apt-get install gcc-7 g++-7 libgflags-dev
- run: CC=gcc-7 CXX=g++-7 V=1 make -j32 check 2>&1 | .circleci/cat_ignore_eagain
- post-steps
build-linux-gcc-8-no_test_run:
machine: machine:
image: ubuntu-1604:202104-01 image: ubuntu-1604:202104-01
resource_class: large resource_class: large
steps: steps:
- pre-steps - pre-steps
- run: sudo apt-get update -y && sudo apt-get install gcc-4.8 g++-4.8 libgflags-dev - run: sudo add-apt-repository -y ppa:ubuntu-toolchain-r/test && sudo apt-get update -y && sudo apt-get install gcc-8 g++-8 libgflags-dev
- run: CC=gcc-4.8 CXX=g++-4.8 V=1 SKIP_LINK=1 make -j8 all 2>&1 | .circleci/cat_ignore_eagain # Linking broken because libgflags compiled with newer ABI - run: CC=gcc-8 CXX=g++-8 V=1 make -j8 all 2>&1 | .circleci/cat_ignore_eagain
- post-steps
build-linux-gcc-8-no_test_run:
machine:
image: ubuntu-2004:202010-01
resource_class: large
steps:
- pre-steps
- run: sudo apt-get update -y && sudo apt-get install gcc-8 g++-8 libgflags-dev
- run: CC=gcc-8 CXX=g++-8 V=1 SKIP_LINK=1 make -j8 all 2>&1 | .circleci/cat_ignore_eagain # Linking broken because libgflags compiled with newer ABI
- post-steps
build-linux-gcc-9-no_test_run:
machine:
image: ubuntu-2004:202010-01
resource_class: large
steps:
- pre-steps
- run: sudo apt-get update -y && sudo apt-get install gcc-9 g++-9 libgflags-dev
- run: CC=gcc-9 CXX=g++-9 V=1 SKIP_LINK=1 make -j8 all 2>&1 | .circleci/cat_ignore_eagain # Linking broken because libgflags compiled with newer ABI
- post-steps - post-steps
build-linux-gcc-10-cxx20-no_test_run: build-linux-gcc-10-cxx20-no_test_run:
machine: machine:
image: ubuntu-2004:202010-01 image: ubuntu-2004:202111-02
resource_class: xlarge resource_class: xlarge
steps: steps:
- pre-steps - pre-steps
- run: sudo apt-get update -y && sudo apt-get install gcc-10 g++-10 libgflags-dev - run: sudo apt-get update -y && sudo apt-get install gcc-10 g++-10 libgflags-dev
- run: CC=gcc-10 CXX=g++-10 V=1 SKIP_LINK=1 ROCKSDB_CXX_STANDARD=c++20 make -j16 all 2>&1 | .circleci/cat_ignore_eagain # Linking broken because libgflags compiled with newer ABI - run: CC=gcc-10 CXX=g++-10 V=1 SKIP_LINK=1 ROCKSDB_CXX_STANDARD=c++20 make -j16 all # Linking broken because libgflags compiled with newer ABI
- post-steps - post-steps
build-linux-gcc-11-no_test_run: build-linux-gcc-11-no_test_run:
machine: machine:
image: ubuntu-2004:202010-01 image: ubuntu-2004:202111-02
resource_class: xlarge resource_class: xlarge
steps: steps:
- pre-steps - pre-steps
- run: sudo apt-get update -y && sudo add-apt-repository -y ppa:ubuntu-toolchain-r/test && sudo apt-get install gcc-11 g++-11 libgflags-dev - run: sudo add-apt-repository -y ppa:ubuntu-toolchain-r/test && sudo apt-get update -y && sudo apt-get install gcc-11 g++-11 libgflags-dev
- run: CC=gcc-11 CXX=g++-11 V=1 SKIP_LINK=1 make -j16 all 2>&1 | .circleci/cat_ignore_eagain # Linking broken because libgflags compiled with newer ABI - run: CC=gcc-11 CXX=g++-11 V=1 SKIP_LINK=1 make -j16 all # Linking broken because libgflags compiled with newer ABI
- post-steps - post-steps
build-linux-clang-13-no_test_run: build-linux-clang-13-no_test_run:
machine: machine:
image: ubuntu-2004:202010-01 image: ubuntu-2004:202111-02
resource_class: xlarge resource_class: xlarge
steps: steps:
- pre-steps - pre-steps
- install-clang-13 - install-clang-13
- run: CC=clang-13 CXX=clang++-13 USE_CLANG=1 make -j16 all 2>&1 | .circleci/cat_ignore_eagain - run: CC=clang-13 CXX=clang++-13 USE_CLANG=1 make -j16 all
- post-steps - post-steps
# This job is only to make sure the microbench tests are able to run, the benchmark result is not meaningful as the CI host is changing. # This job is only to make sure the microbench tests are able to run, the benchmark result is not meaningful as the CI host is changing.
build-linux-microbench: build-linux-microbench:
machine: machine:
image: ubuntu-2004:202010-01 image: ubuntu-2004:202111-02
resource_class: xlarge resource_class: xlarge
steps: steps:
- pre-steps - pre-steps
- install-benchmark - install-benchmark
- run: DEBUG_LEVEL=0 make microbench 2>&1 | .circleci/cat_ignore_eagain - run: DEBUG_LEVEL=0 make microbench
- post-steps - post-steps
build-windows: build-windows:
@ -493,7 +485,7 @@ jobs:
build-linux-java: build-linux-java:
machine: machine:
image: ubuntu-1604:202104-01 image: ubuntu-2004:202111-02
resource_class: large resource_class: large
environment: environment:
JAVA_HOME: /usr/lib/jvm/java-1.8.0-openjdk-amd64 JAVA_HOME: /usr/lib/jvm/java-1.8.0-openjdk-amd64
@ -509,15 +501,15 @@ jobs:
which javac && javac -version which javac && javac -version
- run: - run:
name: "Build RocksDBJava Shared Library" name: "Build RocksDBJava Shared Library"
command: make V=1 J=8 -j8 rocksdbjava 2>&1 | .circleci/cat_ignore_eagain command: make V=1 J=8 -j8 rocksdbjava
- run: - run:
name: "Test RocksDBJava" name: "Test RocksDBJava"
command: make V=1 J=8 -j8 jtest 2>&1 | .circleci/cat_ignore_eagain command: make V=1 J=8 -j8 jtest
- post-steps - post-steps
build-linux-java-static: build-linux-java-static:
machine: machine:
image: ubuntu-1604:202104-01 image: ubuntu-2004:202111-02
resource_class: large resource_class: large
environment: environment:
JAVA_HOME: /usr/lib/jvm/java-1.8.0-openjdk-amd64 JAVA_HOME: /usr/lib/jvm/java-1.8.0-openjdk-amd64
@ -533,7 +525,7 @@ jobs:
which javac && javac -version which javac && javac -version
- run: - run:
name: "Build RocksDBJava Static Library" name: "Build RocksDBJava Static Library"
command: make V=1 J=8 -j8 rocksdbjavastatic 2>&1 | .circleci/cat_ignore_eagain command: make V=1 J=8 -j8 rocksdbjavastatic
- post-steps - post-steps
build-macos-java: build-macos-java:
@ -557,10 +549,10 @@ jobs:
which javac && javac -version which javac && javac -version
- run: - run:
name: "Build RocksDBJava Shared Library" name: "Build RocksDBJava Shared Library"
command: make V=1 J=8 -j8 rocksdbjava 2>&1 | .circleci/cat_ignore_eagain command: make V=1 J=8 -j8 rocksdbjava
- run: - run:
name: "Test RocksDBJava" name: "Test RocksDBJava"
command: make V=1 J=8 -j8 jtest 2>&1 | .circleci/cat_ignore_eagain command: make V=1 J=8 -j8 jtest
- post-steps - post-steps
build-macos-java-static: build-macos-java-static:
@ -584,7 +576,7 @@ jobs:
which javac && javac -version which javac && javac -version
- run: - run:
name: "Build RocksDBJava x86 and ARM Static Libraries" name: "Build RocksDBJava x86 and ARM Static Libraries"
command: make V=1 J=8 -j8 rocksdbjavastaticosx 2>&1 | .circleci/cat_ignore_eagain command: make V=1 J=8 -j8 rocksdbjavastaticosx
- post-steps - post-steps
build-macos-java-static-universal: build-macos-java-static-universal:
@ -608,12 +600,12 @@ jobs:
which javac && javac -version which javac && javac -version
- run: - run:
name: "Build RocksDBJava Universal Binary Static Library" name: "Build RocksDBJava Universal Binary Static Library"
command: make V=1 J=8 -j8 rocksdbjavastaticosx_ub 2>&1 | .circleci/cat_ignore_eagain command: make V=1 J=8 -j8 rocksdbjavastaticosx_ub
- post-steps - post-steps
build-examples: build-examples:
machine: machine:
image: ubuntu-1604:202104-01 image: ubuntu-2004:202111-02
resource_class: large resource_class: large
steps: steps:
- pre-steps - pre-steps
@ -621,12 +613,12 @@ jobs:
- run: - run:
name: "Build examples" name: "Build examples"
command: | command: |
OPT=-DTRAVIS V=1 make -j4 static_lib && cd examples && make -j4 | ../.circleci/cat_ignore_eagain OPT=-DTRAVIS V=1 make -j4 static_lib && cd examples && make -j4
- post-steps - post-steps
build-cmake-mingw: build-cmake-mingw:
machine: machine:
image: ubuntu-1604:202104-01 image: ubuntu-2004:202111-02
resource_class: 2xlarge resource_class: 2xlarge
steps: steps:
- pre-steps - pre-steps
@ -649,7 +641,7 @@ jobs:
build-linux-non-shm: build-linux-non-shm:
machine: machine:
image: ubuntu-1604:202104-01 image: ubuntu-2004:202111-02
resource_class: 2xlarge resource_class: 2xlarge
parameters: parameters:
start_test: start_test:
@ -679,27 +671,27 @@ jobs:
build-linux-arm-test-full: build-linux-arm-test-full:
machine: machine:
image: ubuntu-2004:202101-01 image: ubuntu-2004:202111-02
resource_class: arm.large resource_class: arm.large
steps: steps:
- pre-steps - pre-steps
- install-gflags - install-gflags
- run: make V=1 J=4 -j4 check 2>&1 | .circleci/cat_ignore_eagain - run: make V=1 J=4 -j4 check
- post-steps - post-steps
build-linux-arm: build-linux-arm:
machine: machine:
image: ubuntu-2004:202101-01 image: ubuntu-2004:202111-02
resource_class: arm.large resource_class: arm.large
steps: steps:
- pre-steps - pre-steps
- install-gflags - install-gflags
- run: ROCKSDBTESTS_PLATFORM_DEPENDENT=only make V=1 J=4 -j4 all_but_some_tests check_some 2>&1 | .circleci/cat_ignore_eagain - run: ROCKSDBTESTS_PLATFORM_DEPENDENT=only make V=1 J=4 -j4 all_but_some_tests check_some
- post-steps - post-steps
build-linux-arm-cmake-no_test_run: build-linux-arm-cmake-no_test_run:
machine: machine:
image: ubuntu-2004:202101-01 image: ubuntu-2004:202111-02
resource_class: arm.large resource_class: arm.large
environment: environment:
JAVA_HOME: /usr/lib/jvm/java-8-openjdk-arm64 JAVA_HOME: /usr/lib/jvm/java-8-openjdk-arm64
@ -732,7 +724,7 @@ jobs:
build-format-compatible: build-format-compatible:
machine: machine:
image: ubuntu-1604:202104-01 image: ubuntu-2004:202111-02
resource_class: 2xlarge resource_class: 2xlarge
steps: steps:
- pre-steps - pre-steps
@ -849,9 +841,8 @@ workflows:
jobs: jobs:
- build-linux-clang-no_test_run - build-linux-clang-no_test_run
- build-linux-clang-13-no_test_run - build-linux-clang-13-no_test_run
- build-linux-gcc-4_8-no_test_run - build-linux-gcc-7
- build-linux-gcc-8-no_test_run - build-linux-gcc-8-no_test_run
- build-linux-gcc-9-no_test_run
- build-linux-gcc-10-cxx20-no_test_run - build-linux-gcc-10-cxx20-no_test_run
- build-linux-gcc-11-no_test_run - build-linux-gcc-11-no_test_run
- build-linux-arm-cmake-no_test_run - build-linux-arm-cmake-no_test_run

View File

@ -43,7 +43,6 @@ env:
- JOB_NAME=cmake-gcc9 # 3-5 minutes - JOB_NAME=cmake-gcc9 # 3-5 minutes
- JOB_NAME=cmake-gcc9-c++20 # 3-5 minutes - JOB_NAME=cmake-gcc9-c++20 # 3-5 minutes
- JOB_NAME=cmake-mingw # 3 minutes - JOB_NAME=cmake-mingw # 3 minutes
- JOB_NAME=make-gcc4.8
- JOB_NAME=status_checked - JOB_NAME=status_checked
matrix: matrix:
@ -51,21 +50,12 @@ matrix:
- os : linux - os : linux
arch: arm64 arch: arm64
env: JOB_NAME=cmake-mingw env: JOB_NAME=cmake-mingw
- os : linux
arch: arm64
env: JOB_NAME=make-gcc4.8
- os: linux - os: linux
arch: ppc64le arch: ppc64le
env: JOB_NAME=cmake-mingw env: JOB_NAME=cmake-mingw
- os: linux
arch: ppc64le
env: JOB_NAME=make-gcc4.8
- os: linux - os: linux
arch: s390x arch: s390x
env: JOB_NAME=cmake-mingw env: JOB_NAME=cmake-mingw
- os: linux
arch: s390x
env: JOB_NAME=make-gcc4.8
- os: linux - os: linux
compiler: clang compiler: clang
- if: type = pull_request AND commit_message !~ /FULL_CI/ - if: type = pull_request AND commit_message !~ /FULL_CI/
@ -210,6 +200,7 @@ matrix:
env: JOB_NAME=status_checked env: JOB_NAME=status_checked
install: install:
- CC=gcc-7 && CXX=g++-7
- if [ "${JOB_NAME}" == cmake-gcc8 ]; then - if [ "${JOB_NAME}" == cmake-gcc8 ]; then
sudo apt-get install -y g++-8 || exit $?; sudo apt-get install -y g++-8 || exit $?;
CC=gcc-8 && CXX=g++-8; CC=gcc-8 && CXX=g++-8;
@ -221,9 +212,8 @@ install:
- if [ "${JOB_NAME}" == cmake-mingw ]; then - if [ "${JOB_NAME}" == cmake-mingw ]; then
sudo apt-get install -y mingw-w64 || exit $?; sudo apt-get install -y mingw-w64 || exit $?;
fi fi
- if [ "${JOB_NAME}" == make-gcc4.8 ]; then - if [ "${CXX}" == "g++-7" ]; then
sudo apt-get install -y g++-4.8 || exit $?; sudo apt-get install -y g++-7 || exit $?;
CC=gcc-4.8 && CXX=g++-4.8;
fi fi
- | - |
if [[ "${JOB_NAME}" == cmake* ]]; then if [[ "${JOB_NAME}" == cmake* ]]; then
@ -293,9 +283,6 @@ script:
mkdir build && cd build && cmake -DCMAKE_BUILD_TYPE=Release -DWITH_TESTS=0 -DWITH_GFLAGS=0 -DWITH_BENCHMARK_TOOLS=0 -DWITH_TOOLS=0 -DWITH_CORE_TOOLS=1 .. && make -j4 && cd .. && rm -rf build && mkdir build && cd build && cmake -DJNI=1 .. -DCMAKE_BUILD_TYPE=Release $OPT && make -j4 rocksdb rocksdbjni mkdir build && cd build && cmake -DCMAKE_BUILD_TYPE=Release -DWITH_TESTS=0 -DWITH_GFLAGS=0 -DWITH_BENCHMARK_TOOLS=0 -DWITH_TOOLS=0 -DWITH_CORE_TOOLS=1 .. && make -j4 && cd .. && rm -rf build && mkdir build && cd build && cmake -DJNI=1 .. -DCMAKE_BUILD_TYPE=Release $OPT && make -j4 rocksdb rocksdbjni
;; ;;
make-gcc4.8)
OPT=-DTRAVIS LIB_MODE=shared V=1 SKIP_LINK=1 make -j4 all && [ "Linking broken because libgflags compiled with newer ABI" ]
;;
status_checked) status_checked)
OPT=-DTRAVIS LIB_MODE=shared V=1 ASSERT_STATUS_CHECKED=1 make -j4 check_some OPT=-DTRAVIS LIB_MODE=shared V=1 ASSERT_STATUS_CHECKED=1 make -j4 check_some
;; ;;

View File

@ -14,7 +14,7 @@
# cd build # cd build
# 3. Run cmake to generate project files for Windows, add more options to enable required third-party libraries. # 3. Run cmake to generate project files for Windows, add more options to enable required third-party libraries.
# See thirdparty.inc for more information. # See thirdparty.inc for more information.
# sample command: cmake -G "Visual Studio 15 Win64" -DCMAKE_BUILD_TYPE=Release -DWITH_GFLAGS=1 -DWITH_SNAPPY=1 -DWITH_JEMALLOC=1 -DWITH_JNI=1 .. # sample command: cmake -G "Visual Studio 16 2019" -DCMAKE_BUILD_TYPE=Release -DWITH_GFLAGS=1 -DWITH_SNAPPY=1 -DWITH_JEMALLOC=1 -DWITH_JNI=1 ..
# 4. Then build the project in debug mode (you may want to add /m[:<N>] flag to run msbuild in <N> parallel threads # 4. Then build the project in debug mode (you may want to add /m[:<N>] flag to run msbuild in <N> parallel threads
# or simply /m to use all avail cores) # or simply /m to use all avail cores)
# msbuild rocksdb.sln # msbuild rocksdb.sln
@ -27,7 +27,7 @@
# #
# Linux: # Linux:
# #
# 1. Install a recent toolchain such as devtoolset-3 if you're on a older distro. C++11 required. # 1. Install a recent toolchain if you're on a older distro. C++17 required (GCC >= 7, Clang >= 5)
# 2. mkdir build; cd build # 2. mkdir build; cd build
# 3. cmake .. # 3. cmake ..
# 4. make -j # 4. make -j
@ -92,7 +92,7 @@ else()
endif() endif()
if( NOT DEFINED CMAKE_CXX_STANDARD ) if( NOT DEFINED CMAKE_CXX_STANDARD )
set(CMAKE_CXX_STANDARD 11) set(CMAKE_CXX_STANDARD 17)
endif() endif()
include(CMakeDependentOption) include(CMakeDependentOption)
@ -342,7 +342,7 @@ endif()
# Check if -latomic is required or not # Check if -latomic is required or not
if (NOT MSVC) if (NOT MSVC)
set(CMAKE_REQUIRED_FLAGS "--std=c++11") set(CMAKE_REQUIRED_FLAGS "--std=c++17")
CHECK_CXX_SOURCE_COMPILES(" CHECK_CXX_SOURCE_COMPILES("
#include <atomic> #include <atomic>
std::atomic<uint64_t> x(0); std::atomic<uint64_t> x(0);
@ -369,18 +369,8 @@ endif()
# Reset the required flags # Reset the required flags
set(CMAKE_REQUIRED_FLAGS ${OLD_CMAKE_REQUIRED_FLAGS}) set(CMAKE_REQUIRED_FLAGS ${OLD_CMAKE_REQUIRED_FLAGS})
CHECK_CXX_SOURCE_COMPILES(" # thread_local is part of C++11 and later (TODO: clean up this define)
#if defined(_MSC_VER) && !defined(__thread) add_definitions(-DROCKSDB_SUPPORT_THREAD_LOCAL)
#define __thread __declspec(thread)
#endif
int main() {
static __thread int tls;
(void)tls;
}
" HAVE_THREAD_LOCAL)
if(HAVE_THREAD_LOCAL)
add_definitions(-DROCKSDB_SUPPORT_THREAD_LOCAL)
endif()
option(WITH_IOSTATS_CONTEXT "Enable IO stats context" ON) option(WITH_IOSTATS_CONTEXT "Enable IO stats context" ON)
if (NOT WITH_IOSTATS_CONTEXT) if (NOT WITH_IOSTATS_CONTEXT)

View File

@ -5,6 +5,7 @@
* Fixed more cases of EventListener::OnTableFileCreated called with OK status, file_size==0, and no SST file kept. Now the status is Aborted. * Fixed more cases of EventListener::OnTableFileCreated called with OK status, file_size==0, and no SST file kept. Now the status is Aborted.
### Public API changes ### Public API changes
* Require C++17 compatible compiler (GCC >= 7, Clang >= 5, Visual Studio >= 2017). See #9388.
* Remove HDFS support from main repo. * Remove HDFS support from main repo.
* Remove librados support from main repo. * Remove librados support from main repo.
* Remove obsolete backupable_db.h and type alias `BackupableDBOptions`. Use backup_engine.h and `BackupEngineOptions`. Similar renamings are in the C and Java APIs. * Remove obsolete backupable_db.h and type alias `BackupableDBOptions`. Use backup_engine.h and `BackupEngineOptions`. Similar renamings are in the C and Java APIs.

View File

@ -6,7 +6,7 @@ than release mode.
RocksDB's library should be able to compile without any dependency installed, RocksDB's library should be able to compile without any dependency installed,
although we recommend installing some compression libraries (see below). although we recommend installing some compression libraries (see below).
We do depend on newer gcc/clang with C++11 support. We do depend on newer gcc/clang with C++17 support (GCC >= 7, Clang >= 5).
There are few options when compiling RocksDB: There are few options when compiling RocksDB:
@ -50,7 +50,7 @@ to build a portable binary, add `PORTABLE=1` before your make commands, like thi
## Supported platforms ## Supported platforms
* **Linux - Ubuntu** * **Linux - Ubuntu**
* Upgrade your gcc to version at least 4.8 to get C++11 support. * Upgrade your gcc to version at least 7 to get C++17 support.
* Install gflags. First, try: `sudo apt-get install libgflags-dev` * Install gflags. First, try: `sudo apt-get install libgflags-dev`
If this doesn't work and you're using Ubuntu, here's a nice tutorial: If this doesn't work and you're using Ubuntu, here's a nice tutorial:
(http://askubuntu.com/questions/312173/installing-gflags-12-04) (http://askubuntu.com/questions/312173/installing-gflags-12-04)
@ -62,8 +62,7 @@ to build a portable binary, add `PORTABLE=1` before your make commands, like thi
* Install zstandard: `sudo apt-get install libzstd-dev`. * Install zstandard: `sudo apt-get install libzstd-dev`.
* **Linux - CentOS / RHEL** * **Linux - CentOS / RHEL**
* Upgrade your gcc to version at least 4.8 to get C++11 support: * Upgrade your gcc to version at least 7 to get C++17 support
`yum install gcc48-c++`
* Install gflags: * Install gflags:
git clone https://github.com/gflags/gflags.git git clone https://github.com/gflags/gflags.git
@ -113,11 +112,11 @@ to build a portable binary, add `PORTABLE=1` before your make commands, like thi
make && sudo make install make && sudo make install
* **OS X**: * **OS X**:
* Install latest C++ compiler that supports C++ 11: * Install latest C++ compiler that supports C++ 17:
* Update XCode: run `xcode-select --install` (or install it from XCode App's settting). * Update XCode: run `xcode-select --install` (or install it from XCode App's settting).
* Install via [homebrew](http://brew.sh/). * Install via [homebrew](http://brew.sh/).
* If you're first time developer in MacOS, you still need to run: `xcode-select --install` in your command line. * If you're first time developer in MacOS, you still need to run: `xcode-select --install` in your command line.
* run `brew tap homebrew/versions; brew install gcc48 --use-llvm` to install gcc 4.8 (or higher). * run `brew tap homebrew/versions; brew install gcc7 --use-llvm` to install gcc 7 (or higher).
* run `brew install rocksdb` * run `brew install rocksdb`
* **FreeBSD** (11.01): * **FreeBSD** (11.01):
@ -160,7 +159,7 @@ to build a portable binary, add `PORTABLE=1` before your make commands, like thi
* Install the dependencies for RocksDB: * Install the dependencies for RocksDB:
pkg_add gmake gflags snappy bzip2 lz4 zstd git jdk bash findutils gnuwatch pkg_add gmake gflags snappy bzip2 lz4 zstd git jdk bash findutils gnuwatch
* Build RocksDB from source: * Build RocksDB from source:
@ -182,13 +181,13 @@ to build a portable binary, add `PORTABLE=1` before your make commands, like thi
* **Windows**: * **Windows**:
* For building with MS Visual Studio 13 you will need Update 4 installed. * For building with MS Visual Studio 13 you will need Update 4 installed.
* Read and follow the instructions at CMakeLists.txt * Read and follow the instructions at CMakeLists.txt
* Or install via [vcpkg](https://github.com/microsoft/vcpkg) * Or install via [vcpkg](https://github.com/microsoft/vcpkg)
* run `vcpkg install rocksdb:x64-windows` * run `vcpkg install rocksdb:x64-windows`
* **AIX 6.1** * **AIX 6.1**
* Install AIX Toolbox rpms with gcc * Install AIX Toolbox rpms with gcc
* Use these environment variables: * Use these environment variables:
export PORTABLE=1 export PORTABLE=1
export CC=gcc export CC=gcc
export AR="ar -X64" export AR="ar -X64"
@ -199,9 +198,9 @@ to build a portable binary, add `PORTABLE=1` before your make commands, like thi
export LIBPATH=/opt/freeware/lib export LIBPATH=/opt/freeware/lib
export JAVA_HOME=/usr/java8_64 export JAVA_HOME=/usr/java8_64
export PATH=/opt/freeware/bin:$PATH export PATH=/opt/freeware/bin:$PATH
* **Solaris Sparc** * **Solaris Sparc**
* Install GCC 4.8.2 and higher. * Install GCC 7 and higher.
* Use these environment variables: * Use these environment variables:
export CC=gcc export CC=gcc
@ -210,4 +209,3 @@ to build a portable binary, add `PORTABLE=1` before your make commands, like thi
export EXTRA_LDFLAGS=-m64 export EXTRA_LDFLAGS=-m64
export PORTABLE=1 export PORTABLE=1
export PLATFORM_LDFLAGS="-static-libstdc++ -static-libgcc" export PLATFORM_LDFLAGS="-static-libstdc++ -static-libgcc"

View File

@ -215,7 +215,7 @@ am__v_AR_0 = @echo " AR " $@;
am__v_AR_1 = am__v_AR_1 =
AM_LINK = $(AM_V_CCLD)$(CXX) -L. $(patsubst lib%.a, -l%, $(patsubst lib%.$(PLATFORM_SHARED_EXT), -l%, $^)) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS) AM_LINK = $(AM_V_CCLD)$(CXX) -L. $(patsubst lib%.a, -l%, $(patsubst lib%.$(PLATFORM_SHARED_EXT), -l%, $^)) $(EXEC_LDFLAGS) -o $@ $(LDFLAGS) $(COVERAGEFLAGS)
AM_SHARE = $(AM_V_CCLD) $(CXX) $(PLATFORM_SHARED_LDFLAGS)$@ -L. $(patsubst lib%.$(PLATFORM_SHARED_EXT), -l%, $^) $(LDFLAGS) -o $@ AM_SHARE = $(AM_V_CCLD) $(CXX) $(PLATFORM_SHARED_LDFLAGS)$@ -L. $(patsubst lib%.$(PLATFORM_SHARED_EXT), -l%, $^) $(EXEC_LDFLAGS) $(LDFLAGS) -o $@
# Detect what platform we're building on. # Detect what platform we're building on.
# Export some common variables that might have been passed as Make variables # Export some common variables that might have been passed as Make variables
@ -574,27 +574,29 @@ check-headers: $(HEADER_OK_FILES)
# options_settable_test doesn't pass with UBSAN as we use hack in the test # options_settable_test doesn't pass with UBSAN as we use hack in the test
ifdef COMPILE_WITH_UBSAN ifdef COMPILE_WITH_UBSAN
TESTS := $(shell echo $(TESTS) | sed 's/\boptions_settable_test\b//g') TESTS := $(shell echo $(TESTS) | sed 's/\boptions_settable_test\b//g')
endif endif
ifdef ASSERT_STATUS_CHECKED ifdef ASSERT_STATUS_CHECKED
# TODO: finish fixing all tests to pass this check # TODO: finish fixing all tests to pass this check
TESTS_FAILING_ASC = \ TESTS_FAILING_ASC = \
c_test \ c_test \
env_test \ env_test \
range_locking_test \ range_locking_test \
testutil_test \ testutil_test \
# Since we have very few ASC exclusions left, excluding them from # Since we have very few ASC exclusions left, excluding them from
# the build is the most convenient way to exclude them from testing # the build is the most convenient way to exclude them from testing
TESTS := $(filter-out $(TESTS_FAILING_ASC),$(TESTS)) TESTS := $(filter-out $(TESTS_FAILING_ASC),$(TESTS))
endif endif
ROCKSDBTESTS_SUBSET ?= $(TESTS) ROCKSDBTESTS_SUBSET ?= $(TESTS)
# c_test - doesn't use gtest
# env_test - suspicious use of test::TmpDir # env_test - suspicious use of test::TmpDir
# deletefile_test - serial because it generates giant temporary files in # deletefile_test - serial because it generates giant temporary files in
# its various tests. Parallel can fill up your /dev/shm # its various tests. Parallel can fill up your /dev/shm
NON_PARALLEL_TEST = \ NON_PARALLEL_TEST = \
c_test \
env_test \ env_test \
deletefile_test \ deletefile_test \
@ -852,7 +854,7 @@ endif
parallel_tests = $(patsubst %,parallel_%,$(PARALLEL_TEST)) parallel_tests = $(patsubst %,parallel_%,$(PARALLEL_TEST))
.PHONY: gen_parallel_tests $(parallel_tests) .PHONY: gen_parallel_tests $(parallel_tests)
$(parallel_tests): $(PARALLEL_TEST) $(parallel_tests):
$(AM_V_at)TEST_BINARY=$(patsubst parallel_%,%,$@); \ $(AM_V_at)TEST_BINARY=$(patsubst parallel_%,%,$@); \
TEST_NAMES=` \ TEST_NAMES=` \
(./$$TEST_BINARY --gtest_list_tests || echo " $${TEST_BINARY}__list_tests_failure") \ (./$$TEST_BINARY --gtest_list_tests || echo " $${TEST_BINARY}__list_tests_failure") \
@ -2236,7 +2238,7 @@ rocksdbjavastaticosx_ub: rocksdbjavastaticosx_archs
lipo -create -output ./java/target/$(ROCKSDBJNILIB) java/target/librocksdbjni-osx-x86_64.jnilib java/target/librocksdbjni-osx-arm64.jnilib lipo -create -output ./java/target/$(ROCKSDBJNILIB) java/target/librocksdbjni-osx-x86_64.jnilib java/target/librocksdbjni-osx-arm64.jnilib
$(MAKE) rocksdbjavastatic_jar $(MAKE) rocksdbjavastatic_jar
rocksdbjavastaticosx_archs: rocksdbjavastaticosx_archs:
$(MAKE) rocksdbjavastaticosx_arch_x86_64 $(MAKE) rocksdbjavastaticosx_arch_x86_64
$(MAKE) rocksdbjavastaticosx_arch_arm64 $(MAKE) rocksdbjavastaticosx_arch_arm64

View File

@ -1,74 +0,0 @@
version: 1.0.{build}
image: Visual Studio 2019
environment:
JAVA_HOME: C:\Program Files\Java\jdk1.8.0
THIRDPARTY_HOME: $(APPVEYOR_BUILD_FOLDER)\thirdparty
SNAPPY_HOME: $(THIRDPARTY_HOME)\snappy-1.1.7
SNAPPY_INCLUDE: $(SNAPPY_HOME);$(SNAPPY_HOME)\build
SNAPPY_LIB_DEBUG: $(SNAPPY_HOME)\build\Debug\snappy.lib
SNAPPY_LIB_RELEASE: $(SNAPPY_HOME)\build\Release\snappy.lib
LZ4_HOME: $(THIRDPARTY_HOME)\lz4-1.8.3
LZ4_INCLUDE: $(LZ4_HOME)\lib
LZ4_LIB_DEBUG: $(LZ4_HOME)\visual\VS2010\bin\x64_Debug\liblz4_static.lib
LZ4_LIB_RELEASE: $(LZ4_HOME)\visual\VS2010\bin\x64_Release\liblz4_static.lib
ZSTD_HOME: $(THIRDPARTY_HOME)\zstd-1.4.0
ZSTD_INCLUDE: $(ZSTD_HOME)\lib;$(ZSTD_HOME)\lib\dictBuilder
ZSTD_LIB_DEBUG: $(ZSTD_HOME)\build\VS2010\bin\x64_Debug\libzstd_static.lib
ZSTD_LIB_RELEASE: $(ZSTD_HOME)\build\VS2010\bin\x64_Release\libzstd_static.lib
matrix:
- APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2015
CMAKE_GENERATOR: Visual Studio 14 Win64
DEV_ENV: C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\devenv.com
install:
- md %THIRDPARTY_HOME%
- echo "Building Snappy dependency..."
- cd %THIRDPARTY_HOME%
- curl --fail --silent --show-error --output snappy-1.1.7.zip --location https://github.com/google/snappy/archive/1.1.7.zip
- unzip snappy-1.1.7.zip
- cd snappy-1.1.7
- mkdir build
- cd build
- if DEFINED CMAKE_PLATEFORM_NAME (set "PLATEFORM_OPT=-A %CMAKE_PLATEFORM_NAME%")
- cmake .. -G "%CMAKE_GENERATOR%" %PLATEFORM_OPT%
- msbuild Snappy.sln /p:Configuration=Debug /p:Platform=x64
- msbuild Snappy.sln /p:Configuration=Release /p:Platform=x64
- echo "Building LZ4 dependency..."
- cd %THIRDPARTY_HOME%
- curl --fail --silent --show-error --output lz4-1.8.3.zip --location https://github.com/lz4/lz4/archive/v1.8.3.zip
- unzip lz4-1.8.3.zip
- cd lz4-1.8.3\visual\VS2010
- ps: $CMD="$Env:DEV_ENV"; & $CMD lz4.sln /upgrade
- msbuild lz4.sln /p:Configuration=Debug /p:Platform=x64
- msbuild lz4.sln /p:Configuration=Release /p:Platform=x64
- echo "Building ZStd dependency..."
- cd %THIRDPARTY_HOME%
- curl --fail --silent --show-error --output zstd-1.4.0.zip --location https://github.com/facebook/zstd/archive/v1.4.0.zip
- unzip zstd-1.4.0.zip
- cd zstd-1.4.0\build\VS2010
- ps: $CMD="$Env:DEV_ENV"; & $CMD zstd.sln /upgrade
- msbuild zstd.sln /p:Configuration=Debug /p:Platform=x64
- msbuild zstd.sln /p:Configuration=Release /p:Platform=x64
before_build:
- md %APPVEYOR_BUILD_FOLDER%\build
- cd %APPVEYOR_BUILD_FOLDER%\build
- if DEFINED CMAKE_PLATEFORM_NAME (set "PLATEFORM_OPT=-A %CMAKE_PLATEFORM_NAME%")
- cmake .. -G "%CMAKE_GENERATOR%" %PLATEFORM_OPT% %CMAKE_OPT% -DCMAKE_BUILD_TYPE=Debug -DOPTDBG=1 -DPORTABLE=1 -DSNAPPY=1 -DLZ4=1 -DZSTD=1 -DXPRESS=1 -DJNI=1 -DWITH_ALL_TESTS=0
- cd ..
build:
project: build\rocksdb.sln
parallel: true
verbosity: normal
test:
test_script:
- ps: build_tools\run_ci_db_test.ps1 -SuiteRun db_basic_test,env_basic_test -Concurrency 8
on_failure:
- cmd: 7z a build-failed.zip %APPVEYOR_BUILD_FOLDER%\build\ && appveyor PushArtifact build-failed.zip

View File

@ -45,11 +45,11 @@ if test -z "$OUTPUT"; then
exit 1 exit 1
fi fi
# we depend on C++11, but should be compatible with newer standards # we depend on C++17, but should be compatible with newer standards
if [ "$ROCKSDB_CXX_STANDARD" ]; then if [ "$ROCKSDB_CXX_STANDARD" ]; then
PLATFORM_CXXFLAGS="-std=$ROCKSDB_CXX_STANDARD" PLATFORM_CXXFLAGS="-std=$ROCKSDB_CXX_STANDARD"
else else
PLATFORM_CXXFLAGS="-std=c++11" PLATFORM_CXXFLAGS="-std=c++17"
fi fi
# we currently depend on POSIX platform # we currently depend on POSIX platform
@ -794,24 +794,8 @@ if [ "$?" = 0 ]; then
COMMON_FLAGS="$COMMON_FLAGS -DHAVE_UINT128_EXTENSION" COMMON_FLAGS="$COMMON_FLAGS -DHAVE_UINT128_EXTENSION"
fi fi
# iOS doesn't support thread-local storage, but this check would erroneously # thread_local is part of C++11 and later (TODO: clean up this define)
# succeed because the cross-compiler flags are added by the Makefile, not this COMMON_FLAGS="$COMMON_FLAGS -DROCKSDB_SUPPORT_THREAD_LOCAL"
# script.
if [ "$PLATFORM" != IOS ]; then
$CXX $COMMON_FLAGS -x c++ - -o test.o 2>/dev/null <<EOF
#if defined(_MSC_VER) && !defined(__thread)
#define __thread __declspec(thread)
#endif
int main() {
static __thread int tls;
(void)tls;
}
EOF
if [ "$?" = 0 ]; then
COMMON_FLAGS="$COMMON_FLAGS -DROCKSDB_SUPPORT_THREAD_LOCAL"
fi
fi
if [ "$FBCODE_BUILD" != "true" -a "$PLATFORM" = OS_LINUX ]; then if [ "$FBCODE_BUILD" != "true" -a "$PLATFORM" = OS_LINUX ]; then
$CXX $COMMON_FLAGS $PLATFORM_SHARED_CFLAGS -x c++ -c - -o test_dl.o 2>/dev/null <<EOF $CXX $COMMON_FLAGS $PLATFORM_SHARED_CFLAGS -x c++ -c - -o test_dl.o 2>/dev/null <<EOF

View File

@ -511,10 +511,12 @@ TEST_F(LRUSecondaryCacheTest, BasicFailTest) {
Random rnd(301); Random rnd(301);
std::string str1 = rnd.RandomString(1020); std::string str1 = rnd.RandomString(1020);
TestItem* item1 = new TestItem(str1.data(), str1.length()); auto item1 = std::make_unique<TestItem>(str1.data(), str1.length());
ASSERT_NOK(cache->Insert("k1", item1, nullptr, str1.length())); ASSERT_TRUE(cache->Insert("k1", item1.get(), nullptr, str1.length())
ASSERT_OK(cache->Insert("k1", item1, &LRUSecondaryCacheTest::helper_, .IsInvalidArgument());
ASSERT_OK(cache->Insert("k1", item1.get(), &LRUSecondaryCacheTest::helper_,
str1.length())); str1.length()));
item1.release(); // Appease clang-analyze "potential memory leak"
Cache::Handle* handle; Cache::Handle* handle;
handle = cache->Lookup("k2", nullptr, test_item_creator, Cache::Priority::LOW, handle = cache->Lookup("k2", nullptr, test_item_creator, Cache::Priority::LOW,

4
env/mock_env.cc vendored
View File

@ -790,7 +790,11 @@ IOStatus MockFileSystem::GetChildren(const std::string& dir,
IODebugContext* /*dbg*/) { IODebugContext* /*dbg*/) {
MutexLock lock(&mutex_); MutexLock lock(&mutex_);
bool found_dir = GetChildrenInternal(dir, result); bool found_dir = GetChildrenInternal(dir, result);
#ifndef __clang_analyzer__
return found_dir ? IOStatus::OK() : IOStatus::NotFound(dir); return found_dir ? IOStatus::OK() : IOStatus::NotFound(dir);
#else
return found_dir ? IOStatus::OK() : IOStatus::NotFound();
#endif
} }
void MockFileSystem::DeleteFileInternal(const std::string& fname) { void MockFileSystem::DeleteFileInternal(const std::string& fname) {

View File

@ -19,16 +19,16 @@ CFLAGS += -Wstrict-prototypes
all: simple_example column_families_example compact_files_example c_simple_example optimistic_transaction_example transaction_example compaction_filter_example options_file_example all: simple_example column_families_example compact_files_example c_simple_example optimistic_transaction_example transaction_example compaction_filter_example options_file_example
simple_example: librocksdb simple_example.cc simple_example: librocksdb simple_example.cc
$(CXX) $(CXXFLAGS) $@.cc -o$@ ../librocksdb.a -I../include -O2 -std=c++11 $(PLATFORM_LDFLAGS) $(PLATFORM_CXXFLAGS) $(EXEC_LDFLAGS) $(CXX) $(CXXFLAGS) $@.cc -o$@ ../librocksdb.a -I../include -O2 -std=c++17 $(PLATFORM_LDFLAGS) $(PLATFORM_CXXFLAGS) $(EXEC_LDFLAGS)
column_families_example: librocksdb column_families_example.cc column_families_example: librocksdb column_families_example.cc
$(CXX) $(CXXFLAGS) $@.cc -o$@ ../librocksdb.a -I../include -O2 -std=c++11 $(PLATFORM_LDFLAGS) $(PLATFORM_CXXFLAGS) $(EXEC_LDFLAGS) $(CXX) $(CXXFLAGS) $@.cc -o$@ ../librocksdb.a -I../include -O2 -std=c++17 $(PLATFORM_LDFLAGS) $(PLATFORM_CXXFLAGS) $(EXEC_LDFLAGS)
compaction_filter_example: librocksdb compaction_filter_example.cc compaction_filter_example: librocksdb compaction_filter_example.cc
$(CXX) $(CXXFLAGS) $@.cc -o$@ ../librocksdb.a -I../include -O2 -std=c++11 $(PLATFORM_LDFLAGS) $(PLATFORM_CXXFLAGS) $(EXEC_LDFLAGS) $(CXX) $(CXXFLAGS) $@.cc -o$@ ../librocksdb.a -I../include -O2 -std=c++17 $(PLATFORM_LDFLAGS) $(PLATFORM_CXXFLAGS) $(EXEC_LDFLAGS)
compact_files_example: librocksdb compact_files_example.cc compact_files_example: librocksdb compact_files_example.cc
$(CXX) $(CXXFLAGS) $@.cc -o$@ ../librocksdb.a -I../include -O2 -std=c++11 $(PLATFORM_LDFLAGS) $(PLATFORM_CXXFLAGS) $(EXEC_LDFLAGS) $(CXX) $(CXXFLAGS) $@.cc -o$@ ../librocksdb.a -I../include -O2 -std=c++17 $(PLATFORM_LDFLAGS) $(PLATFORM_CXXFLAGS) $(EXEC_LDFLAGS)
.c.o: .c.o:
$(CC) $(CFLAGS) -c $< -o $@ -I../include $(CC) $(CFLAGS) -c $< -o $@ -I../include
@ -37,16 +37,16 @@ c_simple_example: librocksdb c_simple_example.o
$(CXX) $@.o -o$@ ../librocksdb.a $(PLATFORM_LDFLAGS) $(EXEC_LDFLAGS) $(CXX) $@.o -o$@ ../librocksdb.a $(PLATFORM_LDFLAGS) $(EXEC_LDFLAGS)
optimistic_transaction_example: librocksdb optimistic_transaction_example.cc optimistic_transaction_example: librocksdb optimistic_transaction_example.cc
$(CXX) $(CXXFLAGS) $@.cc -o$@ ../librocksdb.a -I../include -O2 -std=c++11 $(PLATFORM_LDFLAGS) $(PLATFORM_CXXFLAGS) $(EXEC_LDFLAGS) $(CXX) $(CXXFLAGS) $@.cc -o$@ ../librocksdb.a -I../include -O2 -std=c++17 $(PLATFORM_LDFLAGS) $(PLATFORM_CXXFLAGS) $(EXEC_LDFLAGS)
transaction_example: librocksdb transaction_example.cc transaction_example: librocksdb transaction_example.cc
$(CXX) $(CXXFLAGS) $@.cc -o$@ ../librocksdb.a -I../include -O2 -std=c++11 $(PLATFORM_LDFLAGS) $(PLATFORM_CXXFLAGS) $(EXEC_LDFLAGS) $(CXX) $(CXXFLAGS) $@.cc -o$@ ../librocksdb.a -I../include -O2 -std=c++17 $(PLATFORM_LDFLAGS) $(PLATFORM_CXXFLAGS) $(EXEC_LDFLAGS)
options_file_example: librocksdb options_file_example.cc options_file_example: librocksdb options_file_example.cc
$(CXX) $(CXXFLAGS) $@.cc -o$@ ../librocksdb.a -I../include -O2 -std=c++11 $(PLATFORM_LDFLAGS) $(PLATFORM_CXXFLAGS) $(EXEC_LDFLAGS) $(CXX) $(CXXFLAGS) $@.cc -o$@ ../librocksdb.a -I../include -O2 -std=c++17 $(PLATFORM_LDFLAGS) $(PLATFORM_CXXFLAGS) $(EXEC_LDFLAGS)
multi_processes_example: librocksdb multi_processes_example.cc multi_processes_example: librocksdb multi_processes_example.cc
$(CXX) $(CXXFLAGS) $@.cc -o$@ ../librocksdb.a -I../include -O2 -std=c++11 $(PLATFORM_LDFLAGS) $(PLATFORM_CXXFLAGS) $(EXEC_LDFLAGS) $(CXX) $(CXXFLAGS) $@.cc -o$@ ../librocksdb.a -I../include -O2 -std=c++17 $(PLATFORM_LDFLAGS) $(PLATFORM_CXXFLAGS) $(EXEC_LDFLAGS)
clean: clean:
rm -rf ./simple_example ./column_families_example ./compact_files_example ./compaction_filter_example ./c_simple_example c_simple_example.o ./optimistic_transaction_example ./transaction_example ./options_file_example ./multi_processes_example rm -rf ./simple_example ./column_families_example ./compact_files_example ./compaction_filter_example ./c_simple_example c_simple_example.o ./optimistic_transaction_example ./transaction_example ./options_file_example ./multi_processes_example

View File

@ -340,7 +340,7 @@ extern ROCKSDB_LIBRARY_API rocksdb_t* rocksdb_open_as_secondary_column_families(
const char* secondary_path, int num_column_families, const char* secondary_path, int num_column_families,
const char* const* column_family_names, const char* const* column_family_names,
const rocksdb_options_t* const* column_family_options, const rocksdb_options_t* const* column_family_options,
rocksdb_column_family_handle_t** colummn_family_handles, char** errptr); rocksdb_column_family_handle_t** column_family_handles, char** errptr);
extern ROCKSDB_LIBRARY_API char** rocksdb_list_column_families( extern ROCKSDB_LIBRARY_API char** rocksdb_list_column_families(
const rocksdb_options_t* options, const char* name, size_t* lencf, const rocksdb_options_t* options, const char* name, size_t* lencf,

View File

@ -18,15 +18,12 @@
#pragma once #pragma once
#include <assert.h> #include <cassert>
#include <stddef.h> #include <cstddef>
#include <string.h>
#include <cstdio> #include <cstdio>
#include <cstring>
#include <string> #include <string>
#include <string_view> // RocksDB now requires C++17 support
#ifdef __cpp_lib_string_view
#include <string_view>
#endif
#include "rocksdb/cleanable.h" #include "rocksdb/cleanable.h"
@ -44,11 +41,9 @@ class Slice {
/* implicit */ /* implicit */
Slice(const std::string& s) : data_(s.data()), size_(s.size()) {} Slice(const std::string& s) : data_(s.data()), size_(s.size()) {}
#ifdef __cpp_lib_string_view
// Create a slice that refers to the same contents as "sv" // Create a slice that refers to the same contents as "sv"
/* implicit */ /* implicit */
Slice(std::string_view sv) : data_(sv.data()), size_(sv.size()) {} Slice(const std::string_view& sv) : data_(sv.data()), size_(sv.size()) {}
#endif
// Create a slice that refers to s[0,strlen(s)-1] // Create a slice that refers to s[0,strlen(s)-1]
/* implicit */ /* implicit */
@ -96,12 +91,10 @@ class Slice {
// when hex is true, returns a string of twice the length hex encoded (0-9A-F) // when hex is true, returns a string of twice the length hex encoded (0-9A-F)
std::string ToString(bool hex = false) const; std::string ToString(bool hex = false) const;
#ifdef __cpp_lib_string_view
// Return a string_view that references the same data as this slice. // Return a string_view that references the same data as this slice.
std::string_view ToStringView() const { std::string_view ToStringView() const {
return std::string_view(data_, size_); return std::string_view(data_, size_);
} }
#endif
// Decodes the current slice interpreted as an hexadecimal string into result, // Decodes the current slice interpreted as an hexadecimal string into result,
// if successful returns true, if this isn't a valid hex string // if successful returns true, if this isn't a valid hex string

View File

@ -25,6 +25,11 @@
#define ROCKSDB_NOEXCEPT noexcept #define ROCKSDB_NOEXCEPT noexcept
// thread_local is part of C++11 and later (TODO: clean up this define)
#ifndef __thread
#define __thread thread_local
#endif
#undef PLATFORM_IS_LITTLE_ENDIAN #undef PLATFORM_IS_LITTLE_ENDIAN
#if defined(OS_MACOSX) #if defined(OS_MACOSX)
#include <machine/endian.h> #include <machine/endian.h>

View File

@ -60,10 +60,9 @@ using ssize_t = SSIZE_T;
#ifdef _MSC_VER #ifdef _MSC_VER
#define __attribute__(A) #define __attribute__(A)
// Thread local storage on Linux // thread_local is part of C++11 and later (TODO: clean up this define)
// There is thread_local in C++11
#ifndef __thread #ifndef __thread
#define __thread __declspec(thread) #define __thread thread_local
#endif #endif
#endif #endif

View File

@ -15,6 +15,11 @@
#ifndef DEFINE_uint32 #ifndef DEFINE_uint32
// DEFINE_uint32 does not appear in older versions of gflags. This should be // DEFINE_uint32 does not appear in older versions of gflags. This should be
// a sane definition for those versions. // a sane definition for those versions.
#include <cstdint>
#define DEFINE_uint32(name, val, txt) \ #define DEFINE_uint32(name, val, txt) \
DEFINE_VARIABLE(GFLAGS_NAMESPACE::uint32, U, name, val, txt) namespace gflags_compat { \
DEFINE_int32(name, val, txt); \
} \
uint32_t &FLAGS_##name = \
*reinterpret_cast<uint32_t *>(&gflags_compat::FLAGS_##name);
#endif #endif

View File

@ -16,6 +16,13 @@
namespace ROCKSDB_NAMESPACE { namespace ROCKSDB_NAMESPACE {
TEST(SliceTest, StringView) {
std::string s = "foo";
std::string_view sv = s;
ASSERT_EQ(Slice(s), Slice(sv));
ASSERT_EQ(Slice(s), Slice(std::move(sv)));
}
// Use this to keep track of the cleanups that were actually performed // Use this to keep track of the cleanups that were actually performed
void Multiplier(void* arg1, void* arg2) { void Multiplier(void* arg1, void* arg2) {
int* res = reinterpret_cast<int*>(arg1); int* res = reinterpret_cast<int*>(arg1);