dolysis
/
rocksdb
mirror of https://github.com/facebook/rocksdb.git
3
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
rocksdb/build_tools/fbcode.gcc471.sh

62 lines
2.9 KiB
Bash
Raw Normal View History

Compile leveldb with gcc 4.7.1 Test Plan: run unit tests Reviewers: heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D5163
2012-09-05 07:02:16 +00:00
#!/bin/sh
#
# Set environment variables so that we can compile leveldb using
# fbcode settings. It uses the latest g++ compiler and also
# uses jemalloc
Updating fbcode.gcc471.sh to use jemalloc 3.3.1 Summary: Updated TOOL_CHAIN_LIB_BASE to use the third-party version for jemalloc-3.3.1 which contains a bug fix in quarantine.cc. This was detected while debugging valgrind issues with the rocksdb table_test Test Plan: make table_test;valgrind --leak-check=full ./table_test Reviewers: dhruba, sheki, vamsi Reviewed By: sheki Differential Revision: https://reviews.facebook.net/D9387
2013-03-13 21:37:16 +00:00
TOOLCHAIN_REV=83eb773e262fa705eaebbf3de40db71d53fabf2e
Compile leveldb with gcc 4.7.1 Test Plan: run unit tests Reviewers: heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D5163
2012-09-05 07:02:16 +00:00
TOOLCHAIN_EXECUTABLES="/mnt/gvfs/third-party/$TOOLCHAIN_REV/centos5.2-native"
TOOLCHAIN_LIB_BASE="/mnt/gvfs/third-party/$TOOLCHAIN_REV/gcc-4.7.1-glibc-2.14.1"
Updating fbcode.gcc471.sh to use jemalloc 3.3.1 Summary: Updated TOOL_CHAIN_LIB_BASE to use the third-party version for jemalloc-3.3.1 which contains a bug fix in quarantine.cc. This was detected while debugging valgrind issues with the rocksdb table_test Test Plan: make table_test;valgrind --leak-check=full ./table_test Reviewers: dhruba, sheki, vamsi Reviewed By: sheki Differential Revision: https://reviews.facebook.net/D9387
2013-03-13 21:37:16 +00:00
TOOL_JEMALLOC=jemalloc-3.3.1/2f45f3a
Compile leveldb with gcc 4.7.1 Test Plan: run unit tests Reviewers: heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D5163
2012-09-05 07:02:16 +00:00
# location of libhdfs libraries
if test "$USE_HDFS"; then
JAVA_HOME="/usr/local/jdk-6u22-64"
JINCLUDE="-I$JAVA_HOME/include -I$JAVA_HOME/include/linux"
GLIBC_RUNTIME_PATH="/usr/local/fbcode/gcc-4.7.1-glibc-2.14.1"
HDFSLIB=" -Wl,--no-whole-archive hdfs/libhdfs.a -L$JAVA_HOME/jre/lib/amd64 "
HDFSLIB+=" -L$JAVA_HOME/jre/lib/amd64/server -L$GLIBC_RUNTIME_PATH/lib "
HDFSLIB+=" -ldl -lverify -ljava -ljvm "
fi
# location of snappy headers and libraries
Fix ldb segfault and use static libsnappy for all builds Summary: Link statically against snappy, using the gvfs one for facebook environments, and the bundled one otherwise. In addition, fix a few minor segfaults in ldb when it couldn't open the database, and update .gitignore to include a few other build artifacts. Test Plan: make check Reviewers: dhruba Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D6855
2012-11-21 00:14:04 +00:00
SNAPPY_INCLUDE=" -I $TOOLCHAIN_LIB_BASE/snappy/snappy-1.0.3/7518bbe/include"
SNAPPY_LIBS=" $TOOLCHAIN_LIB_BASE/snappy/snappy-1.0.3/7518bbe/lib/libsnappy.a"
Compile leveldb with gcc 4.7.1 Test Plan: run unit tests Reviewers: heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D5163
2012-09-05 07:02:16 +00:00
Add zlib to our builds and tweak histogram output Summary: $SUBJECT -- cosmetic fix for histograms, print P75/P99, and make sure zlib is enabled for our command line tools. Test Plan: compile, test db_bench with --compression_type=zlib Reviewers: heyongqiang Reviewed By: heyongqiang CC: adsharma, leveldb Differential Revision: https://reviews.facebook.net/D8445
2013-02-07 01:12:43 +00:00
# location of zlib headers and libraries
ZLIB_INCLUDE=" -I $TOOLCHAIN_LIB_BASE/zlib/zlib-1.2.5/91ddd43/include"
ZLIB_LIBS=" $TOOLCHAIN_LIB_BASE/zlib/zlib-1.2.5/91ddd43/lib/libz.a"
Compile leveldb with gcc 4.7.1 Test Plan: run unit tests Reviewers: heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D5163
2012-09-05 07:02:16 +00:00
# location of boost headers and libraries
Putting option -std=gnu++0x in Makefile rather than fbcode.gcc471.sh Summary: This option is needed for compilation and the open-sourced rocksdb version wiull need to get it from Makefile Test Plan: make clean;make Reviewers: MarkCallaghan, dhruba, sheki, chip CC: leveldb Differential Revision: https://reviews.facebook.net/D9243
2013-03-07 20:20:13 +00:00
THRIFT_INCLUDE=" -I $TOOLCHAIN_LIB_BASE/boost/boost-1.48.0/2a0840d/include"
Compile leveldb with gcc 4.7.1 Test Plan: run unit tests Reviewers: heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D5163
2012-09-05 07:02:16 +00:00
THRIFT_INCLUDE+=" -I./thrift -I./thrift/gen-cpp -I./thrift/lib/cpp"
Moving VALGRIND_VER which takes the valgrind version from third party to fbcode.gcc471.sh file Summary: the valgrind version being used is in facebook specific path and should be moved to the fbcode.gcc471.sh file instead of the makefile. The execution takes the environment's default valgrind version if the fbcode.gcc471.sh's valgrind_version is not available. Test Plan: make valgrind_check Reviewers: dhruba, sheki, akushner Reviewed By: dhruba Differential Revision: https://reviews.facebook.net/D9213
2013-03-07 19:11:30 +00:00
THRIFT_LIBS=" -L $TOOLCHAIN_LIB_BASE/boost/boost-1.48.0/2a0840d/lib"
Compile leveldb with gcc 4.7.1 Test Plan: run unit tests Reviewers: heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D5163
2012-09-05 07:02:16 +00:00
Add libevent include and lib directories Summary: Without this fix, I see failures like this: [zshao@dev1049 /data/users/zshao/rocksdb] . fbcode.gcc471.sh; gmake clean libleveldb.a . . . ./thrift/lib/cpp/async/TEventUtil.h:22:32: fatal error: event.h: No such file or directory Test Plan: . fbcode.gcc471.sh; make clean libleveldb.a Reviewers: dhruba, emayanke, sheki Reviewed By: sheki CC: leveldb Differential Revision: https://reviews.facebook.net/D7497
2012-12-20 22:30:26 +00:00
# location of libevent
LIBEVENT_INCLUDE=" -I $TOOLCHAIN_LIB_BASE/libevent/libevent-1.4.14b/91ddd43/include"
LIBEVENT_LIBS=" -L $TOOLCHAIN_LIB_BASE/libevent/libevent-1.4.14b/91ddd43/lib"
Enable SSE when building with fbcode support. Summary: fbcode build now support SSE instructions. Delete older version of the compile-helper fbcode.sh. This is subsumed by fbcode.gcc471.sh. Test Plan: run make check Reviewers: heyongqiang, MarkCallaghan Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D6057
2012-10-17 18:59:44 +00:00
# use Intel SSE support for checksum calculations
Do not compile thrift for fbcode build. Summary: 1. The thrift libraries do not need to be built anyore. 2. SSE is dynamically detected via https://github.com/facebook/rocksdb/commit/1aae609b920f8cd4d93ac49798fa96367b9b864c Test Plan: compile and build Reviewers: sheki, emayanke Reviewed By: sheki CC: leveldb Differential Revision: https://reviews.facebook.net/D7665
2012-12-27 22:49:24 +00:00
export USE_SSE=" -msse -msse4.2 "
Enable SSE when building with fbcode support. Summary: fbcode build now support SSE instructions. Delete older version of the compile-helper fbcode.sh. This is subsumed by fbcode.gcc471.sh. Test Plan: run make check Reviewers: heyongqiang, MarkCallaghan Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D6057
2012-10-17 18:59:44 +00:00
Fix ldb segfault and use static libsnappy for all builds Summary: Link statically against snappy, using the gvfs one for facebook environments, and the bundled one otherwise. In addition, fix a few minor segfaults in ldb when it couldn't open the database, and update .gitignore to include a few other build artifacts. Test Plan: make check Reviewers: dhruba Reviewed By: dhruba CC: leveldb Differential Revision: https://reviews.facebook.net/D6855
2012-11-21 00:14:04 +00:00
CC="$TOOLCHAIN_EXECUTABLES/gcc/gcc-4.7.1-glibc-2.14.1/bin/gcc"
Add zlib to our builds and tweak histogram output Summary: $SUBJECT -- cosmetic fix for histograms, print P75/P99, and make sure zlib is enabled for our command line tools. Test Plan: compile, test db_bench with --compression_type=zlib Reviewers: heyongqiang Reviewed By: heyongqiang CC: adsharma, leveldb Differential Revision: https://reviews.facebook.net/D8445
2013-02-07 01:12:43 +00:00
CXX="$TOOLCHAIN_EXECUTABLES/gcc/gcc-4.7.1-glibc-2.14.1/bin/g++ $JINCLUDE $SNAPPY_INCLUDE $ZLIB_INCLUDE $THRIFT_INCLUDE $LIBEVENT_INCLUDE"
Compile leveldb with gcc 4.7.1 Test Plan: run unit tests Reviewers: heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D5163
2012-09-05 07:02:16 +00:00
AR=$TOOLCHAIN_EXECUTABLES/binutils/binutils-2.21.1/da39a3e/bin/ar
RANLIB=$TOOLCHAIN_EXECUTABLES/binutils/binutils-2.21.1/da39a3e/bin/ranlib
CFLAGS="-B$TOOLCHAIN_EXECUTABLES/binutils/binutils-2.21.1/bin/gold -m64 -mtune=generic -fPIC"
Use correct version of jemalloc. Summary: Use correct version of jemalloc. Test Plan: run unit tests Reviewers: heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D5487
2012-09-17 21:34:33 +00:00
CFLAGS+=" -I $TOOLCHAIN_LIB_BASE/jemalloc/$TOOL_JEMALLOC/include -DHAVE_JEMALLOC"
Compile leveldb with gcc 4.7.1 Test Plan: run unit tests Reviewers: heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D5163
2012-09-05 07:02:16 +00:00
Add libevent include and lib directories Summary: Without this fix, I see failures like this: [zshao@dev1049 /data/users/zshao/rocksdb] . fbcode.gcc471.sh; gmake clean libleveldb.a . . . ./thrift/lib/cpp/async/TEventUtil.h:22:32: fatal error: event.h: No such file or directory Test Plan: . fbcode.gcc471.sh; make clean libleveldb.a Reviewers: dhruba, emayanke, sheki Reviewed By: sheki CC: leveldb Differential Revision: https://reviews.facebook.net/D7497
2012-12-20 22:30:26 +00:00
EXEC_LDFLAGS=" -Wl,--whole-archive $TOOLCHAIN_LIB_BASE/jemalloc/$TOOL_JEMALLOC/lib/libjemalloc.a"
Moving VALGRIND_VER which takes the valgrind version from third party to fbcode.gcc471.sh file Summary: the valgrind version being used is in facebook specific path and should be moved to the fbcode.gcc471.sh file instead of the makefile. The execution takes the environment's default valgrind version if the fbcode.gcc471.sh's valgrind_version is not available. Test Plan: make valgrind_check Reviewers: dhruba, sheki, akushner Reviewed By: dhruba Differential Revision: https://reviews.facebook.net/D9213
2013-03-07 19:11:30 +00:00
EXEC_LDFLAGS+=" -Wl,--no-whole-archive $TOOLCHAIN_LIB_BASE/libunwind/libunwind-1.0.1/91ddd43/lib/libunwind.a"
Add zlib to our builds and tweak histogram output Summary: $SUBJECT -- cosmetic fix for histograms, print P75/P99, and make sure zlib is enabled for our command line tools. Test Plan: compile, test db_bench with --compression_type=zlib Reviewers: heyongqiang Reviewed By: heyongqiang CC: adsharma, leveldb Differential Revision: https://reviews.facebook.net/D8445
2013-02-07 01:12:43 +00:00
EXEC_LDFLAGS+=" $HDFSLIB $SNAPPY_LIBS $ZLIB_LIBS $THRIFT_LIBS $LIBEVENT_LIBS"
Various build cleanups/improvements Summary: Specific changes: 1) Turn on -Werror so all warnings are errors 2) Fix some warnings the above now complains about 3) Add proper dependency support so changing a .h file forces a .c file to rebuild 4) Automatically use fbcode gcc on any internal machine rather than whatever system compiler is laying around 5) Fix jemalloc to once again be used in the builds (seemed like it wasn't being?) 6) Fix issue where 'git' would fail in build_detect_version because of LD_LIBRARY_PATH being set in the third-party build system Test Plan: make, make check, make clean, touch a header file, make sure rebuild is expected Reviewers: dhruba Reviewed By: dhruba Differential Revision: https://reviews.facebook.net/D7887
2013-01-14 20:39:24 +00:00
Add optional clang compile mode Summary: clang is an alternate compiler based on llvm. It produces nicer error messages and finds some bugs that gcc doesn't, such as the size_t change in this file (which caused some write return values to be misinterpreted!) Clang isn't the default; to try it, do "USE_CLANG=1 make" or "export USE_CLANG=1" then make as normal Test Plan: "make check" and "USE_CLANG=1 make check" Reviewers: dhruba Reviewed By: dhruba Differential Revision: https://reviews.facebook.net/D7899
2013-01-15 02:37:01 +00:00
PLATFORM_LDFLAGS="-L$TOOLCHAIN_LIB_BASE/libgcc/libgcc-4.7.1/afc21dc/lib -L$TOOLCHAIN_LIB_BASE/glibc/glibc-2.14.1/99df8fc/lib"
Add zlib to our builds and tweak histogram output Summary: $SUBJECT -- cosmetic fix for histograms, print P75/P99, and make sure zlib is enabled for our command line tools. Test Plan: compile, test db_bench with --compression_type=zlib Reviewers: heyongqiang Reviewed By: heyongqiang CC: adsharma, leveldb Differential Revision: https://reviews.facebook.net/D8445
2013-02-07 01:12:43 +00:00
EXEC_LDFLAGS_SHARED="$SNAPPY_LIBS $ZLIB_LIBS"
SNAPPY_LDFLAGS="$SNAPPY_LIBS $ZLIB_LIBS"
Compile leveldb with gcc 4.7.1 Test Plan: run unit tests Reviewers: heyongqiang Reviewed By: heyongqiang Differential Revision: https://reviews.facebook.net/D5163
2012-09-05 07:02:16 +00:00
Moving VALGRIND_VER which takes the valgrind version from third party to fbcode.gcc471.sh file Summary: the valgrind version being used is in facebook specific path and should be moved to the fbcode.gcc471.sh file instead of the makefile. The execution takes the environment's default valgrind version if the fbcode.gcc471.sh's valgrind_version is not available. Test Plan: make valgrind_check Reviewers: dhruba, sheki, akushner Reviewed By: dhruba Differential Revision: https://reviews.facebook.net/D9213
2013-03-07 19:11:30 +00:00
VALGRIND_VER="$TOOLCHAIN_LIB_BASE/valgrind/valgrind-3.8.1/91ddd43/bin/"
export CC CXX AR RANLIB CFLAGS EXEC_LDFLAGS EXEC_LDFLAGS_SHARED SNAPPY_LDFLAGS VALGRIND_VER