Commit graph

43 commits

Author SHA1 Message Date
Richard O'Grady 8325392950 Disable Wimplicit-int-float-conversion warning in googletest
PiperOrigin-RevId: 524031046
2023-04-13 10:04:53 -07:00
Richard O'Grady 00aa9ac61d Disable -Wsign-compare warning.
PiperOrigin-RevId: 523460180
2023-04-11 11:55:49 -07:00
Richard O'Grady cfc573e08f Define missing SNAPPY_PREFETCH macros.
PiperOrigin-RevId: 523287305
2023-04-11 10:38:23 -07:00
Victor Costan dc05e02648 Tag open source release 1.1.10.
PiperOrigin-RevId: 515161676
2023-03-08 15:44:00 -08:00
Snappy Team 8881ba172a Improve the speed of hashing in zippy compression.
This change replaces the hashing function used during compression with
one that is roughly as good but faster.  This speeds up compression by
two to a few percent on the Intel-, AMD-, and Arm-based machines we
tested.  The amount of compression is roughly unchanged.

PiperOrigin-RevId: 485960303
2023-01-12 13:32:54 +00:00
Jun He 734b32bfe3 Add config and header file for NEON support
Signed-off-by: Jun He <jun.he@arm.com>
Change-Id: I3fade568ff92b4303387705f843d0051d5e88349
2021-08-12 15:45:46 +08:00
Victor Costan 2b63814b15 Tag open source release 1.1.9.
PiperOrigin-RevId: 372007801
2021-05-04 22:53:34 +00:00
Victor Costan 5e7c14bd05 Add stubs for abseil flags.
This CL also removes support for using the gflags library to modify the
flags.

PiperOrigin-RevId: 361583626
2021-03-08 17:26:48 +00:00
Victor Costan 4ebd8b2f23 Split benchmarks and test tools into separate targets.
This lets us remove main() from snappy_bench.cc and snappy_unittest.cc,
which simplifies integrating these tests and benchmarks with other
suites.

PiperOrigin-RevId: 347857427
2020-12-16 19:09:56 +00:00
Victor Costan 6badb0a261 Merge pull request #114 from cmumford:werror-only-clang
PiperOrigin-RevId: 347660305
2020-12-15 19:49:13 +00:00
Chris Mumford bc53daa7be Fixed endif clause. 2020-12-15 11:19:53 -08:00
Chris Mumford e9a6a08439 Matching clang. 2020-12-15 11:17:28 -08:00
Chris Mumford 955a5dd1b3 Building with -Werror only with clang.
gcc was unable to inline a function call, which caused a build
failure due to `-Wall -Werror`.

The build error was:

```
../snappy.cc:292:76: error: ignoring attributes on template argument ‘__m128i’ [-Werror=ignored-attributes]
  292 | static inline std::pair<__m128i /* pattern */, __m128i /* reshuffle_mask */>
      |                                                                            ^
../snappy.cc:292:76: error: ignoring attributes on template argument ‘__m128i’ [-Werror=ignored-attributes]
cc1plus: all warnings being treated as errors
```
2020-12-15 11:02:17 -08:00
Victor Costan 549685a598 Remove custom testing and benchmarking code.
Snappy includes a testing framework, which implements a subset of the
Google Test API, and can be used when Google Test is not available.
Snappy also includes a micro-benchmark framework, which implements an
old version of the Google Benchmark API.

This CL replaces the custom test and micro-benchmark frameworks with
google/googletest and google/benchmark. The code is vendored in
third_party/ via git submodules. The setup is similar to google/crc32c
and google/leveldb.

This CL also updates the benchmarking code to the modern Google
Benchmark API.

Benchmark results are expected to be more precise, as the old framework
ran each benchmark with a fixed number of iterations, whereas Google
Benchmark keeps iterating until the noise is low.

PiperOrigin-RevId: 347456142
2020-12-14 21:27:31 +00:00
Snappy Team 01a566f825 Fix opensource version
PiperOrigin-RevId: 343272548
2020-11-19 17:06:26 +00:00
Snappy Team 616b8229b6 Add LZ4 as a benchmark option. Snappy is starting to look really good compared to LZ4. LZ4 is considered the fastest solution by many on internet. We now see that Snappy is actually becoming very competitive with compression a little faster and decompression slower but certainly not terribly slower.
PiperOrigin-RevId: 343140860
2020-11-18 23:22:04 +00:00
Victor Costan c98344f626 Fix Clang/GCC compilation warnings.
This makes it easier to adopt snappy in other projects.

PiperOrigin-RevId: 309958249
2020-05-05 16:15:02 +00:00
Victor Costan 27ff130ff9 Remove platform-dependent code for little-endian loads and stores.
The platform-independent code that breaks down the loads and stores into
byte-level operations is optimized into single instructions (mov or
ldr/str) and instruction pairs (mov+bswap or ldr/str+rev) by recent
versions of Clang and GCC. Tested at https://godbolt.org/z/2BQP-o

PiperOrigin-RevId: 306321608
2020-04-13 22:30:59 +00:00
Victor Costan a4cdb5d133 Introduce SNAPPY_ATTRIBUTE_ALWAYS_INLINE.
An internal CL started using ABSL_ATTRIBUTE_ALWAYS_INLINE
from Abseil. This CL introduces equivalent functionality as
SNAPPY_ALWAYS_INLINE.

PiperOrigin-RevId: 306289650
2020-04-13 19:51:05 +00:00
Victor Costan 537f4ad624 Tag open source release 1.1.8.
PiperOrigin-RevId: 289675084
2020-01-14 10:58:53 -08:00
Victor Costan 26410cc4f8 Merge pull request #85 from bitomaxsp:patch-1
PiperOrigin-RevId: 279633518
2019-11-10 14:10:50 -08:00
Victor Costan 0eec45ed16 Align CMake configuration with related projects.
PiperOrigin-RevId: 279237837
2019-11-07 22:39:04 -08:00
Victor Costan 6617df53fa Remove redundant PROJECT_SOURCE_DIR usage from CMake config.
Inspired by https://github.com/google/crc32c/pull/32

PiperOrigin-RevId: 278718367
2019-11-05 16:35:29 -08:00
Dmitry 38945971d6
Allow build with different standard if lib used as a subproject 2019-10-17 14:17:49 +02:00
Victor Costan e9e11b84e6 Fix Travis CI build.
* Fix bash conditionals: [ a == b ] should be [ a = b ].
* Upgrade to LLVM 9 on Travis.
* Upgrade fuzzer build arguments for LLVM 9.

PiperOrigin-RevId: 271898655
2019-09-29 20:39:28 -07:00
Victor Costan fd79e6f9b2 Merge pull request #78 from bshastry:libfuzzer-harness
PiperOrigin-RevId: 264241380
2019-08-19 14:30:13 -07:00
Bhargava Shastry d71375bf8a Add libFuzzer harnesses, a cmake option to build them 2019-07-12 14:42:48 +02:00
costan 4f0adca400 Wrap BMI2 instruction usage in support checks.
A previous version of this was submitted and rolled back due to breakage
-- an attempt to accommodate Visual Studio resulted in compiler errors
on GCC/Clang with -mavx2 but without -mbmi2. This version makes the BMI2
support check more strict, to avoid the errors.

A previous CL introduced _bzhi_u32 (part of Intel's BMI2 instruction
set, released in Haswell) gated by a check for the __BMI2__ preprocessor
macro. This works for Clang and GCC, but does not work on Visual Studio,
and may not work on other compilers.

This CL plumbs the BMI2 support checks through the CMake configuration
used by the open source build. It also replaces the <x86intrin.h>
header, which does not exist on Visual Studio, with the more scoped
headers <tmmintrin.h> (for SSSE3) and <immintrin.h> (for BMI2/AVX2).
Asides from fixing the open source build, the more scoped headers make
it slightly less likely that newer intrinsics will creep in without
proper gating.
2019-01-08 06:44:11 -08:00
costan 7fefd231a1 C++11 guarantees <cstddef> and <cstdint>.
The build configuration can be cleaned up a bit.
2018-08-16 11:36:45 -07:00
costan 73c31e824c Fix Visual Studio build.
Commit 8f469d97e2 introduced SSSE3 fast
paths that are gated by __SSE3__ macro checks and the <x86intrin.h>
header, neither of which exists in Visual Studio. This commit adds logic
for detecting SSSE3 compiler support that works for all compilers
supported by the open source release.

The commit also replaces the header with <tmmintrin.h>, which only
defines intrinsics supported by SSSE3 and below. This should help flag
any use of SIMD instructions that require more advanced SSE support, so
the uses can be gated by checks that also work in the open source
release.

Last, this commit requires C++11 support for the open source build. This is
needed by the alignas specifier, which was also introduced in commit
8f469d97e2.
2018-08-08 22:25:14 -07:00
costan b02bfa754e Tag open source release 1.1.7. 2017-08-24 16:54:23 -07:00
costan 55924d1109 Add GNUInstallDirs to CMake configuration.
This is modeled after https://github.com/google/googletest/pull/1160.
The immediate benefit is fixing the library install paths on 64-bit
Linux distributions, which tend to support running 32-bit and 64-bit
code side by side by installing 32-bit libraries in /usr/lib and  64-bit
libraries in /usr/lib64.
2017-08-16 19:19:31 -07:00
costan 632cd0f128 Use 64-bit optimized code path for ARM64.
This is inspired by https://github.com/google/snappy/pull/22.

Benchmark results with the change, Pixel C with Android N2G48B

Benchmark            Time(ns)    CPU(ns) Iterations
---------------------------------------------------
BM_UFlat/0             119544     119253       1501 818.9MB/s  html
BM_UFlat/1            1223950    1208588        163 554.0MB/s  urls
BM_UFlat/2              16081      15962      11527 7.2GB/s  jpg
BM_UFlat/3                356        352     416666 540.6MB/s  jpg_200
BM_UFlat/4              25010      24860       7683 3.8GB/s  pdf
BM_UFlat/5             484832     481572        407 811.1MB/s  html4
BM_UFlat/6             408410     408713        482 354.9MB/s  txt1
BM_UFlat/7             361714     361663        553 330.1MB/s  txt2
BM_UFlat/8            1090582    1087912        182 374.1MB/s  txt3
BM_UFlat/9            1503127    1503759        133 305.6MB/s  txt4
BM_UFlat/10            114183     114285       1715 989.6MB/s  pb
BM_UFlat/11            406714     407331        491 431.5MB/s  gaviota
BM_UIOVec/0            370397     369888        538 264.0MB/s  html
BM_UIOVec/1           3207510    3190000        100 209.9MB/s  urls
BM_UIOVec/2             16589      16573      11223 6.9GB/s  jpg
BM_UIOVec/3              1052       1052     165289 181.2MB/s  jpg_200
BM_UIOVec/4             49151      49184       3985 1.9GB/s  pdf
BM_UValidate/0          68115      68095       2893 1.4GB/s  html
BM_UValidate/1         792652     792000        250 845.4MB/s  urls
BM_UValidate/2            334        334     487804 343.1GB/s  jpg
BM_UValidate/3            235        235     666666 809.9MB/s  jpg_200
BM_UValidate/4           6126       6130      32626 15.6GB/s  pdf
BM_ZFlat/0             292697     290560        678 336.1MB/s  html (22.31 %)
BM_ZFlat/1            4062080    4050000        100 165.3MB/s  urls (47.78 %)
BM_ZFlat/2              29225      29274       6422 3.9GB/s  jpg (99.95 %)
BM_ZFlat/3               1099       1098     163934 173.7MB/s  jpg_200 (73.00 %)
BM_ZFlat/4              44117      44233       4205 2.2GB/s  pdf (83.30 %)
BM_ZFlat/5            1158058    1157894        171 337.4MB/s  html4 (22.52 %)
BM_ZFlat/6            1102983    1093922        181 132.6MB/s  txt1 (57.88 %)
BM_ZFlat/7             974142     975490        204 122.4MB/s  txt2 (61.91 %)
BM_ZFlat/8            2984670    2990000        100 136.1MB/s  txt3 (54.99 %)
BM_ZFlat/9            4100130    4090000        100 112.4MB/s  txt4 (66.26 %)
BM_ZFlat/10            276236     275139        716 411.0MB/s  pb (19.68 %)
BM_ZFlat/11            760091     759541        262 231.4MB/s  gaviota (37.72 %)

Baseline benchmark results, Pixel C with Android N2G48B

Benchmark            Time(ns)    CPU(ns) Iterations
---------------------------------------------------
BM_UFlat/0             148957     147565       1335 661.8MB/s  html
BM_UFlat/1            1527257    1500000        132 446.4MB/s  urls
BM_UFlat/2              19589      19397       8764 5.9GB/s  jpg
BM_UFlat/3                425        418     408163 455.3MB/s  jpg_200
BM_UFlat/4              30096      29552       6497 3.2GB/s  pdf
BM_UFlat/5             595933     594594        333 657.0MB/s  html4
BM_UFlat/6             516315     514360        383 282.0MB/s  txt1
BM_UFlat/7             454653     453514        441 263.2MB/s  txt2
BM_UFlat/8            1382687    1361111        144 299.0MB/s  txt3
BM_UFlat/9            1967590    1904761        105 241.3MB/s  txt4
BM_UFlat/10            148271     144560       1342 782.3MB/s  pb
BM_UFlat/11            523997     510471        382 344.4MB/s  gaviota
BM_UIOVec/0            478443     465227        417 209.9MB/s  html
BM_UIOVec/1           4172860    4060000        100 164.9MB/s  urls
BM_UIOVec/2             21470      20975       7342 5.5GB/s  jpg
BM_UIOVec/3              1357       1330      75187 143.4MB/s  jpg_200
BM_UIOVec/4             63143      61365       3031 1.6GB/s  pdf
BM_UValidate/0          86910      85125       2279 1.1GB/s  html
BM_UValidate/1        1022256    1000000        195 669.6MB/s  urls
BM_UValidate/2            420        417     400000 274.6GB/s  jpg
BM_UValidate/3            311        302     571428 630.0MB/s  jpg_200
BM_UValidate/4           7778       7584      25445 12.6GB/s  pdf
BM_ZFlat/0             469209     457547        424 213.4MB/s  html (22.31 %)
BM_ZFlat/1            5633510    5460000        100 122.6MB/s  urls (47.78 %)
BM_ZFlat/2              37896      36693       4524 3.1GB/s  jpg (99.95 %)
BM_ZFlat/3               1485       1441     123456 132.3MB/s  jpg_200 (73.00 %)
BM_ZFlat/4              74870      72775       2652 1.3GB/s  pdf (83.30 %)
BM_ZFlat/5            1857321    1785714        112 218.8MB/s  html4 (22.52 %)
BM_ZFlat/6            1538723    1492307        130 97.2MB/s  txt1 (57.88 %)
BM_ZFlat/7            1338236    1310810        148 91.1MB/s  txt2 (61.91 %)
BM_ZFlat/8            4050820    4040000        100 100.7MB/s  txt3 (54.99 %)
BM_ZFlat/9            5234940    5230000        100 87.9MB/s  txt4 (66.26 %)
BM_ZFlat/10            400309     400000        495 282.7MB/s  pb (19.68 %)
BM_ZFlat/11           1063042    1058510        188 166.1MB/s  gaviota (37.72 %)
2017-08-16 19:18:22 -07:00
costan 77c12adc19 Add unistd.h checks back to the CMake build.
getpagesize(), as well as its POSIX.2001 replacement
sysconf(_SC_PAGESIZE), is defined in <unistd.h>. On Linux and OS X,
including <sys/mman.h> is sufficient to get a definition for
getpagesize(). However, this is not true for the Android NDK. This CL
brings back the HAVE_UNISTD_H definition and its associated header
check.

This also adds a HAVE_FUNC_SYSCONF definition, which checks for the
presence of sysconf(). The definition can be used later to replace
getpagesize() with sysconf().
2017-08-02 10:56:06 -07:00
costan be6dc3db83 Redo CMake configuration.
The style was changed to match the official manual [1], the install
configuration was simplified and now matches the official packaging
guide [2], and the config files use the CMake-specific variable syntax
${VAR} instead of the autoconf-compatible syntax @VAR@, as documented in
[3]. The public header files are declared as such (for CMake 3.3+), and
the generated headers are included in the library target definition.

The tests are only built if SNAPPY_BUILD_TESTS (default ON) is true, so
zippy can be easily used in projects that add_subdirectory() its source
code directly, instead of using find_package().

[1] https://cmake.org/cmake/help/git-master/manual/cmake-language.7.html
[2] https://cmake.org/cmake/help/git-master/manual/cmake-packages.7.html
[3] https://cmake.org/cmake/help/git-master/command/configure_file.html
2017-07-28 10:14:21 -07:00
costan c756f7f5d9 Support both static and shared library CMake builds.
This can be used to fix https://github.com/Homebrew/homebrew-core/issues/15722.
2017-07-27 16:46:54 -07:00
costan 27671c6aec Clean up CMake header and type checks.
Unused macros: HAVE_DLFCN_H, HAVE_INTTYPES_H, HAVE_MEMORY_H,
HAVE_STDLIB_H, HAVE_STRINGS_H, HAVE_STRING_H, HAVE_SYS_BYTESWAP_H,
HAVE_SYS_STAT_H, HAVE_SYS_TYPES_H, HAVE_UNISTD_H.

Used but never set macros: HAVE_LIBLZF, HAVE_LIBQUICKLZ. These only gate
conditional includes. The code that takes advantage of them was removed.

Unused types: ssize_t.

The testing code uses HAVE_FUNC_MMAP, which was not wired in the CMake
build, causing a whole test to be skipped.
2017-07-25 18:17:35 -07:00
costan 548501c988 zippy: Re-release snappy 1.1.5 as 1.1.6.
The migration from autotools to CMake in 1.1.5 wasn't as smooth as
intended. The SONAME / SOVERSION were broken in both build systems,
causing breakages in systems that upgraded from snappy 1.1.4 to 1.1.5,
as reported in https://github.com/Homebrew/homebrew-core/issues/15274
and https://github.com/google/snappy/pull/45.
2017-07-13 03:56:49 -07:00
costan 513df5fb5a Tag open source release 1.1.5. 2017-06-28 18:37:30 -07:00
costan 5bc9c82ae3 Set minimum CMake version to 3.1.
The project only needs CMake 3.1 features, and some Travis CI bots have
CMake 3.2.2. Therefore, requiring CMake 3.4 is inconvenient.
2017-06-28 18:37:08 -07:00
ysaed 82deffcde7 Remove benchmarking support for fastlz. 2017-06-28 18:33:55 -07:00
jyrki 83179dd8be Remove quicklz and lzf support in benchmarks. 2017-06-05 13:54:10 -07:00
vrabaud c8131680d0 Provide a CMakeLists.txt.
This lands https://github.com/google/snappy/pull/29
2017-06-05 13:53:29 -07:00