* cli: initial work on debug; server-status target * debug: add metrics capture target (#7376) * check against DR secondary * debug: add compression * refactor check into preflight func * debug: set short test time on tests, fix exit code bug * debug: use temp dir for output on tests * debug: use mholt/archiver for compression * first pass on adding pprof * use logger for output * refactor polling target capture logic * debug: poll and collect replication status * debug: poll and collect host-info; rename output files and collection refactor * fix comments * add archive test; fix bugs found * rename flag name to singular target * add target output test; scaffold other tests cases * debug/test: add pprof and index file tests * debug/test: add min timing check tests * debug: fix index gen race and collection goroutine race * debug: extend archive tests, handle race between program exit and polling goroutines * update docstring * debug: correctly add to pollingWg * debug: add config target support * debug: don't wait on interrupt shutdown; add file exists unit tests * move pprof bits into its goroutine * debug: skip empty metrics and some pprof file creation if permission denied, add matching unit test * address comments and feedback * Vault debug using run.Group (#7658) * debug: switch to use oklog/run.Group * debug: use context to cancel requests and interrupt rungroups. * debug: trigger the first interval properly * debug: metrics collection should use metrics interval * debug: add missing continue on metrics error * debug: remove the use of buffered chan to trigger first interval * debug: don't shadow BaseCommand's client, properly block on interval capture failures * debug: actually use c.cachedClient everywhere * go mod vendor * debug: run all pprof in goroutines; bump pprof timings in tests to reduce flakiness * debug: update help text
4.3 KiB
Collection of compression libraries for Go
Introduction
NOTE: This library is in active development. As such, there are no guarantees about the stability of the API. The author reserves the right to arbitrarily break the API for any reason.
This repository hosts a collection of compression related libraries. The goal of this project is to provide pure Go implementations for popular compression algorithms beyond what the Go standard library provides. The goals for these packages are as follows:
- Maintainable: That the code remains well documented, well tested, readable, easy to maintain, and easy to verify that it conforms to the specification for the format being implemented.
- Performant: To be able to compress and decompress within at least 80% of the rates that the C implementations are able to achieve.
- Flexible: That the code provides low-level and fine granularity control over the compression streams similar to what the C APIs would provide.
Of these three, the first objective is often at odds with the other two objectives and provides interesting challenges. Higher performance can often be achieved by muddling abstraction layers or using non-intuitive low-level primitives. Also, more features and functionality, while useful in some situations, often complicates the API. Thus, this package will attempt to satisfy all the goals, but will defer to favoring maintainability when the performance or flexibility benefits are not significant enough.
Library Status
For the packages available, only some features are currently implemented:
Package | Reader | Writer |
---|---|---|
brotli | ✅ | |
bzip2 | ✅ | ✅ |
flate | ✅ | |
xflate | ✅ | ✅ |
This library is in active development. As such, there are no guarantees about the stability of the API. The author reserves the right to arbitrarily break the API for any reason. When the library becomes more mature, it is planned to eventually conform to some strict versioning scheme like Semantic Versioning.
However, in the meanwhile, this library does provide some basic API guarantees. For the types defined below, the method signatures are guaranteed to not change. Note that the author still reserves the right to change the fields within each Reader
and Writer
structs.
type ReaderConfig struct { ... }
type Reader struct { ... }
func NewReader(io.Reader, *ReaderConfig) (*Reader, error) { ... }
func (*Reader) Read([]byte) (int, error) { ... }
func (*Reader) Close() error { ... }
type WriterConfig struct { ... }
type Writer struct { ... }
func NewWriter(io.Writer, *WriterConfig) (*Writer, error) { ... }
func (*Writer) Write([]byte) (int, error) { ... }
func (*Writer) Close() error { ... }
To see what work still remains, see the Task List.
Performance
See Performance Metrics.
Frequently Asked Questions
See Frequently Asked Questions.
Installation
Run the command:
go get -u github.com/dsnet/compress
This library requires Go1.9
or higher in order to build.
Packages
Package | Description |
---|---|
brotli | Package brotli implements the Brotli format, described in RFC 7932. |
bzip2 | Package bzip2 implements the BZip2 compressed data format. |
flate | Package flate implements the DEFLATE format, described in RFC 1951. |
xflate | Package xflate implements the XFLATE format, an random-access extension to DEFLATE. |