open-consul/vendor/github.com/hashicorp/go-msgpack/codec
Hans Hasselberg 0d8d7ae052
agent: transfer leadership when establishLeadership fails (#5247)
2019-06-19 14:50:48 +02:00
..
0doc.go Manage dependencies via Godep 2016-02-12 16:50:37 -08:00
README.md Manage dependencies via Godep 2016-02-12 16:50:37 -08:00
binc.go Manage dependencies via Godep 2016-02-12 16:50:37 -08:00
decode.go Update go-msgpack version (#5683) 2019-04-18 15:10:34 -04:00
encode.go Manage dependencies via Godep 2016-02-12 16:50:37 -08:00
helper.go agent: transfer leadership when establishLeadership fails (#5247) 2019-06-19 14:50:48 +02:00
helper_internal.go agent: transfer leadership when establishLeadership fails (#5247) 2019-06-19 14:50:48 +02:00
msgpack.go Manage dependencies via Godep 2016-02-12 16:50:37 -08:00
msgpack_test.py Update vendoring from go mod. (#5566) 2019-03-26 17:50:42 -04:00
rpc.go Manage dependencies via Godep 2016-02-12 16:50:37 -08:00
simple.go Manage dependencies via Godep 2016-02-12 16:50:37 -08:00
time.go Manage dependencies via Godep 2016-02-12 16:50:37 -08:00

README.md

Codec

High Performance and Feature-Rich Idiomatic Go Library providing encode/decode support for different serialization formats.

Supported Serialization formats are:

To install:

go get github.com/ugorji/go/codec

Online documentation: [http://godoc.org/github.com/ugorji/go/codec]

The idiomatic Go support is as seen in other encoding packages in the standard library (ie json, xml, gob, etc).

Rich Feature Set includes:

  • Simple but extremely powerful and feature-rich API
  • Very High Performance.
    Our extensive benchmarks show us outperforming Gob, Json and Bson by 2-4X. This was achieved by taking extreme care on:
    • managing allocation
    • function frame size (important due to Go's use of split stacks),
    • reflection use (and by-passing reflection for common types)
    • recursion implications
    • zero-copy mode (encoding/decoding to byte slice without using temp buffers)
  • Correct.
    Care was taken to precisely handle corner cases like: overflows, nil maps and slices, nil value in stream, etc.
  • Efficient zero-copying into temporary byte buffers
    when encoding into or decoding from a byte slice.
  • Standard field renaming via tags
  • Encoding from any value
    (struct, slice, map, primitives, pointers, interface{}, etc)
  • Decoding into pointer to any non-nil typed value
    (struct, slice, map, int, float32, bool, string, reflect.Value, etc)
  • Supports extension functions to handle the encode/decode of custom types
  • Support Go 1.2 encoding.BinaryMarshaler/BinaryUnmarshaler
  • Schema-less decoding
    (decode into a pointer to a nil interface{} as opposed to a typed non-nil value).
    Includes Options to configure what specific map or slice type to use when decoding an encoded list or map into a nil interface{}
  • Provides a RPC Server and Client Codec for net/rpc communication protocol.
  • Msgpack Specific:
    • Provides extension functions to handle spec-defined extensions (binary, timestamp)
    • Options to resolve ambiguities in handling raw bytes (as string or []byte)
      during schema-less decoding (decoding into a nil interface{})
    • RPC Server/Client Codec for msgpack-rpc protocol defined at: https://github.com/msgpack-rpc/msgpack-rpc/blob/master/spec.md
  • Fast Paths for some container types:
    For some container types, we circumvent reflection and its associated overhead and allocation costs, and encode/decode directly. These types are:
    []interface{} []int []string map[interface{}]interface{} map[int]interface{} map[string]interface{}

Extension Support

Users can register a function to handle the encoding or decoding of their custom types.

There are no restrictions on what the custom type can be. Some examples:

type BisSet   []int
type BitSet64 uint64
type UUID     string
type MyStructWithUnexportedFields struct { a int; b bool; c []int; }
type GifImage struct { ... }

As an illustration, MyStructWithUnexportedFields would normally be encoded as an empty map because it has no exported fields, while UUID would be encoded as a string. However, with extension support, you can encode any of these however you like.

RPC

RPC Client and Server Codecs are implemented, so the codecs can be used with the standard net/rpc package.

Usage

Typical usage model:

// create and configure Handle
var (
  bh codec.BincHandle
  mh codec.MsgpackHandle
)

mh.MapType = reflect.TypeOf(map[string]interface{}(nil))

// configure extensions
// e.g. for msgpack, define functions and enable Time support for tag 1
// mh.AddExt(reflect.TypeOf(time.Time{}), 1, myMsgpackTimeEncodeExtFn, myMsgpackTimeDecodeExtFn)

// create and use decoder/encoder
var (
  r io.Reader
  w io.Writer
  b []byte
  h = &bh // or mh to use msgpack
)

dec = codec.NewDecoder(r, h)
dec = codec.NewDecoderBytes(b, h)
err = dec.Decode(&v) 

enc = codec.NewEncoder(w, h)
enc = codec.NewEncoderBytes(&b, h)
err = enc.Encode(v)

//RPC Server
go func() {
    for {
        conn, err := listener.Accept()
        rpcCodec := codec.GoRpc.ServerCodec(conn, h)
        //OR rpcCodec := codec.MsgpackSpecRpc.ServerCodec(conn, h)
        rpc.ServeCodec(rpcCodec)
    }
}()

//RPC Communication (client side)
conn, err = net.Dial("tcp", "localhost:5555")
rpcCodec := codec.GoRpc.ClientCodec(conn, h)
//OR rpcCodec := codec.MsgpackSpecRpc.ClientCodec(conn, h)
client := rpc.NewClientWithCodec(rpcCodec)

Representative Benchmark Results

A sample run of benchmark using "go test -bi -bench=. -benchmem":

/proc/cpuinfo: Intel(R) Core(TM) i7-2630QM CPU @ 2.00GHz (HT)

..............................................
BENCHMARK INIT: 2013-10-16 11:02:50.345970786 -0400 EDT
To run full benchmark comparing encodings (MsgPack, Binc, JSON, GOB, etc), use: "go test -bench=."
Benchmark: 
	Struct recursive Depth:             1
	ApproxDeepSize Of benchmark Struct: 4694 bytes
Benchmark One-Pass Run:
	 v-msgpack: len: 1600 bytes
	      bson: len: 3025 bytes
	   msgpack: len: 1560 bytes
	      binc: len: 1187 bytes
	       gob: len: 1972 bytes
	      json: len: 2538 bytes
..............................................
PASS
Benchmark__Msgpack____Encode	   50000	     54359 ns/op	   14953 B/op	      83 allocs/op
Benchmark__Msgpack____Decode	   10000	    106531 ns/op	   14990 B/op	     410 allocs/op
Benchmark__Binc_NoSym_Encode	   50000	     53956 ns/op	   14966 B/op	      83 allocs/op
Benchmark__Binc_NoSym_Decode	   10000	    103751 ns/op	   14529 B/op	     386 allocs/op
Benchmark__Binc_Sym___Encode	   50000	     65961 ns/op	   17130 B/op	      88 allocs/op
Benchmark__Binc_Sym___Decode	   10000	    106310 ns/op	   15857 B/op	     287 allocs/op
Benchmark__Gob________Encode	   10000	    135944 ns/op	   21189 B/op	     237 allocs/op
Benchmark__Gob________Decode	    5000	    405390 ns/op	   83460 B/op	    1841 allocs/op
Benchmark__Json_______Encode	   20000	     79412 ns/op	   13874 B/op	     102 allocs/op
Benchmark__Json_______Decode	   10000	    247979 ns/op	   14202 B/op	     493 allocs/op
Benchmark__Bson_______Encode	   10000	    121762 ns/op	   27814 B/op	     514 allocs/op
Benchmark__Bson_______Decode	   10000	    162126 ns/op	   16514 B/op	     789 allocs/op
Benchmark__VMsgpack___Encode	   50000	     69155 ns/op	   12370 B/op	     344 allocs/op
Benchmark__VMsgpack___Decode	   10000	    151609 ns/op	   20307 B/op	     571 allocs/op
ok  	ugorji.net/codec	30.827s

To run full benchmark suite (including against vmsgpack and bson), see notes in ext_dep_test.go