Introduce buzhash chunker

It has the same properties as Rabin but is much faster.
Benchmark results:

```
name             speed
Buzhash2/1K-4    1.71GB/s ± 4%
Buzhash2/1M-4    1.15GB/s ± 2%
Buzhash2/16M-4    892MB/s ± 1%
Buzhash2/100M-4   904MB/s ± 2%
Rabin/1K-4       13.8MB/s ± 3%
Rabin/1M-4        171MB/s ± 3%
Rabin/16M-4       182MB/s ± 4%
Rabin/100M-4      182MB/s ± 3%
Default/1K-4     1.74GB/s ± 4%
Default/1M-4     3.22GB/s ± 2%
Default/16M-4    3.88GB/s ± 2%
Default/100M-4   4.21GB/s ± 6%
```

License: MIT
Signed-off-by: 's avatarJakub Sztandera <kubuxu@protocol.ai>
上级 b15edf28
......@@ -28,7 +28,7 @@ require (
github.com/ipfs/go-filestore v0.0.2
github.com/ipfs/go-fs-lock v0.0.1
github.com/ipfs/go-ipfs-blockstore v0.1.0
github.com/ipfs/go-ipfs-chunker v0.0.1
github.com/ipfs/go-ipfs-chunker v0.0.3
github.com/ipfs/go-ipfs-cmds v0.1.1
github.com/ipfs/go-ipfs-config v0.0.11
github.com/ipfs/go-ipfs-ds-help v0.0.1
......
......@@ -183,6 +183,8 @@ github.com/ipfs/go-ipfs-blocksutil v0.0.1 h1:Eh/H4pc1hsvhzsQoMEP3Bke/aW5P5rVM1IW
github.com/ipfs/go-ipfs-blocksutil v0.0.1/go.mod h1:Yq4M86uIOmxmGPUHv/uI7uKqZNtLb449gwKqXjIsnRk=
github.com/ipfs/go-ipfs-chunker v0.0.1 h1:cHUUxKFQ99pozdahi+uSC/3Y6HeRpi9oTeUHbE27SEw=
github.com/ipfs/go-ipfs-chunker v0.0.1/go.mod h1:tWewYK0we3+rMbOh7pPFGDyypCtvGcBFymgY4rSDLAw=
github.com/ipfs/go-ipfs-chunker v0.0.3 h1:UuXhKoxvxl/vGhie+WXFRZYCwpZjbKF2SzD1Tvxif1I=
github.com/ipfs/go-ipfs-chunker v0.0.3/go.mod h1:RkGJorerOQNTDPgmX7HtJ5YzVQqaIYdzI/hrCHty5Kc=
github.com/ipfs/go-ipfs-cmds v0.1.1 h1:H9/BLf5rcsULHMj/x8gC0e5o+raYhqk1OQsfzbGMNM4=
github.com/ipfs/go-ipfs-cmds v0.1.1/go.mod h1:k1zMXcOLtljA9iAnZHddbH69yVm5+weRL0snmMD/rK0=
github.com/ipfs/go-ipfs-config v0.0.11 h1:5/4nas2CQXiKr2/MLxU24GDGTBvtstQIQezuk7ltOQQ=
......
Markdown 格式
0%
您添加了 0 到此讨论。请谨慎行事。
请先完成此评论的编辑!
注册 或者 后发表评论