updated goprocess to deal with memory leak

bugfix: add was not setting error

corehttp: added support for HEAD requests

This commit adds HEAD support to the IPFS Gateway.
Related: #840

reuseport: env var to turn it off

reuseport is a hack. It is necessary for us to do certain kinds of
tcp nat traversal. Ideally, reuseport would be available in go:

  https://github.com/golang/go/issues/9661

But until that issue is fixed, we're stuck with this. In some cases,
reuseport is strictly a detriment: nodes are not NATed. This commit
introduces an ENV var IPFS_REUSEPORT that can be set to false to
avoid using reuseport entirely:

  IPFS_REUSEPORT=false ipfs daemon

This approach addresses our current need. It could become a config
var if necessary. If reuseport continues to give problems, we should
look into improving it.

Fix various issues in the test makefiles

GNU Make's wildcard function does not recurse into subdirectories when
passed the '**' glob, which results in adding a dependency only to .go
files in the first level of subdirectories under the source root.

We shell out to 'find' instead, which catches all .go files in the
given directory.

Running make -jN would result in the tests starting to execute
before the tests binaries were built, resulting in the error:

 "Cannot find the tests' local ipfs tool"

Each test now depends on the deps. They also depend on a new
target for cleaning the test results, so that the tests can
write new clean results.

The aggregate target also needs to depend on the same test
results clean target, as well as the tests themselves, so
that the aggregation happens when all tests have finished
running.

By introducing a separate target for cleaning test results we
also ensure that we don't end up removing and rebuilding
the binary on each test run.

The result is that the tests *can* be run with with -jN > 1,
but individual tests may still not supports this, so to get
stable test results it's still recommended to run them in
sequence.

If the file doesn't exist, make will conclude that the missing
prerequisite should trigger a rebuild of the binaries.

Make bloom filters simpler

These did not work before, and had some unnecessary complexity.

Now the filters use only one hashing function, no bignum arithmetic, and gets the additional bit positions by repeatedly hashing the result of prior hash.

Since we're not concerned about crypto hashing here, this should be a win.

External interfaces unchanged.

Unify shutdown message format string

ipfs-completion.bash: add the missing get command

Extend logging when peer handshake detects clashing/same keys

Run tests with race flag using checkflags script

refactor task queue to have queues per peer

Add benchmark for just the time it takes to cat

fuse.md: Added troubleshooting instructions for #813

The GOFLAGS variable makes it possible to run all sharness
tests with go binaries built with some special flags.

The "race" target makes it easy run the sharness tests
with go binaries built with the -race flag.

License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>

...and remove bin/* stuff from the .gitignore
as /test/bin is already in the root .gitignore.

License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>

This builds go binaries using the -race flag
and then runs all the tests.

License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>

This makes it possible to build binaries with
different flags.

The content of the GOFLAGS variable is stored
in a IPFS-BUILD-OPTIONS file, so that if GOFLAGS
changes a rebuild of the binaries with the new
flags is forced.

License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>

This script can be used in a Makefile to detect flag changes
and to save the new flags in a file.

License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>

handle error from GetClosestPeers

Correct port in quick-start examples