The change to an array of readers comes from e096060b
(refactor(core/commands2/add) split loop, 2014-11-06), where it's used
to setup readers for each path in the argument list.  However, since
6faeee83 (cmds2/add: temp fix for -r. horrible hack, 2014-11-11) the
argument looping moved outside of add() and into Run(), so we can drop
the multiple-reader support from add().

Adding a file can create multiple nodes (e.g. the splitter can chunk
the file into several blocks), but:

1. we were only appending a single node per reader to our returned
   list, and
2. we are only using the final node in that returned list,

so this commit also adjusts add() to return a single node reference
instead on an array of nodes.

This patch update fixes a few bugs:

* harden shutdown logic by @torarnv
* daemon locking fixes by @travisperson
* don't re-add entire dirs by @whyrusleeping
* tests now wait for graceful shutdown by @jbenet
* default key size is now 2048 by @jbenet

(experimenting with using semver)

init: lower default rsa key size to 2048 for now

sharness: wait for graceful shutdown

I think we should lower the default rsa key size to 2048 for now -- until we have a proper focus on securing everything. It's always a pain for new users to get hung on 4096 rsa key gen, when we have not even made sure we're using the keys perfectly correctly yet. (And 2048 is still considered secure)

sharness should only send the kill signal once, as that is
what a graceful shutdown should do. in the event that doesn't
happen, we should send it again, and then kill -9 to prevent it
lingering and messing with other tests.

The test_must_fail check fails on osx. (it does not seem to fail on
linux). See #1109

Harden shutdown logic

Check to see if the daemon is currently running

At this point I think this test should be moved out of this file

don't readd entire directories recursively

This test to make sure we can't run `ipfs init` while a daemon is
currently running.

If the daemon is running we do not want to proceed with an
an initialization.

Return a client error telling the user to kill the daemon
before proceeding with the command.

Instead of just terminating right there and then, we cancel the
context, and let the daemon exit cleanly. This make take a few
seconds, as the node builder and its child processes do not
care too much about the context state while building nodes,
but this can be improved by injecting checks for ctx.Done()
before time-consuming steps.

Instead of assuming the command is the daemon command and closing
the node, which resulted in bugs like #1053, we cancel the context
and let the context children detect the cancellation and gracefully
clean up after themselves.

The shutdown logging has been moved into the daemon command, where
it makes more sense, so that commands like ping will not print out
the same output on cancellation.

When the response includes the X-Chunked-Output header, we treat that
as channel output, and fire up a goroutine to decode the chunks. This
routine need to look for context cancellation so that it can exit
cleanly.

The context may be cancelled while a request is in flight. We need to
handle this and cancel the request. The code is based on the ideas
from https://blog.golang.org/context

The context passed on from main() may change before we hit callCommand,
so setting it in Parse is a bit optimistic.

If a command invocation such as 'daemon' is interrupted, the interrupt
handler asks the node to close. The closing of the node will result in
the command invocation finishing, and possibly returning from main()
before the interrupt handler is done. In particular, the info logging
that a graceful shutdown was completed may never reach reach stdout.

As the whole point of logging "Gracefully shut down." is to give
confidence when debugging that the shutdown was clean, this is
slightly unfortunate.

The interrupt handler needs to be set up in main() instead of Run(),
so that we can defer the closing of the interrupt handler until just
before returning from main, not when Run() returns with a streaming
result reader.

core: bugfix: bootstrap random permutation

the random permutaton for bootstrap peers was not working as
intended, returning the first four bootstrap peers always.
this commit fixes it to be a random subset.

Once the server is asked to shut down, we stop accepting new
connections, but the 'manners' graceful shutdown will wait for
all existing connections closed to close before finishing.

For keep-alive connections this will never happen unless the
client detects that the server is shutting down through the
ipfs API itself, and closes the connection in response.

This is a problem e.g. with the webui's connections visualization,
which polls the swarm/peers endpoint once a second, and never
detects that the API server was shut down.

We can mitigate this by telling the server to disable keep-alive,
which will add a 'Connection: close' header to the next HTTP
response on the connection. A well behaving client should then
treat that correspondingly by closing the connection.

Unfortunately this doesn't happen immediately in all cases,
presumably depending on the keep-alive timeout of the browser
that set up the connection, but it's at least a step in the
right direction.

When closing a node, the node itself only takes care of tearing down
its own children. As corehttp sets up a server based on a node, it
needs to also ensure that the server is accounted for when determining
if the node has been fully closed.

The server may stay alive for quite a while due to waiting on
open connections to close before shutting down. We should
find ways to terminate these connections in a more controlled
manner, but in the meantime it's helpful to be able to see
why a shutdown of the ipfs daemon is taking so long.

This reverts commit f74e71f9.

The 'Online' flag of the command context does not seem to be set in
any code paths, at least not when running commands such as 'ipfs daemon'
or 'ipfs ping'. The result after f74e71f9 is that we never shutdown
cleanly, as we'll always os.Exit(0) from the interrupt handler.

The os.Exit(0) itself is also dubious, as conceptually the interrupt
handler should ask whatever is stalling to stop stalling, so that
main() can return like normal. Exiting with -1 in error cases where
the interrupt handler is unable to stop the stall is fine, but the
normal case of interrupting cleanly should exit through main().

This commit includes a poor attempt at a changelog.

fsrepo migrations

The "daemon.lock" was really a repo.lock, as the cli also took it
and the purpose was any process mutex. This is part of the 1-to-2
migration, and has already been handled in
https://github.com/ipfs/fs-repo-migrations/tree/master/ipfs-1-to-2

Improved the repo migration errors to provide instructions
to the user.

This changes .go-ipfs to .ipfs everywhere.
And by the way this defines a DefaultPathName const
for this name.

License: MIT
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>

WARNING: No migration performed! That needs to come in a separate
commit, perhaps amended into this one.

Migration must move keyspace "/b" from leveldb to the flatfs subdir,
while removing the "b" prefix (keys should start with just "/").