We were canceling the context in `GarbageCollect` but some functions call `GC`
directly. Move the context cancelation down to where we actually _need_ it.

fixes #6279

License: MIT
Signed-off-by: Steven Allen <steven@stebalien.com>