[NIO] Files.walkFileTree doesn't visit files

[tomwhite]

There are a couple of problems running this code for deleting a directory recursively: https://github.com/HadoopGenomics/Hadoop-BAM/blob/master/src/main/java/org/seqdoop/hadoop_bam/util/NIOFileUtil.java#L63-L74

1. The code for visitFile is never called.
2. Files.delete(dir) fails with "Can't perform I/O on pseudo-directories (trailing slash)"

It would be good to add tests for Files.walkFileTree (and Files.walk) to the unit tests.

[tomwhite]

#1630 addresses some of this, but there's still a remaining issue.

The following prints the files in a directory:

Path p = CloudStorageFileSystem.forBucket("bucket").getPath("dir/");
Files.list(p).forEach(System.out::println);

However, this doesn't work:

Path p = Paths.get(URI.create("gs://bucket/dir/"))
Files.list(p).forEach(System.out::println);

The problem is to do with the underlying path not having a the leading slash removed in the second case. This seems to fix the problem:

--- a/google-cloud-contrib/google-cloud-nio/src/main/java/com/google/cloud/storage/contrib/nio/CloudStorageFileSystemProvider.java
+++ b/google-cloud-contrib/google-cloud-nio/src/main/java/com/google/cloud/storage/contrib/nio/CloudStorageFileSystemProvider.java
@@ -608,7 +608,7 @@ public final class CloudStorageFileSystemProvider extends FileSystemProvider {
     final CloudStoragePath cloudPath = CloudStorageUtil.checkPath(dir);
     checkNotNull(filter);
     initStorage();
-    String prefix = cloudPath.toString();
+    String prefix = cloudPath.toRealPath().toString();
     final Iterator<Blob> blobIterator = storage.list(cloudPath.bucket(),
         Storage.BlobListOption.prefix(prefix), Storage.BlobListOption.currentDirectory(),
         Storage.BlobListOption.fields()).iterateAll();

[jean-philippe-martin]

Thank you @tomwhite for the clear bug report and fix! I put together PR #1644 with the fix.

[jean-philippe-martin]

#1644 was merged in so this bug should be fixed now.

[jean-philippe-martin]

Whoever has the permissions to do it, please close the bug. @tomwhite please reopen if the fix doesn't work for you.

[jean-philippe-martin]

Thank you!