POC of base tests for fsspec behavior.

[TomAugspurger]

xref #650

[TomAugspurger]

Interestingly, MemoryFileSystem is failing:

fsspec/tests/spec/test_memory.py::TestRead::test_ls_raises_filenotfound FAILED                                                                           [100%]

=========================================================================== FAILURES ===========================================================================
_____________________________________________________________ TestRead.test_ls_raises_filenotfound _____________________________________________________________

self = <test_memory.TestRead object at 0x7f3a3edfb760>, fs = <fsspec.implementations.memory.MemoryFileSystem object at 0x7f3a3ee0d340>, prefix = '/root/'

    def test_ls_raises_filenotfound(self, fs, prefix):
        with pytest.raises(FileNotFoundError):
>           fs.ls(f"{prefix}/not-a-key")
E           Failed: DID NOT RAISE <class 'FileNotFoundError'>

fsspec/tests/base/spec.py:25: Failed
=================================================================== short test summary info ====================================================================
FAILED fsspec/tests/spec/test_memory.py::TestRead::test_ls_raises_filenotfound - Failed: DID NOT RAISE <class 'FileNotFoundError'>

Is this deliberate?

In [1]: import fsspec

In [2]: fs = fsspec.get_filesystem_class("memory")()

In [3]: fs.ls("/a")
Out[3]: []

[martindurant]

Yep, MemoryFileSystem should probably raise. It has "pseudo directories" explicitly holding references that don't contain anything (i.e., empty directories), so we should be able to tell the difference between non-existent and empty directory.

[martindurant]

fsspec.implementations.memory.MemoryFileSystem.ls is pretty ugly!

[TomAugspurger]

OK, I've xfailed that MemoryFileSystem test for now: #652.

Some general thoughts / questions:

1. Are we OK with the naming BaseSpecTests, tests/base/spec.py?
2. Are we OK with the layout of tests/base/spec.py defining the base tests, and implementations/tests/spec/test*.py for each implementation?
3. We should consider parametrizing fs by some common arguments to AbstractFileSystem, including use_listings_cache and asynchronous.

cc @isidentical from #650 if you have thoughts on the implementation. It is a bit awkward to have this prefix fixture that has to be defined. AFAICT, I needed that for HTTPFileSystem to work with this setup since its requests like ls include the full URL, rather than the URL relative to some root.