Support of WebHDFS

[ziky90]

My idea is to add possibility to read from WebHDFS.
It would probably need to be implemented as some wrapper on top of WebHDFS API.
https://hadoop.apache.org/docs/r1.0.4/webhdfs.html

Based on the observations from API:

• Streaming seems to be possible
• It will involve probably requests library dependency (is this possible?)
• It might involve necessity of creating some config file with details about login to WebHDFS

Would this addition be appreciated in smart_open? I can try to work on PR, hopefully in the near future.

[piskvorky]

Yes, we accept any new functionality that fits the mission of smart_open = Pythonic API read/write access to large (streamed) data files, no matter the protocol.

requests is a stable library, so depending on it is no problem. I think boto depends on it, so it is a dependency already, anyway.

Authentication / login configs: fine as long as it doesn't affect the other (existing) code paths and is conceptually sane (maintainable).

[ziky90]

Implemented in #29 #31