Some methods of Experiment are not yet released while already referenced in docs

[phamour]

Environment:

• NNI version: 2.0
• NNI mode (local|remote|pai): local
• Client OS: Ubuntu
• Server OS (for remote mode only):
• Python version: 3.6|3.7
• PyTorch/TensorFlow version: torch 1.0|1.4|1.5|1.6
• Is conda/virtualenv/venv used?: conda
• Is running in Docker?: yes

Log message:

• nnimanager.log:
• dispatcher.log:
• nnictl stdout and stderr:
• custom stdout (wrapper using python launcher):

  Traceback (most recent call last):
    File "nni_wrapper_main.py", line 49, in <module>
      print(f"Jobs stats: {experiment.get_job_statistics()}")
  AttributeError: 'Experiment' object has no attribute 'get_job_statistics'
  

What issue meet, what's expected?:
Context:
I'm setting up a wrapper that calls NNI using the new launch-exp-from-Python feature released in v2.0. While checking the online official docs, I was glad to find some useful methods of Experiment class like get_job_statistics and get_job_metrics which are exactly what I needed to monitor the HPO process. I finally realized that they are not yet delivered in v2.0 when I saw the error above and then checked the PR #3396 merging them into master, which is after the released date of v2.0.
Questions:
Is it a little bit misleading to publish unreleased code in official docs ? And could you please tell me if you have a release planning for v2.1 (perhaps) that includes the aforementioned methods ? Thanks in advance.

How to reproduce it?:

Additional information:

[kvartet]

Thank you for trying our new features released in v2.0.

In fact, there are several versions of NNI's official documentation on ReadtheDocs, among which stable corresponds to the recent stable version (i.e. v2.0), and the latest is synchronized with the master branch so that we can debug internally during the development process. The document you are viewing should be the latest version, so you have encountered some problems when using NNI v2.0. We are sorry for this, and we will pay attention to not recommend reading the latest document elsewhere.

But the good news is that NNI will support Python API in v2.1, including the methods you mentioned. Please look forward to our updates, and we will release v2.1 soon!

[phamour]

Hi, thanks for the reply ! Indeed I was on the latest doc version, thanks for pointing that out, and yep I really look forward to that v2.1.