Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

notebook: add experimental Jupyter integration #1813

Closed
wants to merge 1 commit into from

Conversation

wchargin
Copy link
Contributor

@wchargin wchargin commented Feb 6, 2019

Summary:
This commit adds a module tensorboard.notebook that can be loaded as a
Jupyter extension, providing the %tensorboard cell magic to launch and
display TensorBoard instances within a notebook.

This code doesn’t have to be perfect right now; one main goal for this
PR is to sync the structure of this code into google3, where it’s easier
to work on and test Colab integration.

The UI looks like this:
Screenshot of a %tensorboard command in the Jupyter UI.

Test Plan:
Currently, this requires a bit of path hackery to get working:

  1. Run bazel build //tensorboard to build the main binary.
  2. Run which tensorboard to find the tensorboard binary provided by
    your virtualenv, and hit it with chmod -x to make it not
    executable.
  3. Run export PATH="$(readlink -e ./bazel-bin/tensorboard):$PATH" to
    add it to your path, and confirm that which tensorboard points to
    the built version rather than the version in your virtualenv.
  4. Run jupyter notebook to start the Jupyter server.
  5. Create a notebook and execute %load_ext tensorboard.notebook to
    load the extension; henceforth, %tensorboard should work until you
    restart the Jupyter kernel.

(Step (2) is necessary because the jupyter notebook runtime adds the
virtualenv back to the front of your PATH. An alternative is to
patch os.environ["PATH"] from within the Jupyter notebook.)

After setting it up as above, the following makes a good test plan
(assuming that you have no other TensorBoard instances running):

  • %tensorboard --logdir ~/tb/mnist --port 6006 (should launch)

  • %tensorboard --logdir ~/tb/mnist --port 6006 (should reuse)

  • %tensorboard --logdir ~/tb/images_demo --port 6006 (should fail)

  • %tensorboard --logdir ~/tb/images_demo --port 6007 (should launch)

  • %tensorboard --logdir ~/tb/mnist --port 6006 (should reuse Get bazel build tensorboard/components/... to work. #1)

  • multiple %tensorboards in a single cell:

    for i in ("images_demo", "audio_demo"):
        %tensorboard --logdir ~/tb/$i --port 0
  • from tensorboard import notebook

  • notebook.list() (should list four instances)

  • notebook.display(port=6006)

  • notebook.display(height=800)

Finally, if you skip (or revert) step (2) from the setup instructions,
you can see the timeout behavior, because we’ll invoke the tensorboard
provided by PyPI, which does not yet know how to write TensorboardInfo.

wchargin-branch: notebook-jupyter

@wchargin wchargin requested review from stephanwlee and manivaradarajan and removed request for stephanwlee and manivaradarajan February 6, 2019 20:31
@wchargin wchargin force-pushed the wchargin-notebook-jupyter branch 3 times, most recently from 2825bc3 to 98c5d04 Compare February 7, 2019 01:40
@wchargin wchargin changed the base branch from wchargin-manager-start to master February 7, 2019 02:29
@wchargin wchargin force-pushed the wchargin-notebook-jupyter branch 2 times, most recently from cda9875 to dde9c6d Compare February 7, 2019 02:32
tensorboard/notebook.py Outdated Show resolved Hide resolved
tensorboard/notebook.py Outdated Show resolved Hide resolved
tensorboard/notebook.py Show resolved Hide resolved
tensorboard/notebook.py Show resolved Hide resolved
tensorboard/notebook.py Show resolved Hide resolved
tensorboard/notebook.py Outdated Show resolved Hide resolved
tensorboard/notebook.py Show resolved Hide resolved
tensorboard/notebook.py Outdated Show resolved Hide resolved
Summary:
This commit adds a module `tensorboard.notebook` that can be loaded as a
Jupyter extension, providing the `%tensorboard` cell magic to launch and
display TensorBoard instances within a notebook.

This code doesn’t have to be perfect right now; one main goal for this
PR is to sync the structure of this code into google3, where it’s easier
to work on and test Colab integration.

The UI looks like this:
![Screenshot of a `%tensorboard` command in the Jupyter UI.][1]

[1]: https://user-images.githubusercontent.com/4317806/52386325-7ae7eb80-2a3a-11e9-93ab-fc9a689de51c.png

Test Plan:
Currently, this requires a bit of path hackery to get working:

 1. Run `bazel build //tensorboard` to build the main binary.
 2. Run `which tensorboard` to find the `tensorboard` binary provided by
    your virtualenv, and hit it with `chmod -x` to make it not
    executable.
 3. Run `export PATH="$(readlink -e ./bazel-bin/tensorboard):$PATH"` to
    add it to your path, and confirm that `which tensorboard` points to
    the built version rather than the version in your virtualenv.
 4. Run `jupyter notebook` to start the Jupyter server.
 5. Create a notebook and execute `%load_ext tensorboard.notebook` to
    load the extension; henceforth, `%tensorboard` should work until you
    restart the Jupyter kernel.

(Step (2) is necessary because the `jupyter notebook` runtime adds the
virtualenv _back_ to the front of your `PATH`. An alternative is to
patch `os.environ["PATH"]` from within the Jupyter notebook.)

After setting it up as above, the following makes a good test plan
(assuming that you have no other TensorBoard instances running):

  - `%tensorboard --logdir ~/tb/mnist --port 6006` (should launch)
  - `%tensorboard --logdir ~/tb/mnist --port 6006` (should reuse)
  - `%tensorboard --logdir ~/tb/images_demo --port 6006` (should fail)
  - `%tensorboard --logdir ~/tb/images_demo --port 6007` (should launch)
  - `%tensorboard --logdir ~/tb/mnist --port 6006` (should reuse #1)
  - multiple `%tensorboard`s in a single cell:

    ```py
    for i in ("images_demo", "audio_demo"):
        %tensorboard --logdir ~/tb/$i --port 0
    ```
  - `from tensorboard import notebook`
  - `notebook.list()` (should list four instances)
  - `notebook.display(port=6006)`
  - `notebook.display(height=800)`

Finally, if you skip (or revert) step (2) from the setup instructions,
you can see the timeout behavior, because we’ll invoke the `tensorboard`
provided by PyPI, which does not yet know how to write TensorboardInfo.

wchargin-branch: notebook-jupyter
@wchargin
Copy link
Contributor Author

wchargin commented Feb 7, 2019

Updated; PTAL.

tensorboard/notebook.py Show resolved Hide resolved
@wchargin
Copy link
Contributor Author

wchargin commented Feb 7, 2019

For my understanding, why wouldn't you specify the default in the arg?

def _display(port=None, height=600, print_message=False, display_handle=None):

Using None and replacing manually has the advantage that clients can
explicitly ask to use “the default value”. For instance, if you wanted
to implement a wrapper around display with some logic, you could do so
easily:

def display_pid(pid, height=None):
  """Display the TensorBoard instance owned by the given process."""
  port = look_up_port_by_pid(pid)
  display(port, height=height)

…but if display fixes a default and doesn’t allow you to pass None,
then you’re stuck with either* copying the default…

def display_pid(pid, height=600):
  # ...
  display(port, height=height)

(which is unfortunate if display changes its default value) or using
something like splat arguments:

def display_pid(pid, *args, **kwargs):
  port = look_up_port_by_pid(pid)
  display(port, *args, **kwargs)

…which is not what was wanted and has other downsides.

The x=None technique is used throughout TensorFlow, and seems pretty
good to me; I don’t really know of any appreciable downsides.

* It is technically possible to fetch the default value at runtime by
accessing display.func_defaults[0]… but I’ve never seen this actually
done in production code.

@wchargin wchargin removed the request for review from stephanwlee February 7, 2019 22:34
@wchargin
Copy link
Contributor Author

wchargin commented Feb 7, 2019

I merged this via “Squash and merge”, but the GitHub machine broke and
didn’t actually mark this PR as merged. Closing manually; for posterity,
this was merged in d56665b.

@wchargin wchargin closed this Feb 7, 2019
@wchargin wchargin deleted the wchargin-notebook-jupyter branch February 7, 2019 23:00
nfelt pushed a commit to nfelt/tensorboard that referenced this pull request Feb 8, 2019
Summary:
This commit adds a module `tensorboard.notebook` that can be loaded as a
Jupyter extension, providing the `%tensorboard` cell magic to launch and
display TensorBoard instances within a notebook.

This code doesn’t have to be perfect right now; one main goal for this
PR is to sync the structure of this code into google3, where it’s easier
to work on and test Colab integration.

The UI looks like this:
![Screenshot of a `%tensorboard` command in the Jupyter UI.][1]

[1]: https://user-images.githubusercontent.com/4317806/52386325-7ae7eb80-2a3a-11e9-93ab-fc9a689de51c.png

Test Plan:
Currently, this requires a bit of path hackery to get working:

 1. Run `bazel build //tensorboard` to build the main binary.
 2. Run `which tensorboard` to find the `tensorboard` binary provided by
    your virtualenv, and hit it with `chmod -x` to make it not
    executable.
 3. Run `export PATH="$(readlink -e ./bazel-bin/tensorboard):$PATH"` to
    add it to your path, and confirm that `which tensorboard` points to
    the built version rather than the version in your virtualenv.
 4. Run `jupyter notebook` to start the Jupyter server.
 5. Create a notebook and execute `%load_ext tensorboard.notebook` to
    load the extension; henceforth, `%tensorboard` should work until you
    restart the Jupyter kernel.

(Step (2) is necessary because the `jupyter notebook` runtime adds the
virtualenv _back_ to the front of your `PATH`. An alternative is to
patch `os.environ["PATH"]` from within the Jupyter notebook.)

After setting it up as above, the following makes a good test plan
(assuming that you have no other TensorBoard instances running):

  - `%tensorboard --logdir ~/tb/mnist --port 6006` (should launch)
  - `%tensorboard --logdir ~/tb/mnist --port 6006` (should reuse)
  - `%tensorboard --logdir ~/tb/images_demo --port 6006` (should fail)
  - `%tensorboard --logdir ~/tb/images_demo --port 6007` (should launch)
  - `%tensorboard --logdir ~/tb/mnist --port 6006` (should reuse #1)
  - multiple `%tensorboard`s in a single cell:

    ```py
    for i in ("images_demo", "audio_demo"):
        %tensorboard --logdir ~/tb/$i --port 0
    ```
  - `from tensorboard import notebook`
  - `notebook.list()` (should list four instances)
  - `notebook.display(port=6006)`
  - `notebook.display(height=800)`

Finally, if you skip (or revert) step (2) from the setup instructions,
you can see the timeout behavior, because we’ll invoke the `tensorboard`
provided by PyPI, which does not yet know how to write TensorboardInfo.

wchargin-branch: notebook-jupyter
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants