Implement TensorboardWriter #18

akainth015 · 2021-04-06T17:35:02Z

The TensorboardWriter can also log rich images and histograms

vissl/trainer/train_task.py

tests/test_configs.py

vissl/hooks/tensorboard_hook.py

prigoyal · 2021-04-19T17:12:37Z

vissl/hooks/tensorboard_hook.py

+                            #  tag=tag_name,
+                            #  scalar_value=round(acc, 5),
+                            #  global_step=task.train_phase_idx,
+                            # )


Remaining places:

https://github.com/MLH-Fellowship/vissl/blob/write_tensorboard_writer/vissl/hooks/tensorboard_hook.py#L197

https://github.com/MLH-Fellowship/vissl/blob/write_tensorboard_writer/vissl/hooks/tensorboard_hook.py#L207

https://github.com/MLH-Fellowship/vissl/blob/write_tensorboard_writer/vissl/hooks/tensorboard_hook.py#L248

https://github.com/MLH-Fellowship/vissl/blob/write_tensorboard_writer/vissl/hooks/tensorboard_hook.py#L264

https://github.com/MLH-Fellowship/vissl/blob/write_tensorboard_writer/vissl/hooks/tensorboard_hook.py#L270

https://github.com/MLH-Fellowship/vissl/blob/write_tensorboard_writer/vissl/hooks/tensorboard_hook.py#L283

https://github.com/MLH-Fellowship/vissl/blob/write_tensorboard_writer/vissl/hooks/tensorboard_hook.py#L298

https://github.com/MLH-Fellowship/vissl/blob/write_tensorboard_writer/vissl/hooks/tensorboard_hook.py#L307

https://github.com/MLH-Fellowship/vissl/blob/write_tensorboard_writer/vissl/hooks/tensorboard_hook.py#L316-L330

prigoyal · 2021-04-19T17:18:35Z

Follow-up steps:

Pull the VISSL main master and rebase on top
Address the only comment
Update the test plan in this PR (command you ran, ls tb_logs/)
CircleCI cpu_tests are green after the rebase
Merge this PR in MLH fork

The TensorboardWriter can also log rich images and histograms

Co-authored-by: Grace Omotoso <bgo2e@mtmail.mtsu.edu>

grace-omotoso · 2021-04-19T21:53:06Z

Follow-up steps:

Pull the VISSL main master and rebase on top

Address the only comment

Update the test plan in this PR (command you ran, ls tb_logs/)

CircleCI cpu_tests are green after the rebase

Merge this PR in MLH fork

Test
python tools/run_distributed_engines.py config=test/cpu_test/test_cpu_resnet_simclr config.DATA.TRAIN.DATA_SOURCES=[synthetic] config.HOOKS.TENSORBOARD_SETUP.USE_TENSORBOARD=true

TB Log files
events.out.tfevents.1618512637.DESKTOP-RFEIFUI.552.1 events.out.tfevents.1618628331.DESKTOP-RFEIFUI.3103.1
events.out.tfevents.1618513139.DESKTOP-RFEIFUI.1062.1 events.out.tfevents.1618857288.DESKTOP-RFEIFUI.3618.1
events.out.tfevents.1618589737.DESKTOP-RFEIFUI.1420.1 events.out.tfevents.1618859730.DESKTOP-RFEIFUI.3716.1
events.out.tfevents.1618589813.DESKTOP-RFEIFUI.1466.1 events.out.tfevents.1618860232.DESKTOP-RFEIFUI.4013.1
events.out.tfevents.1618592365.DESKTOP-RFEIFUI.1773.1 events.out.tfevents.1618862899.DESKTOP-RFEIFUI.4329.1
events.out.tfevents.1618628142.DESKTOP-RFEIFUI.3054.1

akainth015 · 2021-04-20T04:26:49Z

Tried to repair the branch - it looks like there was a messy merge. I cherry-picked the "made more replacements of tb_writer with event_storage" commit onto the correct version of the branch. Now the events.out.tfevents.1618862899.DESKTOP-RFEIFUI.4329.0 file should be gone, along with duplicated commits.

prigoyal · 2021-04-20T13:52:04Z

this is good to go from my side and no further comments.

thank you @grace-omotoso and @akainth015 :)

akainth015 assigned grace-omotoso and akainth015 Apr 6, 2021

akainth015 requested a review from prigoyal April 6, 2021 17:35

akainth015 force-pushed the master branch 2 times, most recently from edfcced to 795b0c3 Compare April 8, 2021 17:25

akainth015 changed the base branch from master to event-writers-master April 8, 2021 17:29

akainth015 force-pushed the write_tensorboard_writer branch from 621c704 to 513501b Compare April 8, 2021 17:43

prigoyal reviewed Apr 12, 2021

View reviewed changes

vissl/trainer/train_task.py Outdated Show resolved Hide resolved

prigoyal reviewed Apr 12, 2021

View reviewed changes

tests/test_configs.py Outdated Show resolved Hide resolved

grace-omotoso force-pushed the event-writers-master branch from 979419f to bdf5fd1 Compare April 15, 2021 18:23

grace-omotoso force-pushed the write_tensorboard_writer branch from 18bb601 to a8e7400 Compare April 15, 2021 18:48

akainth015 force-pushed the write_tensorboard_writer branch from a8e7400 to 09e8cdd Compare April 16, 2021 04:56

akainth015 force-pushed the event-writers-master branch from bdf5fd1 to e694347 Compare April 16, 2021 06:09

akainth015 force-pushed the write_tensorboard_writer branch from 09e8cdd to c88e234 Compare April 16, 2021 06:09

akainth015 marked this pull request as ready for review April 16, 2021 07:24

akainth015 requested a review from prigoyal April 16, 2021 07:24

akainth015 linked an issue Apr 16, 2021 that may be closed by this pull request

Write TensorboardWriter #16

Open

3 tasks

akainth015 commented Apr 16, 2021

View reviewed changes

vissl/hooks/tensorboard_hook.py Outdated Show resolved Hide resolved

vissl/hooks/tensorboard_hook.py Outdated Show resolved Hide resolved

prigoyal reviewed Apr 19, 2021

View reviewed changes

akainth015 force-pushed the event-writers-master branch from e694347 to fef448c Compare April 19, 2021 19:02

akainth015 and others added 9 commits April 19, 2021 12:03

Implement TensorboardWriter

9fdff75

The TensorboardWriter can also log rich images and histograms

append tensorboard writer to writer

5a6ece8

tentatively made flush_secs 60

f407ebe

Use a dictionary literal instead of a function call

aa1654d

Co-authored-by: Grace Omotoso <bgo2e@mtmail.mtsu.edu>

flush_secs is derived from config

f969ac2

checkpoint dir is derived from config

3b7d4f4

Call hook_generator after event storage is created and initialized

e7ebcb3

Add missing imports for Callable, default_hook_generator

5bfd568

Use tail of edges histogram as bucket_limits for Tensorboard

8232297

replaced add_scalar with put scalar and enabled visualization

f586bf1

akainth015 force-pushed the write_tensorboard_writer branch from fb4da39 to 2b8b646 Compare April 19, 2021 19:03

grace-omotoso and others added 3 commits April 19, 2021 21:16

TB VISUALIZATION key added under TB SETUP

61296cf

Remove import of create_file_symlink

a9747a7

made more replacements of tb_writer with event_storage

496cd70

akainth015 force-pushed the write_tensorboard_writer branch from d7870e7 to 496cd70 Compare April 20, 2021 04:21

prigoyal merged commit 41877d6 into event-writers-master Apr 20, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement TensorboardWriter #18

Implement TensorboardWriter #18

akainth015 commented Apr 6, 2021

prigoyal Apr 19, 2021

prigoyal commented Apr 19, 2021 •

edited by grace-omotoso

Loading

grace-omotoso commented Apr 19, 2021 •

edited

Loading

akainth015 commented Apr 20, 2021

prigoyal commented Apr 20, 2021

Implement TensorboardWriter #18

Implement TensorboardWriter #18

Conversation

akainth015 commented Apr 6, 2021

prigoyal Apr 19, 2021

Choose a reason for hiding this comment

prigoyal commented Apr 19, 2021 • edited by grace-omotoso Loading

grace-omotoso commented Apr 19, 2021 • edited Loading

akainth015 commented Apr 20, 2021

prigoyal commented Apr 20, 2021

prigoyal commented Apr 19, 2021 •

edited by grace-omotoso

Loading

grace-omotoso commented Apr 19, 2021 •

edited

Loading