(4->3) Add GUI Elements for Interacting With RecordingSession

[roomrys]

Description

This is a pretty large PR that aims to do a few high-level GUI tasks:

• Add a SessionsDock (for all/most RecordingSession-related GUI items)
• Allow navigate between views
• Only update "incomplete" nodes
• Add RecordingSessions table to SessionsDock #1654 (@vaibhavtrip29)
• Add CamerasTable to SessionsDock #1671 (@AdvaithRavishankar)
• (4a -> 4) Add UnlinkedVideosTable to SessionsDock #1720 (@7174Andy)
• (4a -> 4) Add menu to assign an Instance to an InstanceGroup #1747 (@ramizhajj1)
• (4c -> 4) Add color by InstanceGroup option #1760 (@7174Andy)
• (4a -> 4) Add instance group dock #1775 (@vaibhavtrip29)
• (4a -> 4) Add visual indicator for current camera #1783 (@justinvshen)
• (4b -> 4) Automatically add videos when “Add Recording Session” #1788 (@7174Andy)
• Add feature to maintain current frame index #1789 (@ramizhajj1)

See the subsections below for more detailed mini-tasks:

---

Add SesssionsDock

• Adds a SessionsDock which has a button for triangulation (and checkbox for auto-triangulation)
• Add SessionsDock to GUI
• Only auto-triangulate if checkbox is selected (meaning GuiState for auto_triangulate == True)

---

Allow navigate between views

• Add command class and methods to navigate between views
• Add menu-item for navigating between views
• Add hotkeys for navigating views Alt+V and Alt+Shift+V and store in shortcuts.yaml

---

Only update "incomplete" nodes

• Allow toggling a node as complete/incomplete with Ctrl-click (only incomplete nodes are updated via reprojection)
• Only update incomplete nodes during reprojection

---

Add RecordingSession table to SessionsDock (#1654)

The overall goal here is to add a new table to the SessionsDock that displays all RecordingSessions in the SessionsDock.main_window.label.sessions: List[RecordingSession] list. The widget should be a table that displays the RecordingSessions hash (eventually name), number of RecordingSession.cameras: List[Camcorder], and number of RecordingSession.videos: List[Video] (see Figure 1). This SessionsTable will be used to dynamically populate our CamerasTable later on.

• Subclass GenericTableModel to create a new model called SessionsTableModel with the class attribute properties = ("id", "videos", "cameras") (similar to VideosTableModel)
• Implement the method SessionsTableModel.item_to_data(self, obj: List[RecordingSession], item: RecordingSession) which returns the following dictionary: {id: item.__hash__(), videos: len(item.videos), cameras: len(item.cameras)} (similar to VideosTableModel.item_to_data)
• Add a sessions_model_type = SessionsTableModel attribute to SessionsDock (similar to SkeletonDock.nodes_model_type) and pass into the model_type list for super().__init__
• Add a SessionsDock.create_models method (or append to it if it already exists) that sets a SessionsDock.sessions_model = SessionsDock.sessions_model_type(items=main_window.state["labels"].sessions, context=main_window.commands) and returns a list of all models created (see SkeletonDock.create_models)
• Add a SessionsDock.create_tables method (or append to it if it already exists) that sets a SessionsDock.sessions_table = GenericTableView(state=..., row_name="session", ...) and returns a list of all tables created (see SkeletonDock.create_tables)
• Add button to "Add Session"
• Add button to "Remove Session"
• Disable "Remove Session" button when no selected RecordingSession
• In the SessionsDock.lay_everything_out method, add the SessionsDock.sessions_table to the SessionsDock.wgt_layout: QVBoxLayout (see the InstancesDock.lay_everything_out).
• Add a sessions option to UpdateTopic: Enum
• In MainWindow.on_data_update add (or append to) the if _has_topic(UpdateTopic.sessions) that updates the items to display in the SessionsTable (see video_dock.table update)

Test the SessionsTable

Tests for the SessionsTable will be added to tests/gui/test_dataviews.py.

• Test that the expected RecordingSession is returned when we select a specific row of the table (see existing test for inspiration)
• Test that when we update a RecordingSession in labels.sessions, the RecordingSession in the table is updated as well

Test the SessionsTable as part of the SessionsDock

Tests for the SessionsDock will be added to tests/gui/widgets/test_docks.py.

• Test interactions with table and GuiState (see existing test)

Figure 1: Depiction of all tables to add, including SessionsTable, CamerasTable, and UnlinkedVideosTable.

---

Add CamerasTable to SessionsDock

The overall goal is to add Camcorder/Video assignment table to the SessionsDock that is linked to the SessionsTable created above. This CamerasTable will have two columns. One column will display the Camcorder.names of all Camcorders in the RecordingSession.cameras: List[Camcorder] list. The second column will show which Videos are assigned to the adjacent Camcorder.name by displaying the returned value of RecordingSession.get_video(Camcorder) (or "" if None).

• Subclass GenericTableModel to create a new model called CamerasTableModel with properties = ("camera", "video")
• Implement the method CamerasTableModel.object_to_items(self, obj: RecordingSession) which returns the list of Camcorders in obj.cameras: List[Camcorder]
• Implement the method CamerasTableModel.item_to_data(self, obj: RecordingSession, item: Camcorder) which returns the following dictionary: {camera: item.name, video: obj.get_video(item)}
• Add a cameras_model_type = CamerasTableModel attribute to SessionsDock (similar to SkeletonDock.nodes_model_type) and pass into the model_type list for super().__init__
• Add a SessionsDock.create_models method (or append to it if it already exists) that sets a SessionsDock.cameras_model = SessionsDock.sessions_model_type(items=main_window.state["selected_session"], context=main_window.commands) and returns a list of all models created (see SkeletonDock.create_models)
• Add a SessionsDock.create_tables method (or append to it if it already exists) that sets a SessionsDock.cameras_table = GenericTableView(state=..., row_name="camera", ...) and returns a list of all tables created (see SkeletonDock.create_tables)
• Add a command class and CommandContext.unlink_video_from_camera method to un-link a Video from a Camcorder
• Add a button to "Unlink Video" and connect it to the CommandContext.unlink_video_from_camera method
• Disable button to "Unlink Video" when there is no selected Camcorder for the CamerasTable
• In the SessionsDock.lay_everything_out method, add the SessionsDock.cameras_table to the SessionsDock.wgt_layout: QVBoxLayout (see the InstancesDock.lay_everything_out).
• Add a sessions option to UpdateTopic: Enum if not already there
• In MainWindow.on_data_update add (or append to) the if _has_topic(UpdateTopic.sessions) that updates the items to display in the CamerasTable (see video_dock.table update)

Test the CamerasTable

Tests for the CamerasTable will be added to tests/gui/test_dataviews.py. These tests will only test the functionality of the table as it's own entity.

• Test that the expected Camcorder is returned when we select a specific row of the table (see existing test for inspiration)
• Test that when we update a Camcorder in labels.sessions, the Camcorder in the table is updated as well

Test the CamerasTable as part of the SessionsDock

Tests for the SessionsDock will be added to tests/gui/widgets/test_docks.py.

• Test interactions with table and GuiState (see existing test)

---

Add UnlinkedVideosTable to SessionsDock

Very similar steps to SessionsTable and CamerasTable (to be filled in), except for these few items:

• Add a dictionary {linked: ..., unlinked: ...} to LabelsDataCache that remembers which videos are linked/unlinked to a RecordingSession in labels.sessions (this dictionary will be used for the items in the UnlinkedVideosTable)
• Add command class and CommandContext.link_video_to_camera method to link a Video to a RecordingSession
• Add button to "Link Video" (connecting it to the CommandContext method created above)

---

Assign Instance to an InstanceGroup (waiting on 3a-3f to be merged)

• Create a sessionsMenu named "Sessions" (similar to the tracksMenu)
• Add a self.inst_groups_menu attribute to the MainWindow that adds a menu to the sessionsMenu called "Set Instance Group" (similar to self.track_menu)
• Enable/disable the self.inst_groups_menu when has_selected_instance == True (similar to self.track_menu)
• Create a MainWindow._update_sessions_menu method that takes in self and frame_idx, then retrieves the session: Optional[RecordingSession] = self.state["session"] and the frame_group: FrameGroup = session.frame_groups[frame_idx]. First clear the self.inst_groups_menu. Then, enumerate through all instance_groups in frame_group.instance_groups: List[InstanceGroup]. For the first 9 InstanceGroups use the key_command = Qt.SHIFT + Qt.Key_0 + inst_group_ind + 1 (otherwise use a default key_command = ""). For all instance_groups, add a self.inst_groups_menu action with name instance_group.name, callback lambda x=instance_group: self.commands.setInstanceGroup(x), and shortcut of key_command. Outside the enumeration, add a "New Instance Group" action to the self.inst_groups_menu with name "New Instance Group", callback self.commands.addInstanceGroup, and hotkey Qt.SHIFT + Qt.Key_0. Very similar to MainWindow._update_track_menu.
• Add a CommandContext.setInstanceGroup(self, instance_group: Optional[InstanceGroup]) method that executes the SetSelectedInstanceGroup class with input instance_group=instance_group (see CommandContext.setInstanceTrack).
• Add a CommandContext.addInstanceGroup(self) method that executes the AddInstanceGroup class (see CommandContext.addTrack).
• In sleap/gui/commands.py add a SetSelectedInstanceGroup class that subclasses EditCommand. Add a SetSelectedInstanceGroup.do_action(context, params) staticmethod that first retrieves the selected_instance = context.state[selected_instance], frame_idx: int = context.state["frame_idx"], video: Video = context.state["video"], session: RecordingSession = context.state["session"], frame_group: FrameGroup = session.frame_groups.get(frame_idx, None), and camera = session.get_camera(video=video). You will need to use frame_group = session.new_frame_group(frame_idx=frame_idx) if frame_group is None. Then, use frame_group.add_instance(instance=selected_instance, camera=camera, instance_group=instance_group) to add the selected_instance to the instance_group (and frame_group). Similar to the SetSelectedInstanceTrack class.
• In sleap/gui/commands.py add a AddInstanceGroup class that subclasses EditCommand. Add a AddInstanceGroup.do_action(context, params) staticmethod that first retrieves the frame_idx: int = context.state["frame_idx"], session: RecordingSession = context.state["session"], and frame_group: FrameGroup = session.frame_groups[frame_idx]. Then, use frame_group.add_instance_group(instance_group=None) to create and add a new empty InstanceGroup to the frame_group. Similar to the AddTrack class.

Using the workflow above (but with less detailed write-up), please also:

• Add a "Delete Instance Group" menu that populates and functions similarly to the "Set Instance Group" menu.

Figure 2: GUI elements for assigning Instances to InstanceGroups will be similar to how we assign Instances to Tracks.

---

Types of changes

• Bugfix
• New feature
• Refactor / Code style update (no logical changes)
• Build / CI changes
• Documentation Update
• Other (explain)

Does this address any currently open issues?

[list open issues here]

Outside contributors checklist

• Review the guidelines for contributing to this repository
• Read and sign the CLA and add yourself to the authors list
• Make sure you are making a pull request against the develop branch (not main). Also you should start your branch off develop
• Add tests that prove your fix is effective or that your feature works
• Add necessary documentation (if appropriate)

Thank you for contributing to SLEAP!

❤️

[codecov]

Codecov Report

Attention: Patch coverage is 81.62252% with 111 lines in your changes missing coverage. Please review.

Project coverage is 74.18%. Comparing base (07ea17b) to head (b55d4ef).

Files with missing lines	Patch %	Lines
sleap/gui/commands.py	84.00%	28 Missing ⚠️
sleap/gui/color.py	20.83%	19 Missing ⚠️
sleap/gui/dataviews.py	56.41%	17 Missing ⚠️
sleap/gui/widgets/video.py	51.42%	17 Missing ⚠️
sleap/gui/app.py	85.91%	10 Missing ⚠️
sleap/io/cameras.py	90.72%	9 Missing ⚠️
sleap/gui/widgets/docks.py	91.48%	8 Missing ⚠️
sleap/util.py	92.85%	3 Missing ⚠️

Additional details and impacted files

@@                                 Coverage Diff                                 @@
##           liezl/asc-initial-update-instances-across-views    #1283      +/-   ##
===================================================================================
+ Coverage                                            73.99%   74.18%   +0.19%     
===================================================================================
  Files                                                  135      135              
  Lines                                                24923    25438     +515     
===================================================================================
+ Hits                                                 18441    18872     +431     
- Misses                                                6482     6566      +84     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.