Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PR2: Add new preference and AnnouncementChecker class #1509

Open
wants to merge 35 commits into
base: shrivaths/changelog-announcement-1
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
de22a57
Add new preference and annoucementchecker class
shrivaths16 Sep 19, 2023
69ff1ed
Merge branch 'shrivaths/changelog-announcement-1' into shrivaths/chan…
shrivaths16 Sep 20, 2023
3fc0e75
Merge branch 'shrivaths/changelog-announcement-1' into shrivaths/chan…
shrivaths16 Sep 21, 2023
0db2904
Add new preferences and bulletin json converter
shrivaths16 Sep 28, 2023
7ab6d8b
Merge branch 'shrivaths/changelog-announcement-1' into shrivaths/chan…
shrivaths16 Sep 29, 2023
7639142
Merge branch 'shrivaths/changelog-announcement-1' into shrivaths/chan…
shrivaths16 Oct 2, 2023
babb86d
Merge branch 'shrivaths/changelog-announcement-1' of https://github.c…
shrivaths16 Oct 3, 2023
6eddcf0
Add some of the suggestions
shrivaths16 Oct 3, 2023
3f99b8b
Typo
shrivaths16 Oct 3, 2023
f8fb1f1
Merge branch 'shrivaths/changelog-announcement-1' into shrivaths/chan…
shrivaths16 Oct 12, 2023
d9947c3
Add test function for the announcement checker class
shrivaths16 Oct 16, 2023
4cb0c6a
Fix path bug
shrivaths16 Oct 16, 2023
3323115
Add boolean for new announcement
shrivaths16 Oct 17, 2023
fc856bf
Add fixtures, pass mainwindow states
shrivaths16 Oct 18, 2023
39ed0b6
Lint files
shrivaths16 Oct 19, 2023
dce4dab
Fix bulletin json creation
shrivaths16 Oct 30, 2023
9e1ebf7
Set latest data with title
shrivaths16 Oct 30, 2023
d35f029
Correct json directory
shrivaths16 Nov 2, 2023
21ea789
additional condition to announcementchecker class
shrivaths16 Nov 30, 2023
3cc4188
Better date comparison, more tests
shrivaths16 Dec 2, 2023
59f07ae
Add more conditions to announcementchecker class
shrivaths16 Dec 5, 2023
acdf130
Correct default value in prefs
shrivaths16 Dec 5, 2023
fbb1039
Merge branch 'shrivaths/changelog-announcement-1' into shrivaths/chan…
shrivaths16 Dec 14, 2023
63e329a
Merge branch 'shrivaths/changelog-announcement-1' into shrivaths/chan…
shrivaths16 Dec 20, 2023
cbb661d
Merge branch 'shrivaths/changelog-announcement-1' into shrivaths/chan…
shrivaths16 Dec 21, 2023
9c726af
Merge branch 'shrivaths/changelog-announcement-1' into shrivaths/chan…
shrivaths16 Jan 3, 2024
68cb64b
add branch to website workflow
shrivaths16 Jan 6, 2024
2c92712
add json file to the website
shrivaths16 Jan 8, 2024
303a2d2
generate json in the correct path
shrivaths16 Jan 9, 2024
b708717
store json as text file
shrivaths16 Jan 9, 2024
a679e61
keep_files is set true
shrivaths16 Jan 9, 2024
02757e6
add json to _static directory
shrivaths16 Jan 9, 2024
090e36f
Modify AnnouncementChecker Class with bulletin url
shrivaths16 Jan 11, 2024
aca6c7a
add date to announcement
shrivaths16 Jan 11, 2024
c218b10
add regex for date check
shrivaths16 Jan 17, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions docs/make_bulletin_json.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
import json
import os

# Set the file paths
input_md_file = os.path.join(os.path.dirname(__file__), 'bulletin.md')
output_json_file = os.path.join(os.path.dirname(__file__), 'bulletin.json')
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The script uses absolute paths to the markdown and JSON files. This could lead to issues if the script is run from a different directory or on a different machine where the file paths may not be the same. Consider using relative paths instead.

- input_md_file = os.path.join(os.path.dirname(__file__), 'bulletin.md')
- output_json_file = os.path.join(os.path.dirname(__file__), 'bulletin.json')
+ input_md_file = 'bulletin.md'
+ output_json_file = 'bulletin.json'


def generate_json_file():
with open(input_md_file, 'r', encoding='utf-8') as md_file:
markdown_content = md_file.read()
bulletin_json = []
content = ''

# Initialize title and date with default values
title = "DEFAULT_TITLE"
date = "DEFAULT_DATE"

for line in markdown_content.split('\n'):
if line.startswith('---'):
bulletin_json.append({'title': title, 'date': date, 'content':content})
content = ''
# Reset title and date to their default values after each section
title = "DEFAULT_TITLE"
date = "DEFAULT_DATE"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The script assumes that each section in the markdown file starts with ---. If this format changes, the script might not correctly identify the start of a new section. Consider adding a comment to document this assumption or use a more robust method to identify section boundaries.

elif line.startswith('##'):
title = line[3:].strip()
elif line.startswith('_'):
date = line[1:len(line)-1].strip()
Copy link

@coderabbitai coderabbitai bot Sep 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The date extraction logic assumes that the date is surrounded by underscores (_). If the date format changes or if there are additional underscores in the date string, this could lead to incorrect results. Consider using a more robust method for extracting the date, such as regular expressions.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We plan on sticking with this format, but perhaps we should add a test that the current bulletin.md changes can be parsed correctly. I'm curious, can you give an example of how you would parse the date?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@roomrys: Sure, I can provide an example using regular expressions which is a more robust way to parse the date. Here's how you could modify your code:

import re

# ...

for line in markdown_content.split('\n'):
    if line.startswith('---'):
        bulletin_json.append({'title': title, 'date': date, 'content':content})
        content = ''
    elif line.startswith('##'):
        title = line[3:].strip()
    else:
        match = re.search(r'_([^_]+)_', line)
        if match:
            date = match.group(1).strip()
        else:
            content += (line + '\n')

In this code, re.search(r'_([^_]+)_', line) looks for a pattern where there are any characters between two underscores. The parentheses in the regular expression create a group, and match.group(1) gets the contents of the first group. This will correctly extract the date even if there are additional underscores in the line.

For testing, you could create a separate function that takes a string as input and returns the parsed title, date, and content. Then, you can write tests that call this function with different inputs and check that the output is as expected.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basically the same logic, but with regex which is more difficult to understand imo.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@roomrys: I understand your concern. Regular expressions can indeed be a bit difficult to grasp initially, but they offer a lot of flexibility and robustness when it comes to text parsing. In this case, the regular expression is used to ensure that the date extraction works correctly even if there are additional underscores in the line.

However, if you prefer to stick with the current approach, it's important to make sure that the format of the dates in the markdown file remains consistent. Any changes could potentially break the date extraction logic. As you suggested earlier, adding tests to verify that the parsing works correctly would be a good way to catch any issues early on.

If you have any other questions or need further clarification, feel free to ask!

else:
content += (line + '\n')
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The script assumes that the title of a section starts with ## and the date starts with _. If these formats change, the script might not correctly extract the title and date. Consider adding comments to document these assumptions or use a more robust method to extract the title and date.

# Append last section
bulletin_json.append({'title': title, 'date': date, 'content':content})
Copy link

@coderabbitai coderabbitai bot Sep 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The variables title and date are used before they are defined. This could lead to a NameError if the first line of the markdown file does not start with ## or _. Initialize these variables before the loop to avoid potential errors.

- content = ''
+ title = date = content = ''

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pretty easy preventative measure. Maybe we can set defaults and raise a warning if the parser never overrides the default value?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@roomrys, that's a great suggestion. We can initialize title and date with default values and then check if they remain unchanged after parsing the markdown file. If they do, we can raise a warning to alert the user. Here's how we could modify the code:

import json
import warnings

# Set the file paths
input_md_file = 'D:\TalmoLab\sleap\docs\bulletin.md'
output_json_file = 'D:\TalmoLab\sleap\docs\\bulletin.json'

def generate_json_file():
    with open(input_md_file, 'r', encoding='utf-8') as md_file:
        markdown_content = md_file.read()
    bulletin_json = []
    content = ''

    # Initialize title and date with default values
    title = "DEFAULT_TITLE"
    date = "DEFAULT_DATE"

    for line in markdown_content.split('\n'):
        if line.startswith('---'):
            bulletin_json.append({'title': title, 'date': date, 'content':content})
            content = ''
            # Reset title and date to their default values after each section
            title = "DEFAULT_TITLE"
            date = "DEFAULT_DATE"
        elif line.startswith('##'):
            title = line[3:].strip()
        elif line.startswith('_'):
            date = line[1:len(line)-1].strip()
        else:
            content += (line + '\n')
    # Append last section   
    bulletin_json.append({'title': title, 'date': date, 'content':content})

    # Check if title or date were never updated from their default values
    if title == "DEFAULT_TITLE" or date == "DEFAULT_DATE":
        warnings.warn("The parser did not find a title or date in the markdown file.")

    with open(output_json_file, 'w', encoding='utf-8') as json_file:
        json.dump(bulletin_json, json_file, ensure_ascii=False, indent=4)

if __name__ == '__main__':
    generate_json_file()

This way, we can ensure that the script doesn't fail due to uninitialized variables and also alert the user if the markdown file is not formatted as expected.


with open(output_json_file, 'w', encoding='utf-8') as json_file:
json.dump(bulletin_json, json_file, ensure_ascii=False, indent=4)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The script writes the generated JSON data to a file without checking if the file already exists. If the file does exist, its contents will be overwritten. Consider checking if the file exists before writing to it and prompt the user for confirmation if it does.

+ if os.path.exists(output_json_file):
+     print(f"File {output_json_file} already exists. Overwrite? (y/n)")
+     response = input().strip().lower()
+     if response != 'y':
+         return
  with open(output_json_file, 'w', encoding='utf-8') as json_file:
      json.dump(bulletin_json, json_file, ensure_ascii=False, indent=4)


if __name__ == '__main__':
generate_json_file()
9 changes: 8 additions & 1 deletion sleap/gui/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@
from sleap.gui.overlays.tracks import TrackListOverlay, TrackTrailOverlay
from sleap.gui.shortcuts import Shortcuts
from sleap.gui.state import GuiState
from sleap.gui.web import ReleaseChecker, ping_analytics
from sleap.gui.web import ReleaseChecker, AnnouncementChecker, ping_analytics
from sleap.gui.widgets.docks import (
InstancesDock,
SkeletonDock,
Expand Down Expand Up @@ -152,6 +152,8 @@ def __init__(
self.state["share usage data"] = prefs["share usage data"]
self.state["skeleton_preview_image"] = None
self.state["skeleton_description"] = "No skeleton loaded yet"
self.state["announcement last seen date"] = prefs["announcement last seen date"]
self.state["announcement"] = prefs["announcement"]
if no_usage_data:
self.state["share usage data"] = False
self.state["clipboard_track"] = None
Expand All @@ -162,6 +164,9 @@ def __init__(
self.state.connect("show non-visible nodes", self.plotFrame)

self.release_checker = ReleaseChecker()
self.announcement_checker = AnnouncementChecker(
app = self
)

if self.state["share usage data"]:
ping_analytics()
Expand Down Expand Up @@ -215,6 +220,8 @@ def closeEvent(self, event):
prefs["color predicted"] = self.state["color predicted"]
prefs["trail shade"] = self.state["trail_shade"]
prefs["share usage data"] = self.state["share usage data"]
prefs["announcement last seen date"] = self.state["announcement last seen date"]
prefs["announcement"] = self.state["announcement"]

# Save preferences.
prefs.save()
Expand Down
40 changes: 39 additions & 1 deletion sleap/gui/web.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,15 @@
import attr
import pandas as pd
import requests
from typing import List, Dict, Any
from typing import List, Dict, Any, Optional, Tuple
import json
import os


REPO_ID = "talmolab/sleap"
ANALYTICS_ENDPOINT = "https://analytics.sleap.ai/ping"
BASE_DIR = os.path.dirname(os.path.abspath(__file__))
BULLETIN_JSON = os.path.join(BASE_DIR, "..", "docs", "bulletin.json")


@attr.s(auto_attribs=True)
Expand Down Expand Up @@ -146,6 +150,40 @@ def get_release(self, version: str) -> Release:
)


@attr.s(auto_attribs=True)
class AnnouncementChecker:
"""Checker for new announcements on the bulletin page of sleap."""

app: "MainWindow"
shrivaths16 marked this conversation as resolved.
Show resolved Hide resolved
bulletin_json_path: str = BULLETIN_JSON
previous_announcement_date: str = app.state["announcement last seen date"]
_latest_data: Optional[Dict[str, str]] = None

def _read_bulletin_data(self) -> Dict[str, str]:
"""Reads the bulletin data from the JSON file."""
try:
with open(self.bulletin_json_path, 'r', encoding='utf-8') as jsf:
data = json.load(jsf)
return data[0]
except FileNotFoundError:
return {}

def get_latest_announcement(self) -> Optional[Tuple[str, str]]:
"""Return latest announcements on the releases page not seen by user."""
self._latest_datalatest_data = self._read_bulletin_data()
if self._latest_data and self._latest_data['date'] != self.previous_announcement_date:
return (self._latest_data['date'], self._latest_data['content'])
return None

def update_announcement(self):
"""Update the last seen date of announcement in preferences."""
announcement = self.get_latest_announcement()
if announcement is None:
return
self.app.state["announcement last seen date"] = announcement[0]
self.app.state["announcement"] = announcement[1]


def get_analytics_data() -> Dict[str, Any]:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This new hunk introduces a new class AnnouncementChecker with methods to read bulletin data from a JSON file, retrieve the latest unseen announcement, and update the last seen date in preferences. However, there's an issue on line 173 where self._latest_datalatest_data should be self._latest_data.

- self._latest_datalatest_data = self._read_bulletin_data()
+ self._latest_data = self._read_bulletin_data()

Also, the attribute previous_announcement_date (line 159) is initialized using app.state["announcement last seen date"] which might raise a KeyError if the key doesn't exist in the app state. It would be safer to use app.state.get("announcement last seen date") which will return None if the key doesn't exist.

- previous_announcement_date: str = app.state["announcement last seen date"]
+ previous_announcement_date: str = app.state.get("announcement last seen date")

"""Gather data to be transmitted to analytics backend."""
import os
Expand Down
3 changes: 3 additions & 0 deletions sleap/prefs.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
"""

from sleap import util
from datetime import date


class Preferences(object):
Expand All @@ -28,6 +29,8 @@ class Preferences(object):
"node label size": 12,
"show non-visible nodes": True,
"share usage data": True,
"announcement last seen date": None,
"announcement": None,
}
_filename = "preferences.yaml"

Expand Down
Loading