Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Topic prompt generator #1809

Merged
merged 165 commits into from
Mar 31, 2024
Merged

Topic prompt generator #1809

merged 165 commits into from
Mar 31, 2024

Conversation

nsantacruz
Copy link
Contributor

@nsantacruz nsantacruz commented Feb 25, 2024 •
edited
Loading

Celery

  • set up celery. this involves connecting to redis sentinel, setting up env vars for celery and redis sentinel and adding them to the celery config.
  • Create new generic tasks pod for handling celery tasks that require Sefaria-Project

APIs

  • Add topic prompt generator API which kicks off the topic prompt generation celery task
  • Add ref topic link bulk API for bulk editing ref topic links

New requirements

  • Celery
  • DNSPython - for looking up redis sentinel URLs
  • Sefaria LLM interface - for making sure objects are validated that are sent over the queue

UI

  • Interface for adding context sentences to ref topic links
  • Intreface for reviewing AI generated prompts
  • Refactor how the admin topic page menu is displayed to allow for arbitrary callbacks. Previously the callbacks needed to exist inside the AdminEditor component

yonadavGit and others added 30 commits January 23, 2024 16:47
@yonadavGit
feat(TopicPageEditor): add new "generate" button and pop up
a45663d
@yonadavGit
feat(TopicPageEditor): change var name
2ca2138
@yonadavGit
feat(TopicPageEditor): display colored 'review state' of topic prompt…
60fb92f 
…s near topic prompt
@yonadavGit
feat(TopicPageEditor): hide unpublished prompts when not signed in as…
6b05a24 
… a 'moderator'
@yonadavGit
fix(TopicPageEditor): add 'is_moderator' to condition
fa30317
@yonadavGit
fix(TopicPageEditor): extend display of prompts' state to hebrew UI
8a0bbeb
@yonadavGit
chore(TopicPageEditor): remove debug print
d57bf57
@yonadavGit
fix(TopicPageEditor): handle non-existent review states
1659ab9
@yonadavGit
fix(topic ref api): make endpoint accept 'fetch' post requests
dbdc002
@yonadavGit
feat(Sefaria object): new post request to topic ref api endpoint
7a068b0
@yonadavGit
feat(topic editor): dynamic review state - admins can mark prompts as…
a29075e 
… reviewed
@yonadavGit
feat(topic editor): dynamic review state for hebrew prompts
570e6ce
@yonadavGit
chore(topic editor): cleaner review state value mapping using mapping…
3f1ca8f 
… object
@yonadavGit
chore(topic editor): cleaner styling of review indicator button, no e…
4650abc 
…xplicit mention of language in css className
@yonadavGit
feat(topic source editor): add new fields to editor
e3a9772
@yonadavGit
fix(topic helper): change API helper function such that fields that a…
0801a9b 
…re not sent by client are not deleted from db
@yonadavGit
feat(topic source editor): change prompt to 'source description' and …
b0c1ef1 
…make 'ai title' field read only
@yonadavGit
feat(topic source editor): not displaying ai fields if they don't exi…
9e9af3f 
…st in object
@yonadavGit
feat(topic source editor): change API endpoint to handle review state…
72a86bd 
… field of description
@yonadavGit
feat(topic source editor): change aiTitle to previousTitle in editor …
2575e6d 
…interface, and make review state change to "edited" upon editing a topic prompt
@yonadavGit
chore(topic source editor): remove redundant lines
beca4e2
@yonadavGit
chore(topic source editor): remove redundant css definition
787333c
@yonadavGit
chore(topic source editor): remove keeping track of 'generate' button…
2d45786 
… state
@yonadavGit
feat(topic source editor): generate button is displayed iff there exi…
cc1591d 
…sts a source-link with context and without prompt
@yonadavGit
chore(topic source editor): remove unused function
6e77612
@yonadavGit
feat(topic source editor): add button 'published', displayed only whe…
c38298e 
…n there exists a prompt explicitly unpublished
@nsantacruz
feat(llm): add topic_prompt.py which acts as an interface to the LLM …
0c64a20 
…repo.
@nsantacruz
feat(llm): add celery settings to all relevant settings files
6087140
@nsantacruz
feat(llm): create basic tasks for celery
8e45a7c
@nsantacruz
feat(llm): save topic prompts after generation
b554a62
nsantacruz and others added 21 commits March 28, 2024 09:31
@nsantacruz
Merge pull request #1857 from Sefaria/topic-prompt-generator-fixes
63935c2 
Topic prompt generator fixes
@nsantacruz
fix(llm): make getLinksWithAiContent only return true if link is publ…
5038d1f 
…ished
@nsantacruz
refactor(llm): refactor useReviewState so it's used inside of ReviewS…
4ba2306 
…tateIndicator
@nsantacruz
refactor(llm): remove all mentions of Sefaria.interfaceLang
52c4442
@nsantacruz
refactor(llm): remove all mentions of Sefaria.interfaceLang
551975c
@yonadavGit
feat(topic prompt generator UI): upload the icon svs file
bf4f359
@yonadavGit
Merge branch 'topic-prompt-generator' of https://github.com/Sefaria/S…
d1f269d 
…efaria-Project into topic-prompt-generator
@nsantacruz
docs(llm): add note to generate_config.py
2607301
@nsantacruz
fix(llm): use redis port as sentinel port since only one port is requ…
fbf7b91 
…ired to be specified
@nsantacruz
Merge remote-tracking branch 'origin/topic-prompt-generator' into top…
159ebb2 
…ic-prompt-generator
@nsantacruz
docs: explain when to pass sentinel config
311da9a
@nsantacruz
fix(llm): fix reference to ref
dcb0db4
@nsantacruz
fix(llm): use full interface language in review state POST
71e499e
@nsantacruz
fix(llm): make sure review state remains none if prompt wasn't ai gen…
543d2b0 
…erated
@nsantacruz
fix(llm): remove review state if merged value is None
9829362
@nsantacruz
chore(llm): set celery values to connect to broker and stuff
f9b81c0
@nsantacruz
fix(llm): fix name of sentinel and redis password values
d2cca3c
@nsantacruz
fix(llm): fix name of sentinel and redis password values
e64f62a
@nsantacruz
fix(llm): add keys for redis secrets
93559b7
@nsantacruz
refactor(llm): use envFrom to get redis secrets
7117427
@nsantacruz
refactor(llm): simplify redis secrets
9548917
edamboritz
edamboritz previously approved these changes Mar 28, 2024
View reviewed changes
@nsantacruz nsantacruz added this pull request to the merge queue Mar 31, 2024
Merged via the queue into master with commit f099595 Mar 31, 2024
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@edamboritz edamboritz edamboritz approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@nsantacruz @edamboritz @yonadavGit @BrendanGalloway