Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FEAT Add scalable and efficient memory #97

Merged

Conversation

rdheekonda
Copy link
Contributor

@rdheekonda rdheekonda commented Mar 13, 2024

Description

  1. Added DuckDB database functionality into PyRIT as primary memory storage.
  2. Deprecated FileMemory module.
  3. Restructured memory models to utilize SQLAlchemy for schema detection.
  4. Implemented data export functionality default to JSON format
  5. Added DuckDB with update functionalities, adaptable to new attack techniques.
  6. Moved analytics functions from core memory interface to analytics sub package.
  7. Updated demo\code for memory pointing to DuckDB memory.
  8. To release database resources, we need to manually invoke dispose_engine at the end of the Jupyter notebook, as automatic exit functions like atexit or __del__ are not reliably triggered by the Jupyter process.
  9. Added cleaning up DB resources logic to base orchestrator class and used in demo notebooks
  10. Clean up FileMemory references in the core repo.
  11. Fixed bug in prompt_target_text, class variable declared as _memory but instance variable is defined as self.memory
  12. Added and fixed unit tests
  13. Doc updated

Tests

  • no new tests required
  • new tests added
  • existing tests adjusted

Documentation

  • no documentation changes needed
  • documentation added or edited
  • example notebook added or updated

@rdheekonda rdheekonda changed the title [DRAFT]Add scalable and efficient memory [DRAFT]FEAT Add scalable and efficient memory Mar 15, 2024
doc/code/memory.py Outdated Show resolved Hide resolved
doc/code/memory.py Outdated Show resolved Hide resolved
@rdheekonda rdheekonda marked this pull request as ready for review March 16, 2024 00:48
@rdheekonda rdheekonda changed the title [DRAFT]FEAT Add scalable and efficient memory FEAT: Add scalable and efficient memory Mar 16, 2024
doc/how_to_guide.py Outdated Show resolved Hide resolved
Copy link
Contributor

@rlundeen2 rlundeen2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is biiiig, so tough to tell smaller issues. But imo let's get it in and tweak it, I think the overall design is really good

@rdheekonda rdheekonda changed the title FEAT: Add scalable and efficient memory FEAT Add scalable and efficient memory Mar 19, 2024
Raja Sekhar Rao Dheekonda added 2 commits March 18, 2024 20:28
@rdheekonda rdheekonda merged commit aa4b5a7 into Azure:main Mar 19, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants