Prompt Normalizer and Refactor

[rlundeen2]

Description

This rearchitects the code to use prompt normalizer, prompt targets, prompt transformers, and orchestrators.

It also has some minor updates with memory, which is used to keep track of conversations in the normalizer.

Tests

• no new tests required
• [x ] new tests added
• existing tests adjusted

Documentation

• no documentation changes needed
• [ x] documentation added or edited
• [x ] example notebook added or updated

[github-actions]

Test Results

83 tests  +11   83 ✅ +11   9s ⏱️ -1s
 1 suites ± 0    0 💤 ± 0 
 1 files   ± 0    0 ❌ ± 0 

Results for commit 9a46037. ± Comparison against base commit 4aace51.

This pull request removes 1 and adds 12 tests. Note that renamed tests count towards both.

tests.memory.test_file_memory ‑ test_file_memory_get_memory_by_session_id

tests.memory.test_file_memory ‑ test_file_memory_get_memory_by_conversation_id
tests.memory.test_file_memory ‑ test_file_memory_get_memory_by_normalizer_id
tests.test_prompt_orchestrator ‑ test_send_multiple_prompts_no_transformer
tests.test_prompt_orchestrator ‑ test_send_prompt_no_transformer
tests.test_prompt_orchestrator ‑ test_send_prompts_b64_transform
tests.test_prompt_orchestrator ‑ test_sendprompts_orchestrator_sets_target_memory
tests.test_prompt_target ‑ test_complete_chat_user_no_system
tests.test_prompt_target ‑ test_complete_chat_user_with_system
tests.test_prompt_target ‑ test_complete_chat_user_with_system_calls_chat_complete
tests.test_prompt_target ‑ test_set_system_prompt
…

♻️ This comment has been updated with latest results.

[romanlutz]

Can you add a more descriptive PR title? 🙂

[cseifert1]

gotcha; that clarfies.

________________________________ From: rlundeen2 ***@***.***> Sent: Friday, February 2, 2024 10:48 AM To: Azure/PyRIT ***@***.***> Cc: Christian Seifert (HE| HIM) ***@***.***>; Comment ***@***.***> Subject: Re: [Azure/PyRIT] Prompt Normalizer and Refactor (PR #32) @rlundeen2 commented on this pull request.

________________________________ In pyrit/prompt_target/prompt_target.py<#32 (comment)>:

@@ -0,0 +1,25 @@

+# Copyright (c) Microsoft Corporation. +# Licensed under the MIT license. + +import abc +from pyrit.memory import MemoryInterface + + +class PromptTarget(abc.ABC): + memory: MemoryInterface + supported_transformers: list The idea is there are only some transformers a target supports. AzureOpenAI won't accept a word document, for example, but a onedrive target will accept most everything. This will need to be flushed out as our target and transformer library increases (e.g. we may want categories and things) but to start I thought a list was enough. — Reply to this email directly, view it on GitHub<#32 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AA4GNNN7BNKGD4N2GY55N33YRUYGXAVCNFSM6AAAAABCTZJ3FWVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMYTQNRQGA4DGMZRGQ>. You are receiving this because you commented.Message ID: ***@***.***>