v0.3.0

What's Changed

• New and improved scorers! Many new scorers have been added, and scorers can now be swapped out and made generic.
• Many new attack techniques and variations have been introduced. These include skeleton key, most of GPTFuzz, adding text to images, repeated token attack, cipherchat, shorten/expand, tone, CodeChameleon, and more. A total of 13 new converters have been added!
• Framework improvements:
  • Ability to duplicate conversations for reuse (this makes implementation easier for attacks like PAIR/TAP/crescendo).
  • Converters can be added to LLM responses.
  • All framework calls are now async and parallelizable.
  • Error handling and intelligent automatic retries in targets (e.g., for network errors) and converters/scorers (e.g., for JSON deserialization).

Full list of Changes

• FEAT: Refactoring and Standardizing Scores and Scorers by @rlundeen2 in #190
• FIX: Making RESULTS_PATH be simple in pip packages by @rlundeen2 in #191
• FIX: Minor Self-Ask Scorer Improvements by @rlundeen2 in #194
• FEAT: Adding Scores to the Database by @rlundeen2 in #195
• MAINT use context manager in XPIA notebook by @romanlutz in #198
• FEAT: Update score_async to add score to database by @rlundeen2 in #200
• FEAT support duplicating memory when cloning orchestrators by @romanlutz in #177
• MAINT: Likert Scoring Tweaks to Reduce False Positives by @rlundeen2 in #201
• FEAT add CSV support by @romanlutz in #197
• FEAT: Adding Human in the Loop Scorer by @rlundeen2 in #202
• FEAT: Azure content filter scorer by @cseifert1 in #206
• FEAT Adding Image Converter: add text on image by @jbolor21 in #205
• FEAT: Score Prompts Orchestrator by @rlundeen2 in #208
• MAINT: Deprecated send_prompt methods by @mart123p in #204
• FEAT Add image generation example with red teaming orchestrator and unify existing orchestrator definitions by @romanlutz in #189
• FEAT: self ask conversation objective and verifier scorer for crescendo by @cseifert1 in #209
• FEAT: Centralize Exception Handling and Implement in GPTv Target by @rdheekonda in #207
• MAINT Making Prompt Converters Async by @jbolor21 in #211
• Update .env_example Typo "Azure Open AI"→"Azure OpenAI" by @hyoshioka0128 in #214
• MAINT: Small scoring updates by @rlundeen2 in #215
• MAINT: Adding pretty print functionality and small RTO updates by @rlundeen2 in #217
• DOC: Re-organizing documentation by @rlundeen2 in #219
• FEAT: Add Suffix Converter by @NaijingGuo in #212
• MAINT: Updating GPT-V to use new exception guide by @rlundeen2 in #220
• FEAT: Add nesting and prepend/append jailbreaks from papers by @jl8771 in #216
• MAINT Adding Error Handling to OpenAIChatInterface by @jbolor21 in #218
• MAINT Add Exception Handling to DALLE Target by @rdheekonda in #221
• FEAT: Add repeated token attack converter by @jl8771 in #224
• MAINT Resolve Install Issues and Add Multiline Text Wrapping in AddTextImageConverter by @rdheekonda in #230
• MAINT: PromptRequestPiece SHA setting update by @rlundeen2 in #231
• FEAT: Implements Crescendo-style attack based on system prompt. by @dlmgary in #237
• MAINT add notebook version disclaimer by @romanlutz in #234
• FEAT: Adding Converters to Output by @rlundeen2 in #236
• DOC: Reorganizing MemoryDocs by @rlundeen2 in #239
• Added complex code jailbreak template by @petebryan in #238
• FEAT: Add prompt converters for atbash, caesar, morse and cipherchat from paper by @jl8771 in #223
• MAINT add test instructions to release guide by @romanlutz in #232
• FIX: Fixing doc links by @rlundeen2 in #245
• FEAT: Adding Master Key Jailbreak by @SafwanA02 in #248
• MAINT Adding Error Handling Code for converters by @jbolor21 in #247
• FIX: Fixing score conversation history by @rlundeen2 in #251
• FEAT: Add shorten/expand converters by @jl8771 in #246
• FEAT: Add CodeChameleon converter by @jl8771 in #240
• FEAT: Adding Noise and Tone Converters by @rlundeen2 in #252
• FEAT: Add persuasion converter with 5 persuasion techniques by @jl8771 in #253
• FEAT Implementation of SQL Server connectivity by @elgertam in #227
• MAINT Error Handling for Scorers by @jbolor21 in #256
• FIX: Skeleton Key Orchestrator by @SafwanA02 in #260
• MAINT upgrading AOAI version by @jbolor21 in #264

New Contributors

• @mart123p made their first contribution in #204
• @hyoshioka0128 made their first contribution in #214
• @jl8771 made their first contribution in #216
• @SafwanA02 made their first contribution in #248
• @elgertam made their first contribution in #227

Full Changelog: v0.2.1...v0.3.0