v0.9.0.13

What's Changed

garak's 1st birthday! 🎂

Headlines in this birthday release:

• Multimodal support! LLaVa + FigStep - HUGE thanks to @DavidLee528
• LiteLLM generator support - thanks to @Tien-Cheng!
• DoNotAnswer probe for prompts an LLM should decline - thanks to @AhsanAyub!
• LangChain Serve generator support - thanks to @GustavFredrikson!
• Support for NIM generators, NVIDIA Inference Microservices
• OpenAI abstraction with parallelisation support
• Windows compatibility enhanced, with optional testing in the workflow
• Hugging Face speedup

plugins

• LiteLLM generator by @Tien-Cheng in #572
• Support for using 'langchain serve' endpoints by @GustavFredrikson in #588
• Enable parallel OpenAI calls by @jmartin-tech in #645
• Multi-modal Jailbreaking Attack on LLaVA by @DavidLee528 in #587
• bump openai module version to match paths in latest litellm by @leondz in #664
• generator: NIM by @leondz in #637
• Probe: Do Not Answer by @AhsanAyub in #608

architecture

• change supported pythons to 3.10-3.12 by @leondz in #503
• add more detailed TAP docs by @leondz in #504
• add multiple-result aggregator by @leondz in #505
• add post buff hook by @erickgalinkin in #506
• Add fleshed-out docs to all probes by @leondz in #507
• add bibtex under citation info in readme by @leondz in #511
• define broad test, all probes, 1 gen per by @leondz in #514
• Feature/taxonomy payloads by @leondz in #519
• include paraphrasing in broad conf by @leondz in #521
• choose whether buffing will also include the original prompt by @leondz in #523
• add config var for capping max # buffed prompts to add per buff by @leondz in #526
• document Probe.probe(); skip a buff hook if no buffs by @leondz in #527
• add type hints to base.Probe; fix base probe rst by @leondz in #528
• Bump datasets package by @shubhobm in #536
• Add ConversationalPipeline for huggingface models by @erickgalinkin in #539
• add generator for supporting openai module v0.x by @leondz in #553
• Update README.md by @erickgalinkin in #558
• Minor typo in FAQ by @jmartin-tech in #562
• Add additional error message when doc is None type by @DavidLee528 in #566
• shared constant & string literal by @jmartin-tech in #571
• Spelling corrections for multiple locations by @jmartin-tech in #564
• Reduce Huggingface GPU utilization by @erickgalinkin in #567
• skip verbose flag in secondary parser by @jmartin-tech in #576
• Added project twitter link and corrected a grammatic error by @codebrain001 in #578
• Convert GGML to expect GGUF format by @jmartin-tech in #581
• Update workflows: CLA asst bump, PR & manual testing by @leondz in #591
• add test de-duping using skip-duplicate-actions by @leondz in #597
• Remove #! entries from files not intended as executables by @jmartin-tech in #612
• Further align shebangs with code that has executable entry points by @leondz in #613
• interactive mode intro by @leondz in #614
• add tests for ggml generator by @jmartin-tech in #618
• add var for generator context_len and populate this for some generators by @leondz in #616
• allow generators.Base.generate() to take an optional param specifying generation count by @leondz in #600
• Enable windows tests as github action by @jmartin-tech in #626
• add on-demand macos testing by @leondz in #631
• macOS test install from correct path by @jmartin-tech in #633
• consolidate test file cleanup by @jmartin-tech in #634
• bump discord link by @leondz in #648
• meta the arguments a bit for GET vs other request types by @jmartin-tech in #640

stability

• fail gracefully if nvcf rejects input; compact zalgo prompts by @leondz in #509
• log & skip past NVCF 4xx errors by @leondz in #533
• fix empty autodan prompts & poor detector behaviour by @leondz in #534
• Fix AutoDAN issues by @erickgalinkin in #537
• fix bad nonetype handling in atkgen probe by @leondz in #538
• Division by zero error fixed in HTML report generation by @CoderMayhem in #545
• cap cohere lib version by @leondz in #569
• rm deprecated model from example by @leondz in #575
• Attack fixes by @erickgalinkin in #555
• More regex as string literal by @jmartin-tech in #586
• Bugfix/action dedupe by @leondz in #598
• wrap cli exec to gracefully catch keyboard exit signal by @jmartin-tech in #603
• Enforce warning output for garak classes. by @jmartin-tech in #605
• bump hf transformers v to avoid transformers#30076 by @leondz in #636
• update avidtools to remove typing reference by @jmartin-tech in #639
• torch v bump by @leondz in #649
• Pause FigStepTiny by @leondz in #652
• Bugfix/visual jailbreak pause by @leondz in #653
• limit push test to main by @jmartin-tech in #661
• Update MANIFEST.in so all resources are installed by @JKL98ISR in #660
• handle extant but closed hitlogfile file by @leondz in #665

New Contributors

• @CoderMayhem made their first contribution in #545
• @jmartin-tech made their first contribution in #562
• @codebrain001 made their first contribution in #578
• @Tien-Cheng made their first contribution in #572
• @GustavFredrikson made their first contribution in #588
• @JKL98ISR made their first contribution in #660
• @AhsanAyub made their first contribution in #608

Full Changelog: v0.9.0.12...v0.9.0.13