Proposal: Arabic & Multilingual Security Contributions to PyRIT

[AbdaullahAG]

Hi PyRIT team 👋

I recently published SemGuard, a research project focused on
detecting prompt attacks across Arabic, Arabizi, and English —
accepted at IEEE AEECT 2026.

As part of it, I built a validated Arabic prompt attack dataset
(807 samples) covering injection/jailbreak, phishing, privacy
leakage, Unicode-based attacks, and safe educational prompts.

Given PyRIT's red-teaming focus, would there be interest in an
Arabic dataset loader or Arabic-focused attack scenarios? Happy
to share more details (including a semantic evaluation approach
we used) if useful — just don't want to front-load everything
before knowing what's actually relevant to the roadmap.

Thanks!

[romanlutz]

@AbdaullahAG thanks for the proposal! I do think it's potentially relevant. We have similar datasets in our datasets module. We typically exclude safe prompts unless they have a specific purpose, e.g., showing overrefusal.

Is the dataset available somewhere to look at?

[AbdaullahAG]

@romanlutz Thanks for the response!

Here's the dataset:
https://huggingface.co/datasets/AG-31625874/SemGuard-Dataset

And the repo (code, preprocessing utilities, etc.):
https://github.com/AbdaullahAG/SemGuard

Good point on safe prompts — in our case they weren't meant as
a standalone benchmark item, but as reference anchors for a
semantic classifier (comparing input intent against attack/safe/
destructive sets rather than binary classification). Happy to
strip them out or restructure if that's a better fit for a PyRIT
dataset loader — whatever aligns best with your existing module's
conventions.