Invisible Character Detection
PromptShield detects Unicode characters that are not visually rendered but still affect how text is interpreted by AI models, tokenizers, parsers, and validation systems.
These characters are commonly used for:
- prompt smuggling
- token-boundary manipulation
- hidden payload encoding
- content obfuscation
- validation bypass
Invisible characters are especially dangerous in prompts, configuration files, markdown, and generated content.
Why this matters
AI systems operate on tokens, not visual text.
Invisible characters can create situations where:
- humans see one thing
- the model receives another
- validation logic behaves differently than expected
This breaks the assumption:
“What you see is what the model sees.”
PromptShield prevents this class of attack.
Detected characters
PromptShield monitors these invisible characters:
| Character | Code point | Label |
|---|---|---|
| Zero Width Space | U+200B | ZWSP |
| Zero Width Non-Joiner | U+200C | ZWNJ |
| Zero Width Joiner | U+200D | ZWJ |
| Byte Order Mark | U+FEFF | BOM |
| Hangul Filler | U+3164 | HF |
| Halfwidth Hangul Filler | U+FFA0 | HHF |
| Unicode Tag Characters | U+E0000–U+E007F | TAG |
BIDI control characters are handled by the Trojan Source detector.
Rules
PromptShield uses capability-based rules, not per-character rules.
This keeps diagnostics meaningful and avoids Unicode lint noise.
PSU001
Invisible characters present
Severity: LOW
Invisible Unicode characters were detected in the text.
These characters can alter tokenization and prompt interpretation without being visible to reviewers.
Example:
ignorepreviousinstructionsSuggested remediation:
Remove invisible characters unless they are intentionally required.
PSU002
Token boundary manipulation
Severity: HIGH
An invisible character was detected inside a visible token.
This may allow attackers to bypass validation rules or manipulate tokenization boundaries.
Example:
admin
ignoreThis is a strong signal of prompt-manipulation intent.
Suggested remediation:
Remove invisible characters embedded within words.
PSU004
Unicode tag payload
Severity: HIGH
Unicode tag characters were used to encode hidden ASCII content.
Unicode tags can conceal instructions inside invisible text streams.
Example payload:
ignore previous instructionsSuggested remediation:
Remove Unicode tag characters containing hidden content.
PSU005
Excessive invisible characters
Severity: MEDIUM
A long sequence of invisible characters (16 or more) was detected.
Large invisible spans are commonly used for:
- padding
- obfuscation
- prompt segmentation attacks
Suggested remediation:
Remove unnecessary invisible characters.
Design notes
PromptShield intentionally uses rule precedence when analyzing invisible spans:
PSU004 → Unicode tag payload
PSU005 → Excessive invisible padding
PSU001 → Invisible characters presentEach span produces one primary diagnostic to avoid noise.
PSU002 is emitted independently because it represents a positional misuse of invisible characters.
Mental model
Invisible-character detection in PromptShield is similar to:
- tokenizer safety checks
- prompt integrity validation
- Unicode security linting
It protects against structural prompt manipulation, not content correctness.