Applying Stringprep
Pythonimport stringprep import unicodedata def prepare_string(s, mapping_table=stringprep.b1_set): """Apply basic stringprep steps: map, then check prohibited chars.""" # Step 1: Mapping - remove characters mapped to nothing (B.1) chars = [c for c in s if c not in mapping_table] # Step 2: Normalize (KC form, per RFC 3454 profiles like Nameprep) normalized = unicodedata.normalize('NFKC', ''.join(chars)) # Step 3: Prohibit check for c in normalized: if stringprep.in_table_c12(c): # Non-ASCII space raise ValueError(f"Prohibited character: {c!r}") return normalized print(prepare_string("Hello\u00ad World")) # soft hyphen (B.1) is stripped
Note: stringprep is deprecated (removal planned per PEP 594-style cleanup for legacy protocol modules). Only use it for interoperability with existing systems; do not build new protocols on it. Prefer modern Unicode normalization (unicodedata, IDNA via encodings.idna or the third-party idna package) for anything new.
Stringprep implements RFC 3454's generic framework. A concrete profile (e.g., Nameprep, SASLprep) picks specific tables/operations from this toolkit. To implement or verify a profile:
Progress:
- Step 1: Identify the profile spec (Nameprep RFC 3491, SASLprep RFC 4013, etc.) and which tables it references
- Step 2: Apply the mapping step using the relevant
in_table_*functions that indicate "map to nothing" or "map to space" - Step 3: Apply normalization (usually NFKC) if the profile requires it
- Step 4: Run the prohibited output check — reject strings containing characters in prohibited tables
- Step 5: Run the bidirectional (BiDi) check — enforce RFC 3454 Section 6 rules on RandALCat/LCat mixing
- Step 6: Return the prepared string, or raise/report the specific violation
Key table functions
| Function | Meaning |
|---|---|
in_table_a1(c) | Unassigned code points |
in_table_b1(c) | Commonly mapped to nothing (soft hyphen, zero-width chars) |
in_table_b2/b3 | Case-folding mapping tables |
in_table_c11–c22 | Prohibited character categories (spaces, control chars, private use, surrogates, tagging chars) |
in_table_c6–c9 | Prohibited: unassigned, inappropriate for canonical rep, tagging, change display/deprecated |
in_table_d1(c) | RandALCat (right-to-left) characters |
in_table_d2(c) | LCat (left-to-right) characters |
Example 1: Nameprep-style prohibited check
Input: "us\u200der" (contains zero-width non-joiner, table B.1)
Output: After mapping step strips B.1 chars → "user". Passes prohibited-check (no C.1–C.9 chars remain).
Example 2: BiDi violation
Input: "abc\u05D0" (Latin followed by Hebrew aleph, a RandALCat char)
Output: Raises — RFC 3454 Section 6 requires a string containing RandALCat chars to only contain RandALCat/LCat consistently (a string can't mix L and R categories per the BiDi rule); implementation must call in_table_d1/in_table_d2 and reject mixed-directionality strings that violate the rule.
Example 3: Full SASLprep-like pipeline
Input: "I\u00adX" (contains soft hyphen)
Output: Map step removes \u00ad (in b1_set) → "IX" → NFKC normalize (no change) → prohibited/BiDi checks pass → final: "IX"
- Always apply mapping before normalization before prohibition/BiDi checks — order matters per RFC 3454.
- Use
unicodedata.normalize('NFKC', s)for the normalization step unless the profile specifies otherwise. - Implement the BiDi rule explicitly — it's easy to skip: if any character satisfies
in_table_d1(RandALCat), the string must not containin_table_d2(LCat) at all, and must start and end with a RandALCat character. - Cache/memoize
in_table_*checks in hot paths — they're pure functions over Unicode data and safe to precompute per-input-string. - When implementing SASLprep specifically, remember it forbids unassigned code points outright (table A.1) rather than allowing them.
- Don't use
stringprepfor new protocol design. It's deprecated; modern designs usePRECIS(RFC 8264) instead. - Don't skip the BiDi check — many implementations only do mapping + prohibition and silently produce RFC-non-compliant results.
- Don't apply NFC when the profile calls for NFKC (or vice versa) — Nameprep/SASLprep specifically require NFKC for compatibility folding.
- Don't assume
in_table_b1characters are prohibited — they're mapped to nothing (deleted), not rejected. - Don't forget
stringprepoperates per-character; you must loop over code points, not bytes — always work on properly decodedstrobjects, never raw bytes.