Struct Normalizer 

pub struct Normalizer { /* private fields */ }

Folds case and diacritics, identically for terms and queries.

Both foldings are on by default (Normalizer::new). Diacritic folding runs first, then case folding; the result depends only on the input and the two flags, and normalising twice gives the same result as normalising once.

use keyhammer::text::Normalizer;

let n = Normalizer::new();
assert_eq!(n.normalize("Ação"), "acao");
assert_eq!(n.normalize("naïve CAFÉ"), "naive cafe");
// A decomposed accent (e + U+0301) is dropped too.
assert_eq!(n.normalize("cafe\u{301}"), "cafe");
// No Turkish tailoring: every i-like letter becomes i.
assert_eq!(n.normalize("Iıİi"), "iiii");
// Other scripts are copied unchanged.
assert_eq!(n.normalize("Ωmega"), "Ωmega");

Implementations

impl Normalizer

pub const fn new() -> Self

Case folding and diacritic folding.

pub const fn with_case_folding(self, on: bool) -> Self

The same normaliser with case folding on or off.

pub const fn with_diacritic_folding(self, on: bool) -> Self

The same normaliser with diacritic folding on or off.

pub const fn folds_case(self) -> bool

Whether case is folded.

pub const fn folds_diacritics(self) -> bool

Whether diacritics are folded.

pub fn normalize_char(self, c: char, emit: impl FnMut(char))

Calls emit with the characters c normalises to (none, one or up to three), in order.

use keyhammer::text::Normalizer;
let mut out = String::new();
Normalizer::new().normalize_char('ß', |c| out.push(c));
assert_eq!(out, "ss");

pub fn normalize(&self, s: &str) -> String

The normalised form of s.

pub fn normalize_into(&self, s: &str, out: &mut String)

Clears out and writes the normalised form of s into it, reusing its allocation.

pub fn normalize_mapped(&self, s: &str, map: &mut SourceMap) -> String

The normalised form of s, and in map where each of its code points came from (see SourceMap). map is cleared first.

use keyhammer::text::{Normalizer, SourceMap};

let mut map = SourceMap::new();
let n = Normalizer::new().normalize_mapped("Maße", &mut map);
assert_eq!(n, "masse");
// "ss" (output code points 2..4) comes from "ß" (input code point 2).
assert_eq!(map.to_source(2..4), Some(2..3));
assert_eq!(map.to_source(3..5), Some(2..4));

Trait Implementations

impl Clone for Normalizer

fn clone(&self) -> Normalizer

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for Normalizer

impl Debug for Normalizer

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for Normalizer

fn default() -> Self

Returns the “default value” for a type.

impl Eq for Normalizer

impl Hash for Normalizer

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl PartialEq for Normalizer

fn eq(&self, other: &Normalizer) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for Normalizer