Cheshire3 Objects: Normalizer

|  | ## Cheshire3 Objects: Normalizer

Description

Normalizer objects are chained after Extractors in order to transform the data from the record or query. A Normalizer might standardise case, perform stemming or transform a date into ISO8601 format. Normalizers are also needed to transform the terms in a request into the same format as the term in the Index. For example a date index might be searched using a free text date and that would need to be parsed into the normalised form in order to compare it with the stored data.

Implementations

- normalizer.SimpleNormalizer
  Simply returns the data with no modifications
- normalizer.CaseNormalizer
  Returns the data in all lower case
- normalizer.ArticleNormalizer
  Strips leading english articles (the, a, an)
- normalizer.PossessiveNormalizer
  Strips trailing possessive 's, eg truck's -> truck
- normalizer.IntNormalizer
  Returns a long int from the numeric data, or None if there is no mapping possible
- normalizer.StringIntNormalizer
  Returns a 0 padded length 12 string for an integer
- normalizer.StoplistNormalizer
  Strip stoplisted words from the data
- normalizer.StemNormalizer
  Apply a stemming algorithm to the data. Implementation can be configured for 11+ different languages.
- normalizer.DiacriticNormalizer
  Map characters with diacritics to their simple form. eg é -> e

API

Function | Parameters | Returns | Description
--- | --- | --- | ---
__init__ | config, parent |  |
process_string | session, string | data | Process a string into an alternative form