Cheshire3 Objects: Index

|  | ## Cheshire3 Objects: Index

Description

An Index is an object which defines an access point into records and is responsable for extracting that information from them. It can then store the information extracted in an IndexStore. The entry point can be defined using one or more XPath expressions, and the extraction process can be defined using a workflow chain of standard objects. These chains must start with an Extractor, but from there might then include PreParsers, Parsers, Transformers, Normalizers and even other Indexes.

A simple chain for a stemmed keyword index might be:

- KeywordExtractor
- CaseNormalizer
- StemNormalizer

Whereas a chain which takes a URL from an attribute, fetches it and then extracts title keywords from it might look like:

- ExactExtractor
- URLNormalizer
- URLFetchPreParser
- HtmlPreParser
- DefaultParser
- TitleIndex
- KeywordExtractor
- CaseNormalizer

Configuration

Further documentation is available on configuring Cheshire3 Indexes:

- Index Configuration -- A tutorial on configuring indexes, with illustrative examples.
- Cheshire3 Configuration: Index -- Cheshire3 Configuration reference material on the tags used for index configuration.

Implementations

- index.SimpleIndex
  An index implementation that handles process chains for indexing and searchClauses for querying. Stores total matching records, total occurences, and record id plus occurences per record.
- index.ProximityIndex
  An index which also stores proximity information. Currently this is word based or element based, but further types could be added with appropriate Extractors.

API

Function | Parameters | Returns | Description
--- | --- | --- | ---
__init__ | domNode, parentObject |  | The constructer takes a DOM tree containing the configuration of the index and the object which the database should consider as its parent, normally a database.
index_record | session, record |  | Index a record according to this index's rules
remove_record | session, record |  | Remove the terms of a record from our index
update_record | session, newRecord, oldRecord |  | Remove the terms from the old record and add the new
search | session, searchClause, database | resultSet | Process search query against self and return resultSet
scan | session, searchClause, numberOfTerms, direction | list of term information | Handle a scan/browse request
sort | session, resultSets | resultSet | Sort resultsets according to the extraction rules of this index.
begin_indexing | session |  | Set up for batch mode indexing (many records will be indexed, optimize and save data on commit)
commit_indexing | session |  | Save batch mode indexing