# SNFEI: Structured Non-Fungible Entity Identifier

## Specification Version 1.1.0

### Abstract

The Structured Non-Fungible Entity Identifier (SNFEI) provides deterministic,
collision-resistant identification for civic entities that lack federal identifiers
(LEI, SAM UEI). This includes school districts, local governments, small nonprofits,
political action committees, and local businesses.

SNFEI uses SHA-256 hashing of normalized entity attributes to produce a stable
64-character hexadecimal identifier that remains constant across different source
systems despite formatting variations.

---

## 1. Architecture: The Normalizing Functor

SNFEI generation follows a Category Theory-inspired architecture with two functors:

```
┌───────────────┐     ┌─────────────────┐     ┌─────────────────┐
│  Raw Entity   │     │  Intermediate   │     │    Canonical    │
│    Data       │────>│    Canonical    │────>│     Entity      │
│               │  L  │                 │  N  │                 │
└───────────────┘     └─────────────────┘     └─────────────────┘
                                                        │
                                                        │ SHA-256
                                                        V
                                                ┌───────────────┐
                                                │     SNFEI     │
                                                │   (64-char)   │
                                                └───────────────┘
```

**L = Localization Functor** (jurisdiction-specific transforms)

-   Converts local abbreviations to canonical form
-   Handles language variants (French/English in Quebec)
-   Expands jurisdiction-specific agency acronyms

**N = Normalizing Functor** (universal normalization)

-   Lowercase conversion
-   Punctuation removal
-   Abbreviation expansion
-   Stop word filtering

---

## 2. The SNFEI Formula

The SNFEI is generated by hashing normalized attributes:

```
SNFEI = SHA256(Concatenate[
    legal_name_normalized,
    address_normalized,
    country_code,
    registration_date
])
```

### 2.1 Canonical Input Format

Fields are concatenated with pipe (`|`) delimiter:

```
legal_name_normalized|address_normalized|country_code|registration_date
```

Empty fields are included as empty strings to maintain field positions:

```
springfield unified school district||US|
```

### 2.2 Example

**Input:**

-   Name: "Springfield Unified School District #12"
-   Address: "123 Education Way, Suite 100"
-   Country: US
-   Registration: (none)

**After Localization (L):** (no changes for this example)

**After Normalization (N):**

-   legal_name_normalized: `springfield unified school district 12`
-   address_normalized: `123 education way`

**Canonical String:**

```
springfield unified school district 12|123 education way|US|
```

**SNFEI:**

```
bd4cdf26529bbd8882815249e7bfa21a8b6ace91166fb710fd41142221f37e67
```

---

## 3. The Localization Functor (L)

The Localization Functor transforms jurisdiction-specific data into an intermediate
canonical form before universal normalization.

### 3.1 Directory Structure

```
/localization/
    base.yaml           # Default/fallback rules
    us/
        base.yaml       # US-wide rules
        ca.yaml         # California-specific
        ny.yaml         # New York-specific
    ca/
        base.yaml       # Canada-wide rules
        on.yaml         # Ontario-specific
        qc.yaml         # Quebec-specific
```

### 3.2 Jurisdiction Rules

| Jurisdiction | Local Rule   | Example    |
| ------------ | ------------ | -----------|
| us/ny        | Expand NY agency acronyms | MTA → Metropolitan Transportation Authority |
| us/ca        | Expand CA agency acronyms | CalTrans → California Department of Transportation |
| ca/on        | Expand Ontario agencies   | TTC → Toronto Transit Commission |
| ca/qc        | French/English variants   | Limitée → Limitee  |

### 3.3 Localization Config Structure

```yaml
# us/ny.yaml
jurisdiction: us/ny
parent: us

abbreviations:
    mta: metropolitan transportation authority
    nycha: new york city housing authority
    nypd: new york police department

agency_names:
    port authority: port authority of new york and new jersey
```

---

## 4. The Normalizing Functor (N)

The Normalizing Functor applies universal transformations after localization.

### 4.1 Pipeline Steps (In Order)

1. **Lowercase** - Convert entire string to lowercase
2. **ASCII Transliteration** - Convert accented characters (é→e, ñ→n)
3. **Punctuation Removal** - Remove ALL punctuation including hyphens
4. **Whitespace Collapse** - Multiple spaces → single space, trim
5. **Abbreviation Expansion** - Expand to full form (see 4.2)
6. **Stop Word Removal** - Remove common words (see 4.3)
7. **Final Trim**

### 4.2 Universal Abbreviation Expansion

Legal suffixes are EXPANDED (not contracted):

| Input       | Output                    |
| ----------- | ------------------------- |
| inc, inc.   | incorporated              |
| corp, corp. | corporation               |
| llc, l.l.c. | limited liability company |
| ltd, ltd.   | limited                   |
| co, co.     | company                   |

Common abbreviations are expanded:

| Input       | Output                  |
| ----------- | ----------------------- |
| dist        | district                |
| sch         | school                  |
| usd         | unified school district |
| dept        | department              |
| assn, assoc | association             |
| natl        | national                |
| intl        | international           |
| st          | saint                   |
| mt          | mount                   |
| ft          | fort                    |

### 4.3 Stop Word Removal

The following words are removed after normalization:

-   the, of, a, an, and, for, in, on, at, to, by

---

## 5. Address Normalization

Addresses follow a similar pipeline with postal-specific expansions.

### 5.1 Pipeline

1. Lowercase
2. ASCII transliteration
3. Remove secondary unit designators (apt, suite, unit, floor)
4. Remove punctuation
5. Expand postal abbreviations
6. Collapse whitespace

### 5.2 Postal Abbreviations (US)

| Input       | Output    |
| ----------- | --------- |
| st, st.     | street    |
| ave, ave.   | avenue    |
| blvd, blvd. | boulevard |
| dr, dr.     | drive     |
| rd, rd.     | road      |
| ln, ln.     | lane      |
| ct, ct.     | court     |
| n, n.       | north     |
| s, s.       | south     |
| e, e.       | east      |
| w, w.       | west      |

### 5.3 Secondary Unit Removal

These patterns are removed:

-   Apt, Apt., Apartment + number
-   Suite, Ste, Ste. + number
-   Unit, # + number
-   Floor, Fl + number
-   Room, Rm + number
-   Bldg, Building + identifier

---

## 4. Entity Type Codes

### 4.1 Standard Codes

Entity type codes are uppercase with underscores:

| Code                   | Description                        |
| ---------------------- | ---------------------------------- |
| `SCHOOL_DISTRICT`      | K-12 public school districts       |
| `CHARTER_SCHOOL`       | Charter schools                    |
| `PRIVATE_SCHOOL`       | Private K-12 schools               |
| `UNIVERSITY`           | Higher education institutions      |
| `COMMUNITY_COLLEGE`    | Community/junior colleges          |
| `COUNTY`               | County governments                 |
| `MUNICIPALITY`         | City, town, village governments    |
| `TOWNSHIP`             | Township governments               |
| `SPECIAL_DISTRICT`     | Water, fire, utility districts     |
| `STATE_AGENCY`         | State government agencies          |
| `NONPROFIT_501C3`      | 501(c)(3) organizations            |
| `NONPROFIT_501C4`      | 501(c)(4) social welfare orgs      |
| `PAC`                  | Political Action Committees        |
| `SUPER_PAC`            | Independent expenditure committees |
| `POLITICAL_PARTY`      | Political party committees         |
| `BUSINESS_CORP`        | For-profit corporations            |
| `BUSINESS_LLC`         | Limited liability companies        |
| `BUSINESS_PARTNERSHIP` | Partnerships                       |
| `BUSINESS_SOLE_PROP`   | Sole proprietorships               |
| `TRIBAL_GOVERNMENT`    | Tribal governments                 |
| `UNKNOWN`              | Type cannot be determined          |

### 4.2 Type Inference Rules

When entity type is ambiguous, use these heuristics:

1. Name contains "SCHOOL DISTRICT" or "UNIFIED" → `SCHOOL_DISTRICT`
2. Name contains "CHARTER" + "SCHOOL" → `CHARTER_SCHOOL`
3. Name contains "COUNTY" at end → `COUNTY`
4. Name contains "CITY OF" or "TOWN OF" → `MUNICIPALITY`
5. Name contains "WATER DISTRICT" or "FIRE DISTRICT" → `SPECIAL_DISTRICT`
6. Name ends in "INC" or "CORP" → `BUSINESS_CORP`
7. Name ends in "LLC" → `BUSINESS_LLC`
8. Registered as 501(c)(3) → `NONPROFIT_501C3`
9. Filed with FEC → `PAC` or `SUPER_PAC`

---

## 5. Jurisdiction ISO Codes

### 5.1 Format

Jurisdiction codes follow ISO 3166-2 for US states:

-   Format: `US-XX` where XX is the two-letter state code
-   Example: `US-CA` (California), `US-NY` (New York)

### 5.2 Special Jurisdictions

| Code     | Jurisdiction                   |
| -------- | ------------------------------ |
| `US-DC`  | District of Columbia           |
| `US-PR`  | Puerto Rico                    |
| `US-GU`  | Guam                           |
| `US-VI`  | US Virgin Islands              |
| `US-AS`  | American Samoa                 |
| `US-MP`  | Northern Mariana Islands       |
| `US-FED` | Federal (multi-state entities) |

### 5.3 International Extension

For non-US entities, use full ISO 3166-2:

-   `CA-ON` (Ontario, Canada)
-   `MX-CMX` (Mexico City)
-   `GB-ENG` (England)

---

## 6. EIN Handling

### 6.1 When to Include EIN

Include EIN in the canonical input ONLY when:

1. EIN is from an authoritative source (IRS, state registry)
2. EIN has been verified against the entity name
3. EIN is complete (9 digits)

### 6.2 EIN Format

EINs are normalized to 9 digits without hyphen:

| Input        | Normalized       |
| ------------ | ---------------- |
| "12-3456789" | "123456789"      |
| "123456789"  | "123456789"      |
| "12-345678"  | INVALID (reject) |

### 6.3 Canonical Input with EIN

When EIN is included, it appears first in the canonical input:

```
123456789|NONPROFIT_501C3|US-CA|SPRINGFIELD COMMUNITY FOUNDATION
```

---

## 7. Collision Handling

### 7.1 Expected Collision Rate

SHA-256 provides 256 bits of entropy. With proper normalization,
collision probability is negligible for the expected entity population
(< 10 million unique US civic entities).

### 7.2 Collision Detection

Implementers SHOULD maintain a registry mapping SNFEIs to source records.
When two different source records produce the same SNFEI:

1. If records represent the same entity → SUCCESS (correct merge)
2. If records represent different entities → COLLISION (requires disambiguation)

### 7.3 Disambiguation

If collision is detected between genuinely different entities:

1. Add additional distinguishing field to canonical input
2. Re-hash with extended input
3. Document the disambiguation in the entity record

---

## 8. Version Migration

### 8.1 Version Identifier

SNFEI includes implicit versioning through the CEP schema version.
The normalization rules in this document are for CEP Schema Version 1.0.0.

### 8.2 Rule Changes

If normalization rules change in future versions:

1. New rules get new schema version
2. Old SNFEIs remain valid for records created under old schema
3. Entity records include `schemaVersion` for disambiguation

---

## 9. Reference Implementation

See `crates/cep-core/src/common/snfei.rs` and
`/src/python/src/civic_exchange_protocol/snfei/generator.py` for
canonical implementations.

# Generation
generate_snfei()
generate_snfei_simple()
generate_snfei_with_confidence()
compute_snfei()
Snfei
SnfeiResult

# Normalization
normalize_legal_name()
normalize_address()
normalize_registration_date()
CanonicalInput
build_canonical_input()

# Localization
apply_localization()
get_localization_config()
LocalizationConfig
LocalizationRegistry
LocalizationRule

### 9.1 Test Vectors

| Input                                                            | SNFEI      |
| ---------------------------------------------------------------- | ---------- |
| `SCHOOL_DISTRICT\|US-CA\|SPRINGFIELD UNIFIED SCHOOL DISTRICT 12` | (computed) |
| `MUNICIPALITY\|US-IL\|SPRINGFIELD`                               | (computed) |
| `123456789\|NONPROFIT_501C3\|US-MA\|BOSTON COMMUNITY FOUNDATION` | (computed) |

---

## 10. Security Considerations

### 10.1 Hash Function

SHA-256 is used for its:

-   Collision resistance
-   Wide availability
-   FIPS 180-4 standardization
-   Performance characteristics

### 10.2 Input Validation

Implementations MUST validate:

-   Jurisdiction code format
-   Entity type code is in allowed list
-   EIN format (if provided)
-   Name is non-empty after normalization

### 10.3 Canonicalization Attacks

The normalization pipeline prevents:

-   Case variation attacks
-   Whitespace injection
-   Homoglyph substitution (via ASCII transliteration)
-   Field boundary confusion (via pipe delimiter)

---

## Appendix A: Full Abbreviation Map

```json
{
    "DIST": "DISTRICT",
    "SCH": "SCHOOL",
    "ELEM": "ELEMENTARY",
    "JR": "JUNIOR",
    "SR": "SENIOR",
    "HS": "HIGH SCHOOL",
    "MS": "MIDDLE SCHOOL",
    "USD": "UNIFIED SCHOOL DISTRICT",
    "ISD": "INDEPENDENT SCHOOL DISTRICT",
    "CTY": "COUNTY",
    "TWP": "TOWNSHIP",
    "GOVT": "GOVERNMENT",
    "AUTH": "AUTHORITY",
    "COMM": "COMMISSION",
    "DEPT": "DEPARTMENT",
    "ASSN": "ASSOCIATION",
    "ASSOC": "ASSOCIATION",
    "NATL": "NATIONAL",
    "INTL": "INTERNATIONAL",
    "FED": "FEDERAL",
    "ST": "SAINT",
    "MT": "MOUNT",
    "FT": "FORT",
    "PT": "POINT",
    "AVE": "AVENUE",
    "BLVD": "BOULEVARD",
    "DR": "DRIVE",
    "LN": "LANE",
    "RD": "ROAD",
    "SQ": "SQUARE",
    "TRL": "TRAIL",
    "PKWY": "PARKWAY",
    "HWY": "HIGHWAY",
    "UNIV": "UNIVERSITY",
    "COLL": "COLLEGE",
    "INST": "INSTITUTE",
    "ACAD": "ACADEMY",
    "CTR": "CENTER",
    "CORP": "CORPORATION",
    "CO": "COMPANY",
    "BROS": "BROTHERS",
    "MFG": "MANUFACTURING",
    "SVCS": "SERVICES",
    "SVC": "SERVICE",
    "MGMT": "MANAGEMENT",
    "GRP": "GROUP",
    "HOLDINGS": "HOLDINGS",
    "ENTERPRISES": "ENTERPRISES",
    "PARTNERS": "PARTNERS"
}
```

## Appendix B: Legal Suffix Map

```json
{
    "INCORPORATED": "INC",
    "INCORP": "INC",
    "CORPORATION": "CORP",
    "LIMITED LIABILITY COMPANY": "LLC",
    "LIMITED LIABILITY CO": "LLC",
    "L.L.C.": "LLC",
    "LIMITED PARTNERSHIP": "LP",
    "L.P.": "LP",
    "LIMITED": "LTD",
    "LTD.": "LTD",
    "PROFESSIONAL CORPORATION": "PC",
    "P.C.": "PC",
    "PROFESSIONAL LIMITED LIABILITY COMPANY": "PLLC",
    "P.L.L.C.": "PLLC",
    "COMPANY": "CO",
    "CO.": "CO",
    "PARTNERSHIP": "PARTNERSHIP",
    "GENERAL PARTNERSHIP": "GP",
    "G.P.": "GP"
}
```

## Appendix C: Roman Numeral Conversion

| Roman | Arabic |
| ----- | ------ |
| I     | 1      |
| II    | 2      |
| III   | 3      |
| IV    | 4      |
| V     | 5      |
| VI    | 6      |
| VII   | 7      |
| VIII  | 8      |
| IX    | 9      |
| X     | 10     |
| XI    | 11     |
| XII   | 12     |
| XX    | 20     |
| XXX   | 30     |
| XL    | 40     |
| L     | 50     |
| C     | 100    |

Note: Roman numerals are only converted when they appear as standalone tokens
(word boundaries), not within words.
