****************
Fault Management
****************
Overview
========

Terminology
============

* Event Class
* Event Category
* Event Classification
* Event Correlation
* Root-cause analysys

Stages of event processing
==========================
.. image:: fm_event_processing.png

Collection
----------
Event collection performed by Activator.
Events are generated by Managed Object and transmitted to Activator encapsulated in Message.
Supported message protocols are:

* SNMP Trap
* Syslog

Activator check did message arrived from valid Managed Object, collects message, converts
it to protocol-independed message format and transmits it to the SAE via SAE RPC stream.

Storing
-------
Protocol-independed messages are received by SAE. SAE does not perform
additional message processing and stores messages into Database as soon as possible

Classification
--------------
noc-classifier (Classifier) daemon looks for not yet classified messages in database
and performs message classification. 

Classifier uses Message Classification Rules and regular expressions pattern matching
to determine Event Class for the message. 

When matching rule found Classifier performs:
* Drops event if required by rule, or
* Check, if event repeats already registred event, if required by Event Class, and suppress repeated event, or
* Set ups Event Class
* Set ups Event Category
* Set ups Event Priority
* Resolves Message's SNMP OIDs via MIB Lookup
* Uses Message Classification Rules to extract refined Message Variables
* Communicates with external systems and databases to extract additional event data and performs Event Enrichment
* Fills Event Subject and Body by applying extracted Message Variables to the Subject and Body Templates

Correlation
-----------
Correlation performed by noc-correlator (Correlator) daemon. Correlator works with Event Window (last 10 minutes by default)
and Event Correlation Rules to perform corellation.

During correlation process following decisions made:

* Does Event be silently discarded?
* Does Event is just repeat of known Event in the Event Window?
* Does Event is caused by known Event in the Event Window? (Does Event has Root)
* Does Event causing known Events in the Event Window? (Is Event a Root for other Events)

Decisions are made considering Event Variables, extracted by Correlator.
Root cause decisions are not permanent. Event may become a Root Cause or receive Root Cause with new events.

Message Format
==============
Each message is a two-column table known as Left and Right parts.

For example

======== =============
source   syslog
message  -- MARK --
severity 6
facility 5
======== =============

or

===================== =========================
source                SNMP Trap
1.3.6.1.6.3.1.1.4.1.0 1.3.6.1.6.3.1.1.5.1
1.3.6.1.2.1.1.3.0     7
1.3.6.1.6.3.1.1.4.3.0 1.3.6.1.4.1.4413.2.10
===================== =========================

Event Classification Rules
==========================

Event Correlation Rules
=======================

Forms
=====
Events
------
Permissions
^^^^^^^^^^^
======= ========================================
add     fm | Event | Can add Event
change  fm | Event | Can change Event
delete  fm | Event | Can delete Event
======= ========================================

Setup
=====
MIBs
----
Permissions
^^^^^^^^^^^
======= ========================================
add     fm | MIB | Can add MIB
change  fm | MIB | Can change MIB
delete  fm | MIB | Can delete MIB
======= ========================================

MIB Data
--------
Permissions
^^^^^^^^^^^
======= ========================================
add     fm | MIB Data | Can add MIB Data
change  fm | MIB Data | Can change MIB Data
delete  fm | MIB Data | Can delete MIB Data
======= ========================================

Event Classes
-------------
Permissions
^^^^^^^^^^^
======= ========================================
add     fm | Event Class | Can add Event Class
change  fm | Event Class | Can change Event Class
delete  fm | Event Class | Can delete Event Class
======= ========================================

Event Categories
----------------
Permissions
^^^^^^^^^^^
======= ========================================
add     fm | Event Category | Can add Event Category
change  fm | Event Category | Can change Event Category
delete  fm | Event Category | Can delete Event Category
======= ========================================

Event Priorities
----------------
Permissions
^^^^^^^^^^^
======= ========================================
add     fm | Event Priority | Can add Event Priority
change  fm | Event Priority | Can change Event Priority
delete  fm | Event Priority | Can delete Event Priority
======= ========================================

Classification Rules
--------------------
Permissions
^^^^^^^^^^^
======= ========================================
add     fm | Event Classification Rule | Can add Event Classification Rule
change  fm | Event Classification Rule | Can change Event Classification Rule
delete  fm | Event Classification Rule | Can delete Event Classification Rule
======= ========================================

Correlation Rules
-----------------
Permissions
^^^^^^^^^^^
======= ========================================
add     fm | Event Correlation Rule | Can add Event Correlation Rule
change  fm | Event Correlation Rule | Can change Event Correlation Rule
delete  fm | Event Correlation Rule | Can delete Event Correlation Rule
======= ========================================

Permissions
^^^^^^^^^^^
======= ========================================
preview ANY
======= ========================================

Reports
=======
Loaded MIBs
-----------
Permissions
^^^^^^^^^^^
======= ========================================
preview ANY
======= ========================================

Event Priority Summary
----------------------
Permissions
^^^^^^^^^^^
======= ========================================
preview ANY
======= ========================================

Managed Object Summary
----------------------
Permissions
^^^^^^^^^^^
======= ========================================
preview ANY
======= ========================================

Event Classification Quality
----------------------------
Permissions
^^^^^^^^^^^
======= ========================================
preview ANY
======= ========================================

Profile Summary
---------------
Permissions
^^^^^^^^^^^
======= ========================================
preview ANY
======= ========================================

Event Class Summary
-------------------
Permissions
^^^^^^^^^^^
======= ========================================
preview ANY
======= ========================================

Unclassified Trap OIDs
----------------------
Permissions
^^^^^^^^^^^
======= ========================================
preview ANY
======= ========================================

Permissions
^^^^^^^^^^^
======= ========================================
preview ANY
======= ========================================

