Metadata-Version: 2.4
Name: mini-siem
Version: 1.0.0
Summary: A pip-installable cross-platform SIEM CLI tool with real-time threat detection
Author-email: Jyoti Kuaner <your-email@example.com>
License: MIT License
        
        Copyright (c) 2025 JYOTIRMAYEE KUANER
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
Project-URL: Homepage, https://github.com/yourusername/mini-siem
Project-URL: Repository, https://github.com/yourusername/mini-siem
Project-URL: Issues, https://github.com/yourusername/mini-siem/issues
Keywords: siem,security,monitoring,threat-detection,log-analysis,cli,cybersecurity,macos
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Information Technology
Classifier: Intended Audience :: System Administrators
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: MacOS
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Security
Classifier: Topic :: System :: Logging
Classifier: Topic :: System :: Monitoring
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: click>=8.1.0
Requires-Dist: colorama>=0.4.6
Requires-Dist: flask>=3.0.0
Provides-Extra: dev
Requires-Dist: pytest>=7.4.0; extra == "dev"
Requires-Dist: pytest-cov>=4.1.0; extra == "dev"
Dynamic: license-file

# 🛡️ Mini SIEM

> A pip-installable, cross-platform Security Information & Event Management (SIEM) tool built in Python.
> Collects real macOS system logs, detects threats in real time, stores everything in a database,
> and displays it all on a password-protected web dashboard.

Inspired by enterprise tools like **Splunk**, **IBM QRadar**, and **Microsoft Sentinel** —
built from scratch using only Python and distributed as a proper CLI package.

---

## ⚡ Install & Run in 3 Commands

```bash
pip install mini-siem
mini-siem init
mini-siem run --demo
```

That's it. No cloning. No manual file execution. Real software distribution.

---

## 📸 What It Looks Like

```
  ╔╦╗╦╔╗╔╦  ╔═╗╦╔═╗╔╦╗
  ║║║║║║║║  ╚═╗║║╣ ║║║
  ╩ ╩╩╝╚╝╩  ╚═╝╩╚═╝╩ ╩
  Cross-Platform Log Monitor & Alert Tool

[1/4] Collecting logs...        → 9 raw entries collected.
[2/4] Parsing & normalising...  → 9 events parsed.
[3/4] Running detection engine...
      → 2 alert(s) suppressed (trusted activity — audit trail preserved)
[4/4] Dispatching alerts...

============================================================
  🚨  SECURITY ALERTS
============================================================
[!!!!!! CRITICAL !!!!!!] THREAT_INTEL_MATCH | Score: 15
  ↳ IP 203.0.113.42 matched threat intelligence list
[!!! HIGH !!!] BRUTE_FORCE | Score: 42
  ↳ 8 failed login attempts in 120s from '198.51.100.7'
============================================================
```

Dashboard at `http://127.0.0.1:5000` — login protected, live charts, attack timeline.

---

## 📁 Project Structure

```
Mini SIEM/
│
├── mini_siem/                  ← Python package (the actual tool)
│   ├── __init__.py
│   ├── __main__.py             ← enables: python -m mini_siem
│   ├── main.py                 ← CLI entry point (all commands live here)
│   └── core/                   ← engine modules
│       ├── __init__.py
│       ├── collector.py        ← macOS log collection
│       ├── parser.py           ← normalisation & validation
│       ├── detector.py         ← 6 detection rules + risk scoring
│       ├── alert.py            ← console, file, email alerts
│       ├── report.py           ← TXT + CSV report generator
│       ├── database.py         ← SQLite persistent storage
│       ├── threat_intel.py     ← malicious IP matching
│       ├── siem_logger.py      ← internal SIEM activity log
│       ├── dashboard.py        ← Flask web dashboard
│       └── whitelist.py        ← false positive suppression
│
├── tests/
│   ├── test_parser.py          ← 20+ parser tests
│   ├── test_detector.py        ← 25+ detection rule tests
│   └── test_database.py        ← 15+ database tests
│
├── data/
│   └── threat_intel.txt        ← known malicious IPs (edit this)
│
├── pyproject.toml              ← package config (how pip install works)
├── Dockerfile                  ← container support
├── requirements.txt            ← dependencies
├── whitelist.json              ← trusted users/IPs (auto-created)
├── LICENSE                     ← MIT
└── README.md
```

**User data** is stored in `~/.mini_siem/` — completely separate from the code:
```
~/.mini_siem/
├── siem.db              ← SQLite database (all events + alerts)
├── threat_intel.txt     ← your threat intel list
├── whitelist.json       ← your trusted users/IPs
└── logs/
    └── siem.log         ← internal SIEM activity log
```

---

## 🍎 Installation on macOS

### Requirements
- Python 3.10 or higher
- macOS Monterey, Ventura, or Sonoma

### Install from source (development)

```bash
# Clone the repo
git clone https://github.com/yourusername/mini-SIEM.git
cd mini-SIEM

# Create and activate virtual environment
python3 -m venv venv
source venv/bin/activate

# Install the package
pip install .

# First time setup
mini-siem init
```

### Shortcut — add to your shell so venv activates automatically

```bash
echo 'alias siem="cd ~/Documents/PROJECTS/Mini\ SIEM && source venv/bin/activate"' >> ~/.zshrc
source ~/.zshrc
```

Now just type `siem` in any terminal window to get started.

---

## 🚀 Daily Usage (macOS)

### Every time you open Terminal

```bash
siem                        # go to project + activate venv
sudo mini-siem run          # collect real macOS logs
mini-siem dashboard         # open web dashboard
```

Then visit `http://127.0.0.1:5000` — login: `admin` / `siem2025`

---

## 📟 All Commands

### `mini-siem init`
First-time setup. Creates `~/.mini_siem/` with database and config files.
```bash
mini-siem init
```

### `mini-siem run` — Analyse logs and fire alerts

```bash
# Real macOS logs (requires sudo)
sudo mini-siem run

# Demo mode — simulated logs, no sudo needed
mini-siem run --demo

# Look back further in time
sudo mini-siem run --hours 48

# Live monitoring — scans every 10 seconds continuously
sudo mini-siem run --live

# Live monitoring with custom interval
sudo mini-siem run --live --interval 30

# Skip saving to database
sudo mini-siem run --no-db

# Skip generating report files
sudo mini-siem run --no-report

# Send email alerts (configure env vars first)
sudo mini-siem run --email
```

### `mini-siem dashboard` — Web dashboard

```bash
mini-siem dashboard                    # uses data from database
mini-siem dashboard --demo             # uses simulated demo data
mini-siem dashboard --port 8080        # use different port
```

### `mini-siem query` — Search the database

```bash
mini-siem query --alerts               # all recent alerts
mini-siem query --ip 203.0.113.42      # events from specific IP
mini-siem query --user admin           # events for specific user
mini-siem query --type FAILED_LOGIN    # filter by event type
mini-siem query --since 2025-01-15     # events after a date
mini-siem query --alerts --limit 10    # limit results
```

### `mini-siem stats` — Summary of everything stored

```bash
mini-siem stats
```

### `mini-siem logs` — View SIEM's own internal log

```bash
mini-siem logs
mini-siem logs --lines 50
```

### Get help anytime

```bash
mini-siem --help
mini-siem run --help
mini-siem query --help
```

### Alternative — run without installing

```bash
python -m mini_siem run --demo
python -m mini_siem dashboard --demo
```

---

## 🔍 Detection Rules

Six independent rules run on every batch of events:

| Rule | What Triggers It | Severity |
|---|---|---|
| `BRUTE_FORCE` | 5+ failed logins from same IP within 120 seconds | HIGH |
| `NEW_IP_LOGIN` | Successful login from a previously unseen IP | MEDIUM |
| `PRIVILEGE_ESCALATION` | Any sudo or privilege escalation event | HIGH |
| `ERROR_SPIKE` | 10+ failures within 60 seconds | MEDIUM |
| `THREAT_INTEL_MATCH` | Source IP found in threat intelligence list | CRITICAL |
| `CRITICAL_RISK_THRESHOLD` | Total system risk score exceeds 20 points | CRITICAL |

---

## 🧠 Risk Scoring

Every event gets a numerical risk score. Scores accumulate to the system total:

| Condition | Points |
|---|---|
| Failed login | +2 |
| Privilege escalation | +5 |
| Account lockout | +4 |
| Explicit credential use | +3 |
| Unknown external IP | +3 |
| Part of brute-force burst | +8 bonus |
| Part of error spike | +6 bonus |
| IP matched threat intel | +10 bonus |

When total score **≥ 20** → CRITICAL alert fires automatically.

---

## 🛡️ Whitelist — False Positive Suppression

Your own legitimate admin activity (like running `sudo`) would normally trigger
PRIVILEGE_ESCALATION alerts. The whitelist suppresses these so you only see real threats.

Your username is added automatically when you run `mini-siem init`.

**Edit the whitelist** at `~/.mini_siem/whitelist.json`:

```json
{
  "trusted_users": ["jyotikuaner", "root"],
  "trusted_ips":   ["127.0.0.1", "::1", "192.168.1.5"],
  "trusted_processes": ["sudo", "python3", "venv"]
}
```

Whitelisted events are still saved to the database — full audit trail is preserved.
They just don't show up as loud alerts.

---

## 🌐 Threat Intelligence

Every source IP is checked against `~/.mini_siem/threat_intel.txt`.
Any match fires a **CRITICAL** alert immediately.

**Add your own malicious IPs** — one per line:

```
# ~/.mini_siem/threat_intel.txt
203.0.113.42        # known scanner
198.51.100.7        # brute force bot
185.220.101.0/24    # tor exit nodes (CIDR block)
```

CIDR blocks (like `/24`) are supported — matches entire subnets.

In production, replace with real threat feeds:
- [AbuseIPDB](https://www.abuseipdb.com)
- [Emerging Threats](https://rules.emergingthreats.net)
- [Spamhaus DROP list](https://www.spamhaus.org/drop/)

---

## 💻 Web Dashboard

```bash
mini-siem dashboard --demo
```

Visit `http://127.0.0.1:5000` — login: `admin` / `siem2025`

**Features:**
- Login page — password protected
- 7 KPI cards — failed logins, escalations, critical alerts, risk score, threat intel hits
- Attack timeline — 5-minute bucket chart, bars turn red during burst attacks
- Failed vs successful logins by hour
- Top suspicious IPs chart
- Top targeted users chart
- Alert severity distribution
- Colour-coded alerts table
- Live SIEM internal log viewer
- REST API — `/api/events` and `/api/alerts` return JSON

**Change the dashboard password** — open `mini_siem/core/dashboard.py`:
```python
ADMIN_USERNAME = "admin"
ADMIN_PASSWORD = "siem2025"    # change this
```

---

## 💾 Database

All events and alerts are stored permanently in `~/.mini_siem/siem.db` (SQLite).
Data builds up over time — your history is always there.

```python
from mini_siem.core.database import query_events, query_alerts, get_top_ips

# Find all events from a suspicious IP
events = query_events(ip="203.0.113.42")

# Find critical alerts from last week
alerts = query_alerts(severity="CRITICAL", since="2025-01-08")

# Top 10 attacking IPs all time
top = get_top_ips(10)
```

Clean up old data:
```python
from mini_siem.core.database import clear_old_data
clear_old_data(days=30)
```

---

## 📊 Output Files

After each `run`, files are saved to `~/.mini_siem/reports/`:

| File | Description |
|---|---|
| `alerts.txt` | All alerts, human-readable |
| `security_report.txt` | Full report with hourly activity chart |
| `security_events.csv` | All events — open in Excel or Numbers |
| `security_alerts.csv` | All alerts — open in Excel or Numbers |

---

## 📧 Email Alerts (Optional)

```bash
export SIEM_SENDER="you@gmail.com"
export SIEM_PASSWORD="your_app_password"
export SIEM_RECIPIENT="alerts@email.com"
export SIEM_MIN_SEVERITY="HIGH"

sudo mini-siem run --email
```

Gmail requires an App Password — generate at:
Google Account → Security → 2-Step Verification → App Passwords

---

## 🧪 Running Tests

```bash
python -m pytest tests/ -v
```

```bash
# Run individual test files
python -m pytest tests/test_detector.py -v
python -m pytest tests/test_parser.py -v
python -m pytest tests/test_database.py -v
```

```bash
# Run with coverage report
pip install pytest-cov
python -m pytest tests/ --cov=mini_siem --cov-report=term-missing
```

---

## 🐳 Docker

```bash
# Build
docker build -t mini-siem .

# Run dashboard
docker run -p 5000:5000 mini-siem

# Keep data between restarts
docker run -p 5000:5000 -v ~/.mini_siem:/root/.mini_siem mini-siem
```

---

## 🍎 macOS Log Collection

Real log collection uses two macOS sources:

**`log show` command** — macOS unified logging. Captures authentication
failures, SSH events, sudo usage, and privilege escalation.

**`/var/log/system.log`** — Traditional syslog fallback.

**Sudo is required for real logs:**
```bash
sudo mini-siem run
```

Without sudo, the tool automatically falls back to demo mode.

**What gets detected from real macOS logs:**
- SSH brute-force attempts
- Failed password events
- Sudo / privilege escalation
- Invalid user login attempts
- Account lockouts

**Generate real test events:**
```bash
# In a second Terminal window
ssh wronguser@localhost      # creates failed login entries
sudo ls                      # creates privilege escalation entry
sudo whoami
```

Then scan:
```bash
sudo mini-siem run --hours 1
```

---

## 📦 Building for Distribution

```bash
pip install build
python -m build
```

Creates in `dist/`:
- `mini_siem-1.0.0.tar.gz` — source distribution
- `mini_siem-1.0.0-py3-none-any.whl` — installable wheel

Install from wheel:
```bash
pip install mini_siem-1.0.0-py3-none-any.whl
```

Publish to PyPI (optional):
```bash
pip install twine
twine upload dist/*
```

Then anyone in the world can install with:
```bash
pip install mini-siem
```

---

## 🔧 Troubleshooting

**`mini-siem: command not found`**
```bash
source venv/bin/activate
```

**Port 5000 already in use**
```bash
mini-siem dashboard --port 8080
# visit http://127.0.0.1:8080
```

**Zero events collected (no sudo)**
```bash
sudo mini-siem run
```

**Zero events even with sudo**
- Go to System Settings → Privacy & Security → Full Disk Access
- Add Terminal and enable it
- Restart Terminal and try again

**Database issues**
```bash
rm ~/.mini_siem/siem.db
mini-siem init
```

**ModuleNotFoundError after code changes**
```bash
pip uninstall mini-siem -y
pip install .
```

---

## 📚 Concepts This Project Demonstrates

| Concept | Where |
|---|---|
| pip package distribution | `pyproject.toml`, `mini_siem/` structure |
| CLI design | Click framework, `mini_siem/main.py` |
| macOS log collection | `core/collector.py` |
| Log parsing & normalisation | `core/parser.py` |
| Rule-based threat detection | `core/detector.py` |
| Risk scoring | Numerical danger scores per event |
| False positive suppression | `core/whitelist.py` |
| SQLite persistence | `core/database.py` |
| Threat intelligence | `core/threat_intel.py`, CIDR matching |
| Web dashboard + auth | `core/dashboard.py`, Flask sessions |
| Internal logging | `core/siem_logger.py`, log rotation |
| Email alerting | `core/alert.py`, smtplib |
| Unit testing | `tests/`, pytest, 60+ test cases |
| Docker containerisation | `Dockerfile` |

---

## 📄 License

MIT License — see [LICENSE](LICENSE) for full text.

Free to use, modify, and distribute. Keep the copyright notice.

---

*Built as a portfolio project demonstrating defensive security engineering.*
*Inspired by Splunk, IBM QRadar, and Microsoft Sentinel.*
