Metadata-Version: 2.1
Name: realtimekql
Version: 2.2.4
Summary: A module for exploring real-time streams of events
Home-page: https://github.com/microsoft/kqltools
Author: CDOC Engineering Open Source
Author-email: CDOCEngOpenSourceAdm@microsoft.com
License: UNKNOWN
Platform: UNKNOWN
Classifier: Environment :: Console
Classifier: Intended Audience :: End Users/Desktop
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Natural Language :: English
Classifier: Operating System :: Microsoft :: Windows
Classifier: Programming Language :: C#
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Security
Classifier: Topic :: System :: Logging
Classifier: Topic :: System :: Monitoring
Classifier: Topic :: System :: Networking
Classifier: Topic :: System :: Networking :: Monitoring
Classifier: Topic :: System :: Operating System
Classifier: Topic :: System :: Operating System Kernels
Description-Content-Type: text/markdown

# Real-Time KQL for Python

To process data with Kusto Query Language (KQL) queries today, users generally have to upload their data to storage first and then query it. The Kql Tools eliminate this need by processing event streams with KQL queries **as events arrive, in real-time.**



## <a id="Usage">Usage

Real-Time KQL is broken up into three parts: the output, the query, and the input. 

### The Output

Real-Time KQL for Python has a `PythonOutput` class that allows you to customize what happens to events when they are outputted. The simplest usage of the `PythonOutput` class is to instantiate it with no parameters. This will print events to console in JSON format:

```
>>> from realtimekql import *
>>> o = PythonOutput()
```

To customize the output, you can pass in any Python function that takes a dictionary as the only parameter to the `PythonOutput` class. For example, this function stores events in a list to use them later:

```
>>> events = []
>>> def storeEvents(event):
...		events.append(event)
...
>>> from realtimekql import *
>>> o = PythonOutput(storeEvents)
```

The `PythonAdxOutput` class allows you to ingest data to an Azure Data Explorer (Kusto) table through queued ingestion. The class can be instantiated as follows:

```
>>> from realtimekql import *
>>> o = PythonAdxOutput("YourCluster.kusto.windows.net", "YourDatabase", "YourTable", "YourClientId", "YourClientSecret", "YourAuthorityId", resetTable=True)
```



### The Query

You can optionally pass a .kql query into Real-Time KQL to filter, transform, and enrich your events before they even reach the output stage.



### The Input

Real-Time KQL supports various real-time and file input sources. Each input class takes a unique set of arguments, an instance of one of the output classes, as well as an optional path to a query file. This prints real-time Etw TCP events to console in JSON format:

```
>>> from realtimekql import *
>>> o = PythonOutput()
>>> e = EtwSession("tcp", o)
>>> e.Start()
```

Here are all the supported input options and how to use them:

```
EtwSession(sessionName, o, q)
EtlFileReader(filePath, o, q)
WinlogRealTime(logName, o, q)
EvtxFileReader(filePath, o, q)
CsvFileReader(filePath, o, q)
```

The variables `o` and `q` represent the output part and the query part respectively. The query part is optional and can be left out.



