Metadata-Version: 2.1
Name: myprotosql
Version: 0.0.2
Summary: Read protobuf binary data using vanilla mysql stored functions (protoc plugin)
Home-page: https://github.com/janickr/myprotosql
Author: Janick Reynders
License: GNU Lesser General Public License v3 or later (LGPLv3+)
Keywords: protoc,protobuf,mysql
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: License :: OSI Approved :: GNU Lesser General Public License v3 or later (LGPLv3+)
Classifier: Operating System :: OS Independent
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Requires-Dist: protobuf>=5.27.2

# Myprotosql

A set of mysql stored functions/procedures to read protobuf binary data  

[![Tests](https://github.com/janickr/myprotosql/actions/workflows/tests.yml/badge.svg)](https://github.com/janickr/myprotosql/actions/workflows/tests.yml)
[![PyPi](https://img.shields.io/pypi/v/myprotosql)](https://pypi.org/project/myprotosql/)

## Getting started

### Without `*.proto` files
This is similar to `protoc --decode_raw`.  

Run the `myproto.sql` script on your MySQL DB. This script creates the stored functions and procedures necessary to decode protobuf.

#### Decode to textformat
For example to decode_raw the `0x1a03089601` binary data to textformat:
```mysql
select myproto_decode_to_textformat(0x1a03089601, null, null);
```
Returns:
```prototext
3: {
 1: 150
}
```
#### Decode to JSON
```mysql
select myproto_decode_to_jsonformat(0x1a03089601, null, null);
```
Returns:
```json
{"3": {"1": 150}}
```

#### Limitations of decode_raw
Decode raw has limitations because protobuf binary data does not contain all info to properly decode the data.
- output will not contain field names, only field numbers
- packed repeated scalar values will be decoded as one binary string
- numbers will be decoded as unsigned integers

If you need proper decoding, then read on and learn how to use information in your `*.proto` files

### Using .proto files
The functions and stored procedures in `myproto.sql` are still needed: run the `myproto.sql` script in MySQL.

Let's say we have a `.proto` file like this:
```protobuf
package foo.bar;

message SubMessage {
  optional int32 a = 1;
}

message ParentMessage {
  optional SubMessage c = 3;
}
```
We need to compile these `*.proto` files in something MySQL can understand. 

1) [Download and install](https://github.com/protocolbuffers/protobuf?tab=readme-ov-file#protobuf-compiler-installation) protoc
2) Install the myprotosql protoc plugin (you need python for this): 
    ```bash
    pip install myprotosql
    ```
3) Run the plugin using protoc:  
    `protoc  --proto_path=<the-path-to-your-proto-files> --myprotosql_out=<the-output-path> --plugin=protoc-gen-myprotosql=<the-path-to-the-myprotosql-plugin> <the-path-to-your-proto-files>\*`   
    This will generate a `myproto_descriptors.sql` file.  
    For example:
    - on Windows if you used Virtualenv, with your proto files located in `.\proto` and your virtual env path is `.\venv`:
        ```bash
        protoc.exe  --proto_path=proto --myprotosql_out=build --plugin=protoc-gen-myprotosql=.\venv\Scripts\protoc-gen-myprotosql.exe .\proto\*
        ```
    - on Ubuntu without virtualenv with your proto files located in `./proto`:
        ```bash
        protoc  --proto_path=proto --myprotosql_out=build ./proto/*
        ```
    
4) Run the `myproto_descriptors.sql` script in MySQL, this will create a `myproto_descriptors` 
function which returns the necessary information to decode protobuf data that conforms to the `.proto` files.


#### Decode to textformat

For example to decode_raw the `0x1a03089601` binary data to textformat:
```mysql
select myproto_decode_to_textformat(0x1a03089601, 'foo.bar.ParentMessage', myproto_descriptors());
```
Returns:
```prototext
c: {
 a: 150
}
```
#### Decode to JSON
```mysql
select myproto_decode_to_jsonformat(0x1a03089601, 'foo.bar.ParentMessage', myproto_descriptors());
```
Returns:
```json
{"c": {"a": 150}}
```

```
.\bin\protoc-27.2-win64\bin\protoc.exe  --proto_path=proto --python_out=build --myprotosql_out=build --plugin=protoc-gen-myprotosql=.\venv\Scripts\protoc-gen-myprotosql.exe .\proto\*
```

## Todo
- enums
- todos in code
- maps
- bytes/proper escaping
