# **Wiliot Tag Tester API** #

WiliotTagTester is the primary way to test Wiliot tag functionality along with data analysis


---

## **Overview**

\
Wiliot Tester API was created in order to test Wiliot tags on every desired
machine.                                                             
The Wiliot Tester API will generate test logs, data CSV files and the whole testing
procedure.                                      
The configuration for our tests is being imported from external Test Suite json file and implemented inside the Tester
API. The output from the API will tell us either the tag that was under test is passed or failed our test configuration.

We will show how to use our Wiliot Tester API, and which inputs are valid, as well as how to interpret the output data.
We will overview here how to use each class of the API, which inputs are required or optional, which outputs we will get
when using the API and how to analyze the results. We will show examples and high-level API flow.


---

\
The Wiliot Tester API is built on 3 different classes.

**Wiliot Tag Tester** - Contains main flow of testing.

**Wiliot Test Log** - Generates log files along with CSV output files with all the data.

**Wiliot Tag Result** - Contains the data analyze functions and Wiliot data structure.



---

## **Wiliot Test Log:**

WiliotTesterLog is python class that generates both log files and CSV output files.

+ Log - contains all the traffic generated by our API, we are creating 3 loggers with different handles:

    * Test log - shows all the data received and how it’s handled

    * GW log - shows all the GW [Gateway] configs, patterns and commands received

    * Result log - shows which test was executed [test input] and the results of the test output.

  All the loggers are written into one .log file

+ CSV files - the API will generate 2 CSV files as output, those files are used for 2 uses:

    1. Data from tests

    2. Serialization of tags - connect ExternalID of tags to UID in cloud.

  The CSV files:

    * Packet_data - shows all the data for filtered [by RSSI] packets received by the GW.

    * Run_data - shows the total run summary.

### **Class Initialization:**

Wiliot Tester Log Class Example:

```python
Logger = WiliotTesterLog(run_name = run_name)
``` 

Declare Wiliot loggers [optional]:

```python
Logger.set_logger(log_path = log_path)
```

Generate CSV files :

```python
Logger.create_data_dir(data_path = log_path, tester_name = 'offline', run_name = run_name)
```

![alt text](C:/Users/Eduard/Desktop/WiliotTesterAPI_Docs/WiliotTestLog_init_table.png "need to be here table 1")

---

## **Wiliot Tag Tester:**

WiliotTesterTagTest is the main class to test Wiliot tags.

The test values are generated from **test_suite** JSON file which looks like this:

```json
{
  "Single Band" : {
    "plDelay": 150,
    "rssiThreshold": 53,
  "maxTtfp": 5,
    "tests":
    [
      {"name":"first_packets_ble_band",
    "rxChannel": 37,
        "energizingPattern": 18,
        "timeProfile": [3,7],
        "absGwTxPowerIndex" : -1,
        "maxTime": 5,
    "stop_criteria": {"num_packets": [3, 99]},
        "quality_param" : {"ttfp" : [0,5]}
      }
    ]
  },
  "Dual Band" : {
    "plDelay": 100,
  "rssiThreshold": 65,
  "maxTtfp": 5,
    "tests": [
      {"name":"first_packets_ble_band",
  "rxChannel": 37,
        "energizingPattern": 18,
        "timeProfile": [5,10],
        "absGwTxPowerIndex" : -1,
        "maxTime": 5,
        "delayBeforeNextTest": 0,
    "stop_criteria": {"num_packets": [1, 99]},
        "quality_param" :
        {
          "ttfp" : [0,5]
        }
      },
      {"name":"sub1G_band",
  "rxChannel": 37,
        "energizingPattern": 52,
        "timeProfile": [5,10],
        "absGwTxPowerIndex" : -1,
        "maxTime": 5,
    "stop_criteria": {"num_packets": [2, 99]},
        "quality_param" :
        {
        }
      }
    ]
  }
}
```

### **Class Initialization:**

-- Table 2 should be here

Wiliot Tester Log Class Example:

```python
WiliotTagTester = WiliotTesterTagTest(
                  selected_test='Single Band', 
                  test_suite = test_suite, stop_event_trig =
                  my_stop_event,
                  logger_name = Logger.logger.name, 
                  logger_result_name = 
                  Logger.results_logger.name, 
                  logger_gw_name = Logger.gw_logger.name, 
                  black_list = black_list, 
                  inlay = InlayTypes.TEO_086.value,
                  tester_name = TesterName.OFFLINE)
```

Class **run** Initialization:

-- Table 3

```python
WiliotResult = WiliotTagTester.run(wait_for_gw_trigger = 5)
```

The WiliotResult is a list that contains all the data from the tag under test- for example if we had 2 stages to the
test (2 different GW configuration or energizing patterns) we will receive list of 2 results (If the tag passed both of
the stages)

We can also add devices to the black list from this class with given advertising address:

```python
WiliotTagTester.add_to_blacklist(<Advertising Address>)
```

---

## **Wiliot Tag Result:**

\
This class contains the output from our tester class and contains all the data required.

`all_packets` → {PacketList class} Packet list of all the packets received by the GW during the stage

`all_tags` → {TagCollection class} Tags list of all the tags that transmitted during the stage

`filtered_tags` → {TagCollection class} The tag that was filtered and selected from all_tags list

`is_test_passed` → {bool} True/False

`run_data` → {dict} GW configuration of the stage {rx_channel , energizing_pattern, time_profile}

`selected_tag` → {PacketList} Selected tag packets organized as sprinkler (same cycle)

`test_end` → {datetime} - Test end time [YY-MM-DD HH:MM:SS]

`test_start` → {datetime}- Test start time [YY-MM-DD HH:MM:SS]

`test_status` → {FailureCodes [enum]} - Status of the tag (Pass/ Fail) and extension of failure

`selected_tag_statistics` → {dict} All the data statistics across all cycles of the selected tag

`FailureCodes.(STATUS)` → Indicates which fail accured or if tag passed

\
\
For example to generate all the packets received by the GW in the first stage we will use:

```python
stage_1_all_packets = WiliotTagTester[0].all_packets
```

To generate all packets from all stages we will have to concatenate the stages:

```python
all_stages_packets = PacketList()
for StageResult in WiliotTagTester:
        stage_packets = StageResult.all_packets
        all_stages_packets.__add__(stage_packets)
```

To get final result if tag has passed or failed:

```python
tag_result = WiliotTagTester.is_all_tests_passed()
```

To get the status code according to FailureCodes:

```python
tag_result_status = WiliotTagTester.get_total_fail_bin()
```

---

## **Different Data Structures**

<br />
<br />

### **Selected Tag Statistics**

\
'num_packets' = {int} \
'num_cycles' = {int} \
'sprinkler_counter_mean' = {float} \
'sprinkler_counter_std' = {float} \
'sprinkler_counter_min' = {int} \
'sprinkler_counter_max' = {int} \
'tbp_mean' = {float} \
'tbp_std' = {float} \
'tbp_min' = {int} \
'tbp_max' = {int} \
'tbp_num_vals' = {int} \
'per_mean' = {float} \
'per_std' = {float} \
'rssi_mean' = {float} \
'rssi_std' = {float} \
'rssi_min' = {int} \
'rssi_max' = {int} \
'ttfp' = {float} \
'end_time' = {float} \
'duration' = {float} \
'rx_rate_normalized' = {float}\
'rx_rate' = {float}
\
<br />
<br />

### **PacketList**

\
Holding all the packets (Packet class), can filter and analyze all packets data.

2 main section for each packet: gw_data and packet_data

* **gw_data**:
    * 'rssi'
    * 'stat_param'
    * 'time_from_start'
    * 'counter_tag'
    * 'is_valid_tag_packet'
      <br />
      <br />


* **packet_data**:
    * 'raw_packet'
    * 'adv_address'
    * 'decrypted_packet_type'
    * 'group_id'
    * 'flow_ver'
    * 'test_mode'
    * 'en'
    * 'type'
    * 'data_uid'
    * 'nonce'
    * 'enc_uid'
    * 'mic'
    * 'enc_payload'

<br />
<br />

### **TagCollection:**

<br />
Hold different tags from the same run, every TagCollection built from PacketList classes, manages to do statistics and analyze data from different PacketList types
<br />
<br />
<br />

### **FailureCodes:**

\
`NONE = 0`\
<br />
`PASS = 1`                              
Pass - default bin when no failure observed\
<br />
`NO_RESPONSE = 2  `                     
No response - no packets received at all in this location\
<br />
`NO_PACKETS_UNDER_RSSI_THR = 3 `        
No packets under RSSI threshold\
<br />
`MISSING_LABEL = 4  `                   \
Missing label - failure due to missing label\
<br />
`FIRST_TAG_IN_RUN = 5 `                 \
First tag in run\
<br />
`HIGH_TTFP = 6  `                       \
Quality Test Failure: High TTFP - time to first packet did not satisfy the configuration file\
<br />
`NOT_ENOUGH_PACKETS = 7  `              
Quality Test Failure: Not enough packets - did not reach packet threshold as defined in configuration file\
<br />
`TBP_AVG_OUT_OF_BOUNDS = 8 `            
Quality Test Failure: TBP Avg out of bounds - TBP Average value is out of bounds\
<br />
`TBP_MIN_OUT_OF_BOUNDS = 9    `         
Quality Test Failure: TBP Min out of bounds - TBP Min value is out of bounds
<br />
`TBP_MAX_OUT_OF_BOUNDS = 10  `          
Quality Test Failure: TBP Max out of bounds - TBP Max value is out of bounds
<br />
`NO_TBP = 11`                           
Quality Test Failure: Could not calculate TBP - could not calculate TBP statistic at all\
<br />
`NOT_ENOUGH_PACKETS_IN_SPRINKLER = 12  `
Quality Test Failure: Not enough packets in sprinkler - received packet count in sprinkler is lower than specified in
configuration file\
<br />
`TIME_BTWN_CYCLES_OUT_OF_BOUNDS = 13 `  
Quality Test Failure: Time between cycles out of bounds\
<br />
`DUPLICATION_OFFLINE = 14 `             
Duplicate offline - tag was found as duplicate during offline tester run (ADVA duplicate)\
<br />
`DUPLICATION_POST_PROCESS = 15 `        
Duplicate post process - tag was found as duplicate in post process (UID duplicate)\
<br />
`GW_ERROR = 16 `                        
GW error
<br />
`STOP_BY_USER = 17 `                    
Test stopped by the user\
<br />
`SEVERAL_TAGS_UNDER_TEST = 18 `         
Cannot decided which tag is under test\
<br />
`FAILED_QUALITY_TEST = 19 `             
Failed Quality Test according to the specified statistics limits.

---

see the api flow [here](docs/tester_api_flow.pdf)



