# Glean Pings

A ping is a bundle of related metrics, gathered in a payload to be transmitted.
The ping payload will be encoded in JSON format and contains one or more of the [common sections](#ping-sections) with shared information data.

If data collection is enabled, the Glean SDK provides a set of [built-in pings](./sent-by-glean.md) that are assembled out of the box without any developer intervention.

## Ping sections

Every ping has the following keys at the top-level:

- The [`ping_info` section](#the-ping_info-section) contains core metadata that is included in **every** ping.

- The [`client_info` section](#the-client_info-section) contains information that identifies the client.
  It is included in most pings (including all built-in pings), but may be excluded from pings where we don't want to connect client information with the other metrics in the ping.

- The `metrics` section contains the submitted values for all metric types except for [events](../metrics/event.md).
  It has keys for each of the metric types, under which is data for each metric.

- The `events` section contains the events recorded in the ping.

See the [payload documentation](../../../dev/core/internal/payload.md) for more details for each metric type in the `metrics` and `events` section.

### The `ping_info` section
The following fields are included in the `ping_info` section, for every ping.
Optional fields are marked accordingly.

| Field name | Type | Description |
|---|---|---|
| `seq` | Counter | A running counter of the number of times pings of this type have been sent |
| `experiments` | Object | *Optional*. A dictionary of [active experiments](#the-experiments-object) |
| `start_time` | Datetime | The time of the start of collection of the data in the ping, in local time and with minute precision, including timezone information. |
| `end_time` | Datetime | The time of the end of collection of the data in the ping, in local time and with minute precision, including timezone information. This is also the time this ping was generated and is likely well before ping transmission time. |
| `reason` | String | *Optional*. The reason the ping was submitted. The specific set of values and their meanings are defined for each metric type in the `reasons` field in the `pings.yaml` file. |

All the metrics surviving application restarts (e.g. `seq`, ...) are removed once the application using the Glean SDK is uninstalled.

#### The `experiments` object

This object (included in the [`ping_info` section](#the-ping_info-section)) contains experiment annotations keyed by the experiment `id`. Each annotation contains the experiment `branch` the client is enrolled in and may contain a string to string map with additional data in the `extra` key. Both the `id` and `branch` are truncated to 30 characters.
See [Using the Experiments API](../experiments-api.md) on how to record experiments data.

```json
{
  "<id>": {
    "branch": "branch-id",
    "extra": {
      "some-key": "a-value"
    }
  }
}
```

### The `client_info` section
The following fields are included in the `client_info` section.
Optional fields are marked accordingly.

| Field name | Type | Description |
|---|---|---|
| `app_build` | String | The build identifier generated by the CI system (e.g. "1234/A"). For language bindings that provide automatic detection for this value, (e.g. Android/Kotlin), in the unlikely event that the build identifier can not be retrieved from the OS, it is set to `inaccessible`. For other language bindings, if the value was not provided through configuration, this metric gets set to `Unknown`. |
| `app_channel` | String | *Optional* The product-provided release channel (e.g. "beta") |
| `app_display_version` | String | The user-visible version string (e.g. "1.0.3"). The meaning of the string (e.g. whether semver or a git hash) is application-specific. In the unlikely event this value can not be obtained from the OS, it is set to "inaccessible". If it is accessible, but not set by the application, it is set to "Unknown". |
| `architecture` | String | The architecture of the device (e.g. "arm", "x86") |
| `client_id` | UUID |  *Optional* A UUID identifying a profile and allowing user-oriented correlation of data |
| `device_manufacturer` | String | *Optional* The manufacturer of the device |
| `device_model` | String | *Optional* The model name of the device. On Android, this is [`Build.MODEL`], the user-visible name of the device. |
| `first_run_date` | Datetime | The date of the first run of the application, in local time and with day precision, including timezone information. |
| `os` | String | The name of the operating system (e.g. "linux", "Android", "ios") |
| `os_version` | String | The user-visible version of the operating system (e.g. "1.2.3") |
| `android_sdk_version` | String | *Optional*. The Android specific SDK version of the software running on this hardware device (e.g. "23") |
| `telemetry_sdk_build` | String | The version of the Glean SDK |
| `locale` | String | *Optional*. The locale of the application during initialization (e.g. "es-ES"). If the locale can't be determined on the system, the value is "und", to indicate "undetermined". |

All the metrics surviving application restarts (e.g. `client_id`, ...) are removed once the application using the Glean SDK is uninstalled.

[`Build.MODEL`]: https://developer.android.com/reference/android/os/Build.html#MODEL

## Ping submission

The pings that the Glean SDK generates are submitted to the Mozilla servers at specific paths, in order to provide additional metadata without the need to unpack the ping payload.

A typical submission URL looks like

  `"<server-address>/submit/<application-id>/<doc-type>/<glean-schema-version>/<document-id>"`

where:

- `<server-address>`: the address of the server that receives the pings;
- `<application-id>`: a unique application id, automatically detected by the Glean SDK; this is the value returned by [`Context.getPackageName()`](http://developer.android.com/reference/android/content/Context.html#getPackageName());
- `<doc-type>`: the name of the ping; this can be one of the pings available out of the box with the Glean SDK, or a custom ping;
- `<glean-schema-version>`: the version of the Glean ping schema;
- `<document-id>`: a unique identifier for this ping.

### Limitations

To keep resource usage in check, the Glean SDK enforces some limitations on ping uploading and ping storage.

- **Rate limiting**: only up to 10 ping submissions every 60 seconds are allowed. There are no exposed methods to change these rate limiting defaults yet, follow [Bug 1647630](https://bugzilla.mozilla.org/show_bug.cgi?id=1647630) for updates.
- **Request body size limiting**: the body of a ping request may have up to 1MB. Pings that exceed this size are discarded and don't get uploaded. Size and number of discarded pings are recorded on the internal Glean metric [`glean.upload.discarded_exceeding_pings_size`](../collected-metrics/metrics.md#metrics-1).
- **Storage quota**: Pending pings are stored on disk. The storage directory is scanned every time Glean is initialized and upon scanning Glean checks its size. If this directory exceeds a size of 10MB or 250 pending ping files, pings are deleted to get the directory back to an accepted size. Pings are deleted oldest first, until the directory size is below the quota.
  The number of deleted pings due to exceeding storage quota is recorded on the metric [`glean.upload.deleted_pings_after_quota_hit`](../collected-metrics/metrics.md#metrics-1) and the size of the pending pings directory is recorded (regardless on whether quota has been reached) on the metric [`glean.upload.pending_pings_directory_size`](../collected-metrics/metrics.md#metrics-1)
  **Note** Deletion request pings are stored in a different directory, are not subject to this limitation and never get deleted.

### Submitted headers
A pre-defined set of headers is additionally sent along with the submitted ping:

| Header | Value | Description |
|--------|-------|-------------|
| `Content-Type` | `application/json; charset=utf-8` | Describes the data sent to the server |
| `User-Agent` | Defaults to e.g. `Glean/0.40.0 (Kotlin on Android)`, where `0.40.0` is the Glean SDK version number and `Kotlin on Android` is the name of the language used by the binding that sent the request plus the name of the platform it is running on. | Describes the application sending the ping using the Glean SDK |
| `Date` | e.g. `Mon, 23 Jan 2019 10:10:10 GMT+00:00` | Submission date/time in GMT/UTC+0 offset |
| `X-Client-Type` | `Glean` | Custom header to support handling of Glean pings in the legacy pipeline |
| `X-Client-Version` | e.g. `0.40.0` | The Glean SDK version, sent as a custom header to support handling of Glean pings in the legacy pipeline |
| `X-Debug-ID` | *Optional*, e.g. `test-tag` | Debug header attached to Glean pings when using the [debug tools](../../user/debugging/index.md) |
| `X-Source-Tags` | *Optional*, e.g. `automation, perf` | A list of tags to associate with the ping, useful for clustering pings at analysis time, for example to tell data generated from CI from other data. |
