# Pings

A ping is a bundle of related metrics, gathered in a payload to be transmitted.
The ping payload is encoded in JSON format and contains one or more of the
[common sections](#payload-structure) with shared information data.

If data collection is enabled, the chosen Glean SDK may provide a set of [built-in pings](./sent-by-glean.md)
that are assembled out of the box without any developer intervention.

## Table of contents

<!-- toc -->

## Payload structure

Every ping payload has the following keys at the top-level:

- The [`ping_info` section](#the-ping_info-section) contains core metadata
  that is included in **every** ping.

- The [`client_info` section](#the-client_info-section) contains information that identifies the client.
  It is included in most pings (including all built-in pings), but may be excluded from pings
  where we don't want to connect client information with the other metrics in the ping.

The following keys are only present if any metrics or events were recorded for the given ping:

- The `metrics` section contains the submitted values for all metric types
  except for [events](../metrics/event.md). It has keys for each of the metric types,
  under which is data for each metric.

- The `events` section contains the events recorded in the ping.

See the [payload documentation](../../../dev/core/internal/payload.md)
for more details for each metric type in the `metrics` and `events` section.

### The `ping_info` section

Metadata about the ping itself.
This section is included in **every** ping.

The following fields are included in the `ping_info` section.
Optional fields are marked accordingly.

#### `seq`

_Type: [Counter](../../reference/metrics/counter.md),
Lifetime: [User](../../reference/yaml/metrics.md#user)_

A running counter of the number of times pings of this type have been sent.

#### `start_time`

_Type: [Datetime](../../reference/metrics/datetime.md),
Lifetime: [User](../../reference/yaml/metrics.md#user)_

The time of the start of collection of the data in the ping, in local time
and with minute precision, including timezone information.

#### `end_time`

_Type: [Datetime](../../reference/metrics/datetime.md),
Lifetime: [Ping](../../reference/yaml/metrics.md#ping-default)_

The time of the end of collection of the data in the ping, in local time and with minute precision,
including timezone information. This is also the time this ping was generated
and is likely well before ping transmission time.

#### `reason` _(optional)_

The reason the ping was submitted. The specific set of values
and their meanings are defined for each metric type in the [`reasons`](../../reference/yaml/pings.md#reasons)
field in the `pings.yaml` file.

#### `experiments` _(optional)_

A dictionary of active experiments.

This object contains experiment annotations keyed by the experiment `id`.
Each annotation contains the experiment `branch` the client is enrolled in
and may contain a string to string map with additional data in the `extra` key.

Both the `id` and `branch` are truncated to 30 characters.
See [Using the Experiments API](../../reference/general/experiments-api.md)
on how to record experiments data.

```json
{
  "<id>": {
    "branch": "branch-id",
    "extra": {
      "some-key": "a-value"
    }
  }
}
```

### The `client_info` section

A limited amount of metrics that are generally useful across products.
The data is provided by the embedding application or automatically fetched by the Glean SDK.
It is collected at initialization time and sent in every ping afterwards.
For historical reasons it contains metrics that are only useful on a certain platform.

{{#include ../../../shared/blockquote-info.html}}

##### Additional metrics require a proposal

> Adding new metrics maintained by the Glean SDKs team will require a full proposal
> and details on why that value is useful across multiple platforms and products and needs Glean SDKs team ownership.
>
> The Glean SDKs are not taking ownership of new metrics that are platform- or product-specific.

The following fields are included in the `client_info` section.
Optional fields are marked accordingly.

#### `app_build`

_Type: [String](../../reference/metrics/string.md),
Lifetime: [Application](../../reference/yaml/metrics.md#application)_

The build identifier generated by the CI system (e.g. "1234/A").
If the value was not provided through configuration, this metric gets set to `Unknown`.

#### `app_channel` _(optional)_

_Type: [String](../../reference/metrics/string.md),
Lifetime: [Application](../../reference/yaml/metrics.md#application)_

The product-provided release channel (e.g. "beta").

#### `app_display_version`

_Type: [String](../../reference/metrics/string.md),
Lifetime: [Application](../../reference/yaml/metrics.md#application)_

The user-visible version string (e.g. "1.0.3").
The meaning of the string (e.g. whether semver or a git hash) is application-specific.
If the value was not provided through configuration, this metric gets set to `Unknown`.

#### `build_date` _(optional)_

_Type: [Datetime](../../reference/metrics/datetime.md),
Lifetime: [Application](../../reference/yaml/metrics.md#application)_

#### `architecture`

_Type: [String](../../reference/metrics/string.md),
Lifetime: [Application](../../reference/yaml/metrics.md#application)_

The architecture of the device (e.g. "arm", "x86").

#### `client_id` _(optional)_

_Type: [UUID](../../reference/metrics/uuid.md),
Lifetime: [User](../../reference/yaml/metrics.md#user)_

A UUID identifying a profile and allowing user-oriented correlation of data.

#### `device_manufacturer` _(optional)_

_Type: [String](../../reference/metrics/string.md),
Lifetime: [Application](../../reference/yaml/metrics.md#application)_

The manufacturer of the device the application is running on.
Not set if the device manufacturer can't be determined (e.g. on Desktop).

#### `device_model` _(optional)_

_Type: [String](../../reference/metrics/string.md),
Lifetime: [Application](../../reference/yaml/metrics.md#application)_

The model of the device the application is running on.
On Android, this is `Build.MODEL`, the user-visible marketing name, like "Pixel 2 XL".
Not set if the device model can't be determined (e.g. on Desktop).

#### `first_run_date`

_Type: [Datetime](../../reference/metrics/datetime.md),
Lifetime: [User](../../reference/yaml/metrics.md#user)_

The date of the first run of the application, in local time and with day precision,
including timezone information.

#### `os`

_Type: [String](../../reference/metrics/string.md),
Lifetime: [Application](../../reference/yaml/metrics.md#application)_

The name of the operating system (e.g. "Linux", "Android", "iOS").

#### `os_version`

_Type: [String](../../reference/metrics/string.md),
Lifetime: [Application](../../reference/yaml/metrics.md#application)_

The user-visible version of the operating system (e.g. "1.2.3").
If the version detection fails, this metric gets set to `Unknown`.

#### `android_sdk_version` _(optional)_

_Type: [String](../../reference/metrics/string.md),
Lifetime: [Application](../../reference/yaml/metrics.md#application)_

The Android specific SDK version of the software running on this hardware device (e.g. "23").

#### `telemetry_sdk_build`

_Type: [String](../../reference/metrics/string.md),
Lifetime: [Application](../../reference/yaml/metrics.md#application)_

The version of the Glean SDK.

#### `locale` _(optional)_

_Type: [String](../../reference/metrics/string.md),
Lifetime: [Application](../../reference/yaml/metrics.md#application)_

The locale of the application during initialization (e.g. "es-ES").
If the locale can't be determined on the system, the value is "und", to indicate "undetermined".

[`Build.MODEL`]: https://developer.android.com/reference/android/os/Build.html#MODEL

## Ping submission

The pings that the Glean SDKs generate are submitted to the Mozilla servers at specific paths,
in order to provide additional metadata without the need to unpack the ping payload.

### URL

A typical submission URL looks like

```
"<server-address>/submit/<application-id>/<doc-type>/<glean-schema-version>/<document-id>"
```

where:

- `<server-address>`: the address of the server that receives the pings;
- `<application-id>`: a unique application id, automatically detected by the Glean SDK; this is the value returned by [`Context.getPackageName()`](http://developer.android.com/reference/android/content/Context.html#getPackageName());
- `<doc-type>`: the name of the ping; this can be one of the pings available out of the box with the Glean SDK, or a custom ping;
- `<glean-schema-version>`: the version of the Glean ping schema;
- `<document-id>`: a unique identifier for this ping.

### Limitations

To keep resource usage in check, the Glean SDK enforces some limitations on ping uploading and ping storage.

#### Rate limiting

Only up to 15 ping submissions every 60 seconds are allowed.

For the JavaScript SDK that limit is higher
and up to 40 ping submissions every 60 seconds are allowed.

#### Request body size limiting

The body of a ping request may have up to 1MB (after compression). Pings that exceed this size are discarded
and don't get uploaded. Size and number of discarded pings are recorded on the internal
Glean metric [`glean.upload.discarded_exceeding_pings_size`](../collected-metrics/metrics.md#metrics-1).

#### Storage quota

Pending pings are stored on disk. Storage is scanned every time Glean is initialized and upon scanning
Glean checks its size. If it exceeds a size of 10MB or 250 pending pings, pings are deleted to get the
storage back to an accepted size. Pings are deleted oldest first, until the storage size is below the quota.

The number of deleted pings due to exceeding storage quota is recorded on the metric
[`glean.upload.deleted_pings_after_quota_hit`](../collected-metrics/metrics.md#metrics-1)
and the size of the pending pings directory is recorded (regardless on whether quota has been reached)
on the metric [`glean.upload.pending_pings_directory_size`](../collected-metrics/metrics.md#metrics-1).

Deletion request pings are not subject to this limitation and never get deleted.

### Submitted headers

A pre-defined set of headers is additionally sent along with the submitted ping.

#### `Content-Type`

Describes the data sent to the server. Value is always `application/json; charset=utf-8`.

#### `Date`

Submission date/time in GMT/UTC+0 offset, e.g. `Mon, 23 Jan 2019 10:10:10 GMT+00:00`.

#### `User-Agent` (deprecated)

Up to Glean v44.0.0 and Glean.js v0.13.0 this contained the Glean SDK version and platform information.
Newer Glean SDKs do not overwrite this header.
See [`X-Telemetry-Agent`](#x-telemetry-agent) for details.

Clients might still send it, for example, when sending pings from browsers
it will contain the characteristic [browser UA string](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/User-Agent#syntax).

This header is parsed by the Glean pipeline and can be queried at analysis time through
the [`metadata.user_agent.*`](https://docs.telemetry.mozilla.org/datasets/pings.html?highlight=metadata#ping-metadata)
fields in the ping tables.

#### `X-Telemetry-Agent`

The Glean SDK version and platform this ping is sent from.
Useful for debugging purposes when pings are sent to [the error stream](https://docs.telemetry.mozilla.org/concepts/pipeline/filtering.html?highlight=error%20stream#querying-the-error-stream).
as it describes the application and the Glean SDK used for sending the ping.

It's looks like `Glean/40.0.0 (Kotlin on Android)`, where `40.0.0` is the Glean Kotlin SDK version number
and `Kotlin on Android` is the name of the language used by the SDK that sent the request
plus the name of the platform it is running on.

#### `X-Debug-Id` _(optional)_

Debug header attached to Glean pings by using the [debug APIs](../../reference/debug/debugViewTag.md),
e.g. `test-tag`.

When this header is present, the ping is redirected to the
[Glean Debug View](../debugging/index.md#glean-debug-view).

#### `X-Source-Tags` _(optional)_

A list of tags to associate with the ping, useful for clustering pings at analysis time,
for example to tell data generated from CI from other data e.g. `automation, perf`.

This header is attached to Glean pings by using the [debug APIs](../../reference/debug/sourceTags.md).
