Jsonnet - The Data Templating Language
Description: Elegantly and easily describe JSON data for a range of applications.
Keywords: JSON, parser, language, configuration, configuration management, configuration language, build, config, functional, declarative, lazy, structured, elegant, semantics, clean, mixins, inheritance, template, expansion, expand

The data templating language
Jsonnet

1. Home
2. Design
   Objectives Rationales Comparisons Specification
3. Learning
   Tutorial Demo Standard Library
4. Implementation
   Cmdline Tool Libraries C++ Internals Javascript Tests
5. Contributing

# Objectives

JSON has emerged as the defacto standard for communication of structured data, both between machines and at the human / machine boundary. However, in large quantities JSON can be unwieldy for humans, with duplication that needs to be kept in sync between different parts of the data structure. Many people address this by writing scripts that generate JSON, but typically these are written in general purpose programming languages like Python where maintaining them can be non-trivial.

Jsonnet attempts to solve this problem in a specialized and principled way, according to the following criteria:

- Hermiticity: Code may be treated as data. The same JSON should be generated regardless of the environment. There should be no non-deterministic or system-dependent behaviors.
- Templating Language: Code should be interleaved with verbatim data so that it is easy to maintain the two in synchrony. The success of template languages are proof of this approach's effectiveness.
- Familiarity: Raw data should be specified as JSON. Computation constructs should behave predictably.
- Powerful yet simple: Trivial cases should be trivial. More complex cases should be approachable with additional complexity being localized. Everything must be possible, yet the language must still have a small footprint for learning and building new tools.
- Modularity: As configurations grow, it must be possible to manage the complexity using standard techniques for programming in the large.
- Reuse: The same language should be usable on a wide variety of systems, ideally all things should be controlled via a well-maintained centralized configuration.
- Rigor: Different implementations of the language must be possible, via an authoritative specification that is complete and simple enough to understand. The language should not be defined by implementation.

## General Design

The language is designed to be implementable via desugaring, i.e. there is a simple core language and the other constructs are translated down to this core language before interpretation. This technique allows a language to have considerable expressive power, while remaining easy to implement.

## Modularity and Encapsulation

In Jsonnet, a module is typically a Jsonnet file that defines an object whose fields contain useful values, such as functions or objects that can specialized for a particular purpose via extension. Using an object at the top level of the module allows adding other fields later on, without having to alter user code. When writing such a module, it is advisable to expose only the interface to the module, and not its implementation. This is called encapsulation, and it allows changing the implementation later, despite the module being imported by many other Jsonnet files.

Jsonnet's primary feature for encapsulation is the `local` keyword. This makes it possible to define variables that are visible only to the module, and impossible to access from outside. The following is a simple example. Other code can import `util.jsonnet` but will not be able to see the `internal` object, and therefore not the function `square`.

    // util.jsonnet
    local internal = {
        square(x):: x*x,
    };
    {
        euclidianDistance(x1, y1, x2, y2)::
            std.sqrt(internal.square(x2-x1) + internal.square(y2-y1)),
    }

It is also possible to store `square` in a field, which exposes it to those importing the module:

    // util2.jsonnet
    {
        square(x):: x*x,
        euclidianDistance(x1, y1, x2, y2)::
            std.sqrt(self.square(x2-x1) + self.square(y2-y1)),
    }

This allows users to redefine the square function, as shown in the very strange code below. In some cases, this is actually what you want and is very useful. But it does make it harder to maintain backwards compatibility. For example, if you later change the implementation of `euclidianDistance` to inline the square call, then user code will behave differently.

    // myfile.jsonnet
    local util2 = import "util2.jsonnet" { square(x):: x*x*x };
    {
        crazy: util2.euclidianDistance(1,2,3,4)
    }

In conclusion, Jsonnet allows you to either expose these details or hide them. So choose wisely, and know that everything you expose will potentially be used in ways that you didn't expect. Keep your interface small if possible.

A common belief is that languages should make `local` the default state, with an explicit construct to allow outside access. This ensures that things are not accidentally (or apathetically) exposed. In the case of Jsonnet, backwards compatibility with JSON prohibits that design since in JSON everything is a visible field.

## Rationale for Turing-Completeness

Jsonnet is Turing-complete as it is possible to write non-terminating programs. Configurations should always terminate, so ideally we would prevent this. However doing so in general is impossible and even when constrained to practical use cases, it remains impractical. Typical approaches for enforcing termination either restrict the language (e.g. to primitive recursion), which makes some programs impossible / difficult to write, or, alternatively, require the programmer to provide evidence that the program terminates, via some sort of annotation / energy function. Both of these make the programmer's life more difficult.

Furthermore, non-Turing complete languages can take arbitrary CPU or RAM by running intensive algorithms on large input. So enforcing termination is firstly not practical, and secondly does not actually solve the more practical problem of bounding resource consumption during execution.

For Jsonnet we decided restricting termination would create more problems than it would solve.

## Rationale for Dynamic Typing

Programmers do not like bureaucracy. Typically type systems either require annotations, use type inference, or are dynamic. The former two approaches have two advantages: Some errors can be detected earlier (conservatively -- some correct programs are also rejected). Also, they can be implemented more efficiently since the additional knowledge about the program means special memory representations can be used, and also some instructions can be elided. However annotations are additional bureaucracy and type inference produces unification errors that the programmer has to understand and fix.

Dynamic typing means checking types at run-time, and raising errors via the language's existing error reporting mechanism. This is conceptually much simpler for the programmer. In a configuration language, runtime performance is not as important as other languages, and additionally programs tend to execute quickly so testing is much easier.

For Jsonnet we decided that simplicity was far more important than the minimal benefits of runtime efficiency and static error detection in this context.

## Rationale for Object-Oriented Semantics

The object oriented features are ideal for deriving variants from templates. object-based mixins

## Rationale for Pure Functional Semantics

Since it has no side-effects, the JSON can be regenerated at any time, and since it is deterministic, this will always result in the same output JSON. Functional languages are ideal for refining and processing structured data.

## Rationale for Lazy Semantics

The late binding from the object oriented semantics already embodies some of the features of a lazy language. Errors do not occur unless a field is actually dereferenced, and cyclic structures can be created. For example the following is valid even in an eager version of the language:

    local x = {a:"apple", b:y.b},
          y = {a:x.a, b:"banana"};
    x

It would therefore be confusing if the following was not also valid, which leads us to lazy semantics for arrays.

    local x = ["apple", y[1]],
          y = [x[0], "banana"];
    x

Therefore, for consistency, the whole language is lazy. It does not harm the language to be lazy: Performance is not significantly affected, stack traces are still possible, and it doesn't interfere with I/O (because there is no I/O). There is also a precedent for laziness, e.g. in Makefiles and the Nix expression language.

Arguably, laziness brings real benefits in terms of abstraction and modularity. It is possible to build infinite data-structures, and there is unrestricted beta expansion. For example, the following 2 snippets of code are only equivalent in a lazy language.

    if x == 0 then 0 else if x > 0 then 1 / x else -1/x

    local r = 1 / x;
    if x == 0 then 0 else if x > 0 then r else -r

## Rationale for string formatting via % operator

## Rationale for && and || operators

Except as noted, this content is licensed under Creative Commons Attribution 2.5.