# What is Jupyter?

![The JupyterLab notebook editor, with an open notebook showing an interactive view of the insulin receptor](_static/_images/notebook_insulin_lab_small.png)
**Pictured above:** *A notebook inside JupyterLab, showing an interactive*
*view of the insulin receptor.*

> 📘 **About:** This document explains fundamental ideas and software inside the Jupyter ecosystem.

"Jupyter" can mean a lot of things, so let's start from the beginning and
go step-by-step:

Above all, "Project Jupyter" is a large umbrella project that covers
many different software offerings and tools. That includes [Jupyter Notebook](https://jupyter-notebook.readthedocs.io/en/latest/)
and [JupyterLab](https://jupyterlab.readthedocs.io/en/latest/),
which are both popular notebook-editor programs. **The Jupyter project, and**
**its subprojects, all center around providing tools (and standards) for**
**interactive computing with computational notebooks.** If you're not familiar
with some of those things, you can take a guided tour below that explains each
one from the start.

The term "Jupyter" is often used as a shorthand to refer to one of those
products or ideas (which might cause confusion). Let's take a look at each piece,
and provide some clarity, below.

# What is a "Computational Notebook" anyway?

A famous computer programmer ([Donald Knuth](https://en.wikipedia.org/wiki/Donald_Knuth))
popularized the idea to combine explanatory plain english text with computer
code, which is commonly called [literate programming](https://en.wikipedia.org/wiki/Literate_programming).
**By adopting this practice, computer programs, as well as other complex**
**information and ideas, could be better explained to a wide range of people.**

A program written in this way could be printed on paper and shared by hand
as an actual "notebook", but in modern times, they are shared digitally as
"notebook files", and can contain additional rich media like images, 3D models
and interactive figures, along with data and other program outputs.

By opening a notebook with an editor program like [JupyterLab](https://jupyterlab.readthedocs.io/en/latest/),
you can also run the code inside the notebook. Since a notebook can contain
code that does virtually anything, you can do nearly anything regular software
can do inside of a notebook. For example, a notebook file might be used to:

- Read spreadsheets (or create them) to build reports about your household spending
- Show proof of your college thesis with interactive graphs and source data
- [Process astronomical data](https://jwst-docs.stsci.edu/jwst-science-calibration-pipeline/jwst-pipeline-notebooks#gsc.tab=0)
  from the James Webb Space Telescope (JWST)
- Generate an [image of a black hole](https://numfocus.org/case-studies/first-photograph-black-hole)
  by processing telescope data
- Calculate the presence of [gravitational waves]((https://blog.jupyter.org/congratulations-to-the-ligo-and-virgo-collaborations-from-project-jupyter-5923247be019))
  from observatory data

Those last three are real world examples that demonstrate the scientific
community's usage of computational notebooks. Because scientists,
engineers, and other technical people so often need to perform and communicate
sophisticated calculations to describe their work, these "computational
notebooks" became a very popular way to share their work and ideas.

Notebooks are not just for NASA scientists either: Students, hobbyists, and
business people commonly use the fast, interactive workflows in JupyterLab to
make notebooks that help solve everyday problems. You can use them to explain
your own ideas to others, to learn, to automate tasks at home or at work,
visualize complex information, and more.

When someone uses the term "notebook", they might be referring to:

- A notebook file on their computer
- The idea of combining computer code, explanatory text, images and more into
  the "notebook format"
- The "Jupyter Notebook" application, used to author and edit digital notebook
  files
- Jupyter's ``.ipynb`` notebook file format (used to save your notebook files
  on your computer), which is interpreted by the ``nbformat`` software library

And the term "Jupyter" might refer to:

- "Project Jupyter", the overarching umbrella project
- The "Jupyter Notebook" or "JupyterLab" editing programs (or other Jupyter
  products)

The name Jupyter comes from the three programming languages the project
originally supported: Julia (ju), Python (pyt) and R (r).

# How do the Jupyter notebook-editor programs work?

It is common for notebook-editor programs like JupyterLab, or Jupyter
Notebook, to share some features and workflows, because they are influenced
by a common set of ideas about computational notebooks, their advantages,
and how best to work with them effectively.

Let's take a look at some of those ideas.

## Interactive programming (the REPL)

In the past, writing programs, running them, and seeing results was commonly
a slower and more deliberative process than it is today.

As time passed, new techniques were discovered that sped up the process
between writing code, running it, and seeing results. Languages like [Python](https://en.wikipedia.org/wiki/Python_(programming_language))
were also later introduced that offered some unique advantages over
older languages.

For instance, Python is an interpreted language, so it does not need to be [compiled](https://en.wikipedia.org/wiki/Compiler)
before it can be run (saving programmers steps and time). It is also a
dynamically-typed language that does not require the programmer to specify
the type of their data ahead of time before using it, which can sometimes
save time and reduce the complexity of code (particularly for smaller,
simpler programs).

Another programming technique (which is key to the Jupyter notebook-editing
programs) is the Read-Eval-Print-Loop ([REPL](https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop)),
which allows a programmer to interactively write code, run it, keep their
data and variables intact, then rewrite, re-run, and refine their code
on-the-fly (without losing data after the program finishes running).

A REPL is so named because the programmer writes snippets of code that are
first read (R), then Evaluated (E) or in other words executed, the results
are printed (P) to some kind of display or output, and that process happens
repeatedly in a loop (L), where the REPL waits until the programmer has
another snippet of code to execute.

New snippets of code can refer to variables defined in previous Eval steps,
because the REPL keeps objects that were created (by previous runs of the
loop) in-memory, until the user closes their REPL.

## Kernels

Notebook editor programs like JupyterLab create a REPL (read more about those
above) for each of your open notebook files, in a language of your choice. In
the Jupyter ecosystem, each of these REPL's is called a kernel, and it holds
the data and objects you create with your notebook code in a long-running
program on your computer (the management and creation of these kernels
is orchestrated by the [Jupyter Server](https://jupyter-server.readthedocs.io/en/latest/#who-s-this-for),
more on that later).

This is why you don't lose your data and variables when you execute multiple
notebook cells in a row, they stay alive inside the REPL so that you can
continue to use them to explore the data and problem you are investigating
(though if you shut down the Jupyter server that holds the kernels, you will
lose those variables).

The kernels that run in the background for each of your notebooks are what
power the fast, exploratory programming workflows that Jupyter notebook
editor programs excel at.

## Multiple programs, one experience (client-server architecture)

Most Notebook editor programs in Project Jupyter, like JupyterLab, may seem like
a single experience, but when you run JupyterLab on your laptop, there are
actually several programs running independently that all coordinate with
each other to offer you the notebook editing experience you may already be
familiar with.

This might seem like an unnecessary background detail, but some clarity on
the inner workings can help demystify your notebook authoring experience, and
open up new features, ideas and workflows that you can take advantage of.

A many-pieces (modular) approach also has advantages for people using Jupyter
software, and to the people who write it.

Let's look at some of the pieces:

- JupyterLab's interface is a web app that you open in a browser program
  like FireFox or Google Chrome (websites themselves commonly have code
  that defines their behavior, and JupyterLab's interface is no exception).

- [Jupyter Server](https://jupyter-server.readthedocs.io/en/latest/#who-s-this-for)
  runs in the background, and it orchestrates the creation, management of, and
  communication with, your notebook kernels while you're running your notebooks.

- Jupyter kernels themselves are independent programs (operating system
  processes), and each one is its own REPL in whatever language you
  requested (commonly Python, though hundreds of other languages and
  kernels are [also available](https://docs.jupyter.org/en/latest/projects/kernels.html)).

With this design, any program (that you enable) can talk to your kernels
using common internet communication technologies like HTTP. This gives
you the power to edit, view, share, and manipulate your data across many
different programs. One kernel can even connect to multiple editing programs
simultaneously! (Similarly, in JupyterLab, you can connect a notebook and a
console to the same kernel!)

Project Jupyter actually defines a [standard](https://jupyter-protocol.readthedocs.io/en/latest/messaging.html)
that other programs can follow that will allow them to hook into your
kernels in virtually any way you can imagine. You can invent new editing
and viewing experiences for your data this way, using the interactive
computing capabilities provided by the kernels.

## Benefits of a many-piece design

By breaking up a program like JupyterLab into multiple component pieces, you
can customize the software to meet your needs. If one piece is missing something,
you can replace it with a custom version made by yourself or another person to
add whatever features you would like to see.

You can also invent completely new experiences using those pieces that the
designers may not have imagined when they started, and they can often inter-
operate seamlessly with existing Jupyter software.

Because Jupyter Server provides kernel communication and management features
in a cohesive, self-contained package, for instance, new notebook editor
programs can focus solely on adding new interface and editing experiences,
leaving the task of creating and managing new kernels completely up to the
Jupyter Server.

[JupyterCAD](https://jupytercad.readthedocs.io/en/latest/) is a 3D modeling
tool built on top of Jupyter software, for instance, and there are
[other examples too](https://github.com/glue-viz/glue-jupyter).

![The JupyterCAD program](_static/_images/jupytercad-screenshot.jpg)
**Pictured above:** *The JupyterCAD program.*

## Talking to Kernels (The Jupyter Protocol)

Anyone can make new software that talks to Jupyter's kernels (read more
about those above), by using the [Jupyter Protocol](https://jupyter-client.readthedocs.io/en/latest/messaging.html).
The protocol provides a standardized blueprint for passing information
back and forth between the kernels and other software.

By implementing the designs described in the Jupyter Protocol, you could
invent a completely new interactive programming experience, or add support
for a new programming language to Jupyter.

Because Project Jupyter is free and open, it encourages anyone to explore
new ways of working with their notebooks and kernels, and likes to offer
compatibility and interoperation with other software.

# What else should I know about Project Jupyter?

Jupyter software is free and open-source, developed by a global community
of volunteers and contributors, available for the benefit of all.

Project Jupyter welcomes people from all backgrounds and with many types
of skills (not just software!) so we encourage you to join us! You can
share feedback about your experiences directly with the people who make
Jupyter software, or volunteer and contribute to help Jupyter in many
different ways, like:

- Testing newly released versions
- Writing tutorials
- Adding software features
- Hosting the [weekly video meetings](https://jupyter.org/community#live-events)
- Helping others in the community
  - Ask for help or give help at the [Jupyter Discourse Forums](https://discourse.jupyter.org/)
- ...and more!

The project is split into largely independent subprojects which handle
different aspects of Jupyter software and the community. A central council,
the [Jupyter Executive Council](https://jupyter.org/governance/executive_council.html),
makes decisions about project-wide goals and policies, while different
subprojects handle the actual development of the various software components.

Some subprojects take care of broader topics, such as the [Accessibility](https://jupyter-accessibility.readthedocs.io/en/latest/),
[Security](https://jupyter.org/security), [Community](https://docs.jupyter.org/en/latest/community/content-community.html), and [Documentation](https://github.com/jupyter/docs-team-compass) projects.

# A (Partial) Tour of the Jupyter Universe

In the following sections, we are going to look at some popular components of
the Jupyter ecosystem. This is *not* a comprehensive reference of every aspect
of Jupyter, but rather a big-picture summary that should help illustrate some
important parts that were discussed earlier.

If you are completely new to *Jupyter*, the project's [About](https://docs.jupyter.org/en/latest/#sub-project-documentation)
page is good reading that will introduce you to many of the different subprojects.

If you want to know more about the organizational structure, check out the
[Governance](https://jupyter.org/governance/overview.html) pages.

And if you would like to contribute to the project, have a look at the
[Get Involved](https://jupyter.org/community#join-the-jupyter-community) page.

The graph below presents the best known software components of the Project.

Connections and groups in this diagram are not formal relationships, but
simple indicators to help us draw the big picture in the next sections.

![Diagram of well known Jupyter software](_static/_images/jupyter_software_diagram_alpha.png)

### Notebook Editor Programs

- The [JupyterLab](https://jupyterlab.readthedocs.io/en/latest/) notebook editor
- [Jupyter Notebook](https://jupyter-notebook.readthedocs.io/en/stable/) notebook editor

Two popular notebook-editor programs made by Project Jupyter are Jupyter-Notebook and
JupyterLab, though there are others. Which one you choose to use is
mostly a matter of preference (both Notebook and Lab provide many of the
same capabilities for editing and running notebooks):

- Jupyter Notebook offers a simplified, lightweight notebook authoring
  experience

- JupyterLab offers a more customizable, feature-rich, tabbed multi-notebook
  editing environment, and has additional tools like a customizable layout
  and system console (a common tool used by programmers)

- And more!

A professor, for instance, might use JupyterLab for their daily work as a data
analyst, and Jupyter Notebook while teaching to provide a simpler, more focused
view of the notebook's content to their students.

### ipython

- [The IPython REPL](https://ipython.readthedocs.io/en/stable/index.html)

A long time ago, a precursor to Jupyter was created, called `ipython` (A
soon to be released document "The History of Jupyter" will cover this in
more detail). It provides advanced REPL features that are still used today
in Jupyter software, as it is the default Python kernel bundled with
JupyterLab and Jupyter Notebook.

`ipython` makes some additional commands available that you can type directly
alongside your Python code in your notebook cells. Read more about some of
these features below:

- [Interactive shell](https://ipython.readthedocs.io/en/stable/interactive/tutorial.html)
- [Magic commands](https://ipython.readthedocs.io/en/stable/interactive/magics.html)

For more details on IPython features check [`docs.jupyter.org > Projects > IPython`](https://docs.jupyter.org/en/latest/projects/ipython_projects.html).

### The notebook format

- [The nbformat software library](https://nbformat.readthedocs.io)

Jupyter uses the `.ipynb` file format to store notebook files on your computer
(where the data and outputs of your notebook reside for long term storage).
The .ipynb format is interpreted and modified by the nbformat software library.

The `.ipynb` format is a type of json document, which holds your notebook cells,
program outputs, and metadata inside a structured text file.

### JupyterHub

- [JupyterHub for multi-user setups](https://jupyterhub.readthedocs.io)

JupyterHub provides a multi-user management system where many different people
can log in and use their own isolated notebook editor program and environment.

With JupyterHub, editing and running notebook files is still performed by an
editor program like Lab or Notebook. The Hub is responsible for authenticating
users and providing them their corresponding Lab/Notebook instance connection.

JupyterHub can be set up in different system configurations:

- In a single computer
- In a cluster of computers
- In containers in the cloud

JupyterHub is quite flexible for many different multi-user scenarios.

### Interactive Buttons, Sliders and more with ipywidgets

- [Jupyter Widgets (AKA ipywidgets)](https://ipywidgets.readthedocs.io/en/latest/)

Jupyter Widgets (also called ipywidgets) provides interactive interface
elements that you can add directly into your notebooks.

> 📘 **Note:** The subproject was renamed to Jupyter-Widgets, but Jupyter-Widgets
> and IPyWidgets are actually the same thing. To avoid breaking compatibility with
> older code, the software library is still called `ipywidgets`.
