Python_Jupyter_Introduction

# Welcome to the Python using Jupyter Tutorial!¶

(Python Version 2.7, 07/05/2016)
This tutorial is aimed at people who wish to use python with jupyter. If working on a computer that does not have python or jupyter installed, read and start with What you Need to get Started and Installing and Loading Python.

The best way to use this tutorial is to download the jupyter notebook and enter code and comments as you progress.

## Table of Contents¶

1. ### What You Need to get Started

2. ### Installing and Loading Python

3. ### Beginning a Jupyter Session

4. ### Basic Python Syntax

5. ### Basic Python Types

6. ### Opening and Reading Files

7. ### Defining a Function

8. ### Creating a Class

9. ### Basic Plotting

10. ### Intermediate Plotting

11. ### Using Jupyter's Markdown Cells to Document Code

12. ### Basic Fitting

13. ### Analysis of a Fit

14. ### Basic Instrument Control

15. ### More Information

</ol>

back to top

### What you Need to get Started

1. A computer running one of the following Operating Systems (OS): Windows 7 or greater, Mac OSX, or a unix flavor from the last 5 years.
2. An internet connection.
3. Some time and patience. It takes some effort to learn a new language.

In [1]:

    # If using jupyter right now, put the cursor inside of this box and press shift-enter to see how it works
    import matplotlib.pyplot as plt

    # Make a data set
    x=[i for i in range(-100,100)]
    y=[j**2 for j in x]

    # Basic Plot
    plt.plot(x,y)
    plt.show()

    <Figure size 640x480 with 1 Axes>

back to top

### Installing and Loading Python

To install python, it can be downloaded from its website. However, this tutorial will use the distribution package, Anaconda.

Anaconda comes with more tools than this tutorial has time to explore. A key element of it is that it comes with the package installer __pip__. In a command line type `pip install django` and django is magically installed for the default version of python. This tutorial uses Python 2 due to it's compatibility with many of the current scientific based software packages. Python 3 is available, but many scientific packages have yet to upgrade. Once Anaconda is installed, call python from a command prompt by typing python or start a code editing tool (such as Jupyter or [PyCharm][pycharm]).

For this tutorial, you will need to download and install Anaconda for Python 2.7 (64bit version).

back to top

### Beginning a Jupyter Session

To begin a jupyter notebook session, open a command prompt or terminal and type `jupyter notebook`. This starts the server and then opens the web address http://localhost:8888/tree in the default web browser. Then click an existing notebook or open a new one.

#### Beginning a Jupyter Session in Windows¶

1. To start a command prompt in windows
2. Once in the command prompt type: jupyter notebook
3. After entering the command line, the jupyter browser should automatically open the default browser at http://localhost:8888/tree.
4. Click on open a new notebook

#### Beginning a Jupyter Session in OSX¶

1. To open a terminal in OSX:
   a) Click on the magnifying glass in the screen's top right corner to do a Spotlight Search
   b) Type terminal in the Spotlight Search
2. Once in the terminal type: jupyter notebook
3. After entering the command line, the jupyter browser should automatically open the default browser at http://localhost:8889/tree.
4. Click on New and then Python 2 to open a new notebook.

back to top

### Basic Python Syntax

Here are the basics of python syntax, including a few things to note for people that have used other languages:

1. Whitespace is important! Every different level of code block is indented
2. Flow commands and definitions of functions and classes have a colon (`:`) at the end
3. Reserved parameters are denoted with double underscores before and after `__` as in `__doc__`.
4. Comments begin with `#`
5. Everything is zero indexed
6. Power is `**` not `^`
7. No declaring variable types
   Here are some examples, try them in a code cell:

    # my python code
    def my_function(variable_one):
        """This is a proper function definition"""
        print("My function has executed and the value of variable_one is {0}".format(variable_one))

    # this is an if statement
    value=True
    if value:
        print("True!")
    else:
        print("False!")

In [2]:

    # My python code that has an error
    def my_function(variable_one):
    """This is an improper function definition"""
    print("My function has executed and the value of variable_one is {0}".format(variable_one))
    my_function('Hello There!')

      File "<ipython-input-2-82b82a0495b4>", line 3
        """This is an improper function definition"""
                                                    ^
    IndentationError: expected an indented block

In [3]:

    # My python code
    def my_function(variable_one):
        """This is a proper function definition"""
        print("My function has executed and the value of variable_one is {0}".format(variable_one))
    my_function('Hello There!')

    # The documentation string is stored in my_function.__doc__
    print(my_function.__doc__)

    My function has executed and the value of variable_one is Hello There!
    This is a proper function definition

In [4]:

    # This is an 'if' statement
    value=True
    if value:
        print("True!")
    else:
        print("False!")

    value=1
    if value==1:# equality test is == not =!
        print("Value is 1")
    elif value<0:
        print("Value is negative")

    True!
    Value is 1

In [5]:

    # Lists always start at index = 0
    test_list_1=['a','b','c']
    print("The first element of test_list_1 is {0}".format(test_list_1[0]))
    test_list_2=[i for i in range(100)]
    print("The first element of test_list_2 is {0}".format(test_list_2[0]))

    The first element of test_list_1 is a
    The first element of test_list_2 is 0

In [6]:

    # This is how to square a number
    print(10**2)
    # To square every number in a list
    test_list=[i for i in range(10)]
    test_list_squared=[x**2 for x in test_list]

    # Text formatting for the output
    print("-"*80)
    print("test_list is : {0}".format(test_list))
    print("-"*80)
    print("test_list_squared is : {0}".format(test_list_squared))
    print("-"*80)

    # As a side note the ^ operator is bitwise exclusive or for ints only.
    int_1=1
    int_2=0
    int_1^int_2

    100
    --------------------------------------------------------------------------------
    test_list is : [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
    --------------------------------------------------------------------------------
    test_list_squared is : [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
    --------------------------------------------------------------------------------

Out[6]:

    1

back to top

### Basic Python Types

### Python has some basic "types". Below are some most commonly used types.¶

1. Integer is denoted by int
2. String is denoted by str
3. Floating point number is denoted by float
4. Complex number is denoted by complex()
5. List type, a list of objects that can be amended, is denoted by list
6. Tuple (tuple) and set (set) are specific or special lists.
7. Dictionary, an associative array, or a list with a named index are denoted by dict

In [7]:

    # Here are some quick examples:
    # Print the result of adding the numbers below

    # integers
    print(1+1)
    print(type(1+1))

    # strings
    print('1'+'1')
    print(type('1'+'1'))

    # floats
    print(1.0+1.0)
    print(type(1.0+1.0))

    # complex
    print(1.0+1.0j)
    print(type(1.0+1.0j))

    # list
    print([1.0]+['1'])
    print(type([1.0]+['1']))

    2
    <class 'int'>
    11
    <class 'str'>
    2.0
    <class 'float'>
    (1+1j)
    <class 'complex'>
    [1.0, '1']
    <class 'list'>

In [8]:

    # To make lists and dictionaries

    # Explicitly
    my_list=[1,1,2,4,'5','space monkey']
    my_dictionary={'key 1':1,'key 2':2,'key 3':my_list}

    print(my_list)
    print(my_dictionary)

    [1, 1, 2, 4, '5', 'space monkey']
    {'key 1': 1, 'key 2': 2, 'key 3': [1, 1, 2, 4, '5', 'space monkey']}

In [9]:

    # Add to the end of a list with append
    my_list.append('New One')
    print(my_list)

    [1, 1, 2, 4, '5', 'space monkey', 'New One']

In [10]:

    # Add to a dictionary by assigning the value directly
    my_dictionary['New Key']='new value'
    print(my_dictionary)

    {'key 1': 1, 'key 2': 2, 'key 3': [1, 1, 2, 4, '5', 'space monkey', 'New One'], 'New Key': 'new value'}

In [11]:

    # Slicing lists

    # To get elements 0-2
    print("The first two elements of my_list are "+str(my_list[0:2]))

    # or
    my_list[:2]
    # Last 2 elements are
    my_list[-2:]

    # To get the last element
    # use -1 for selector
    print("The last element of my_list is "+str(my_list[-1]))

    The first two elements of my_list are [1, 1]
    The last element of my_list is New One

back to top

### Opening and Reading Files

One of the most basic operations for a scientist is to read in data from a file and prepare it for analysis. There are many libraries that exist for python which have many of the difficult parts of the code already complete. In an effort to better understand python, this tutorial will cover the "manual" way to read in the data vs using a library. In particular, this will cover some of the functionality in numpy.loadtxt() and panadas.read_csv(). It is important to note that the libraries of choice would be numpy or pandas.

In [ ]:

    # # THIS IS THE CODE TO WRITE THE EXAMPLE FILE IN CASE IT DOES NOT ALREADY EXIST.
    # # IT PROBABLY WILL NOT WORK UNLESS PYMEASURE HAS BEEN INSTALLED PROPERLY.
    # # A detour to write the needed example file:
    # import os
    # import scipy
    # import numpy as np
    # import random
    # from pyMez import *

    # start=-10
    # stop=10
    # num=1000
    # xdata=np.linspace(start,stop,num)
    # ydata_with_noise_2=[.5*scipy.exp(-(x-.1)**2/(2.0*3.0**2))+random.gauss(0,.03) for x in xdata]

    # # This gives the ability to look for files
    # #os.listdir(os.getcwd()) #list all the files in the current working directory

    # # This should be the directory where the example files are located.
    # file_directory=os.path.join(os.getcwd(),'Python_Introduction_Files')

    # # If the files are in the right place, open the following:
    # file_name='Example_File_1.txt'
    # data=[]
    # for index,item in enumerate(xdata.tolist()):
    #     data.append([item,ydata_with_noise_2[index]])
    # options={"data":data,"column_names":['Position','Amplitude'],"column_types":['float','float']}
    # table=AsciiDataTable(None,**options)
    # table.path=os.path.join(file_directory,file_name)
    # table.save()

In [12]:

    # To open a file
    # First it is useful to ‘import os’, a library that deals with the operating system.
    import os

    # This gives the ability to look for files
    #os.listdir(os.getcwd()) #list all the files in the current working directory

    # This should be the directory where the example files are located.
    file_directory=os.path.join(os.getcwd(),'Python_Introduction_Files')

    # If the files are in the right place, open the following:
    file_name='Example_File_1.txt'

    file_path=os.path.join(file_directory,file_name)

    # To open the file use the built in function open, in the read mode.
    in_file=open(file_path,'r')

    # The complete contents of the file can be retrieved by use
    # of ‘contents=in_file.read()’, however it is preferable for processing
    # reasons, to use a pattern that reads each line of the file into a
    # separate element of a list.
    lines=[]
    for line in in_file:
        lines.append(line)

    # The list named lines contains the information in the file. It is a list of strings.
    print("After reading lines[0] and lines[1] are now {0}".format(lines[0:2]))

    # It is good practice to close in_file
    in_file.close()

    # It is necessary at this point to conver the list of strings to a list of numbers.
    # The conversion requires the use of float() and split.
    # First convert each line into a list instead of a single string. This is accomplished via the split() method.
    # The first line of the file contains a column heading so the conversion process should begin with the
    # Second line.
    lines=[line.split(',') for line in lines[1:]]
    print("After splitting lines[0] is now {0}".format(lines[0]))
    # Next convert all values to the type 'float' instead of 'str'
    lines=[map(lambda x:float(x.rstrip().lstrip()),row) for row in lines]
    print("After converting lines[0] is now {0}".format(lines[0]))

    After reading lines[0] and lines[1] are now ['Position,Amplitude\n', '-10.0,-0.00931982497851\n']
    After splitting lines[0] is now ['-10.0', '-0.00931982497851\n']
    After converting lines[0] is now <map object at 0x0000000007E926A0>

back to top

### Defining a Function

A function can have none, one, or many arguments. A function can also return a value or not. This section will cover how to create a function that does what the code in the previous section did.
First, a few important things to remember about functions:

1. Functions start with the keyword `def`
2. The name of a function cannot have any spaces.
3. The name of a function cannot start with a number.
4. Functions end when the indented block ends.

Below is a basic example to show how it works:

    def function_one(variable_one,variable_two='default value',*list_arguments,**keyword_arguments):
        """function_one is an example function that will take a positionial argument, a named argument or the "packed" arguments
        in list_arguments or keyword arguments, it prints them out but does not return a value"""
        print("{0}, is the value of variable_one and {1} is the value of variable_two".format(variable_one,variable_two))
        for index,item in enumerate(list_arguments):
            print("Item number {0} in list_arguments is {1}".format(index,item))
        for key, value in keyword_arguments:
            print("keyword_arguments[{0}]={1}".format(key,value))

In [14]:

    def function_one(variable_one,variable_two='default value',*list_arguments,**keyword_arguments):
        """function_one is an example function that will take a positionial argument, a named argument or the "packed" arguments
        in list_arguments or keyword arguments, it prints them out but does not return a value"""

        print("{0}, is the value of variable_one and {1} is the value of variable_two".format(variable_one,variable_two))
        for index,item in enumerate(list_arguments):
            print("Item number {0} in list_arguments is {1}".format(index,item))
        for key, value in keyword_arguments.items():
            print("keyword_arguments[{0}]={1}".format(key,value))

    function_one('Hello',"hi",*['a list','of','values','or',123],**{'A Key':'A Value'})

    function_one('Hello again',*['a list','of','values','or',123],**{'A Key':'A Value'})

    function_one(*['a list','of','values','or',123],**{'A Key':'A Value'})

    function_one(**{'A Key':'A Value','variable_one':1})

    Hello, is the value of variable_one and hi is the value of variable_two
    Item number 0 in list_arguments is a list
    Item number 1 in list_arguments is of
    Item number 2 in list_arguments is values
    Item number 3 in list_arguments is or
    Item number 4 in list_arguments is 123
    keyword_arguments[A Key]=A Value
    Hello again, is the value of variable_one and a list is the value of variable_two
    Item number 0 in list_arguments is of
    Item number 1 in list_arguments is values
    Item number 2 in list_arguments is or
    Item number 3 in list_arguments is 123
    keyword_arguments[A Key]=A Value
    a list, is the value of variable_one and of is the value of variable_two
    Item number 0 in list_arguments is values
    Item number 1 in list_arguments is or
    Item number 2 in list_arguments is 123
    keyword_arguments[A Key]=A Value
    1, is the value of variable_one and default value is the value of variable_two
    keyword_arguments[A Key]=A Value

In [19]:

    # The code below uses a function to do the same thing as the code in the 'opening a file section'.
    # Note: Do not forget the doc strings.
    def process_data_file(file_path,skiprows=1):
        """process_data_file is a function that opens a data file and returns a 2-d list"""
        in_file=open(file_path,'r')
        x_data=[]
        y_data=[]
        for index,line in enumerate(in_file):
            if index>skiprows-1:
                line_list=line.split(',')
                line_list=list(map(lambda x:float(x),line_list))
                x_data.append(line_list[0])
                y_data.append(line_list[1])
            elif index<=skiprows-1:
                pass
        return [x_data,y_data]

    # Try the function
    #process_data_file(os.path.join(file_directory,file_name))

back to top

### Creating a Class

A class is a block of code, including variables and functions, that lives on its own. By defining chunks of code with classes it is stating: "these are important functions and variables when dealing with this problem". It is impossible to teach in a limited amount of time how functional, procedural, and object oriented programming (OOP) methodologies differ. OOP can be extremely helpful and/or necessary when programming for scientific purposes. When dealing with the repeated opening and analyzing of the same kind of file, the code and useful functions that do this should live in one or a few classes. Another time when it is helpful to know OOP is when dealing with any Graphical User Interface (GUI). Most GUI toolboxes use classes.

In [17]:

    class CSVData():
        """CSVData is a class that holds example data"""
        def __init__(self,file_path=None,**options):
            """Intializes the CSVData class, requires a file_path"""

            # This pattern allows for the setting of defaults for a large number of options
            # It also provides the ability to potentially add options after the initial class is created
            defaults={'directory':None,'file_name':None,'skiprows':1}
            self.options={}
            for key,value in defaults.items():
                self.options[key]=value
            for key,value in options.items():
                self.options[key]=value
            # Using the function from above and some error handling.
            # If there is an error in the code inside of the try: block it will be sent to an except block
            try:
                # First see if the user set file_path or directory and file_name in options
                if file_path is None:
                    file_path=os.path.join(self.options['directory'],self.options["file_name"])
                # Then use the function process_data_file to make the x_data and y_data attributes
                [self.x_data,self.y_data]=process_data_file(file_path)
            except:
                print("There was an error in opening file {0}".format(file_path))
                raise

In [20]:

    # Now use the class on the original file
    new_data=CSVData(os.path.join(os.getcwd(),'Python_Introduction_Files','Example_File_1.txt'))
    #print new_data.x_data

Those steps might seem pointless, but were preparation for the fact that through functions and variables, those chunks of code can be passed on to make more complicated programs. This is called class inheritance.

In [22]:

    # Example of an inherited class
    class MyData(CSVData):
        """MyData is a class that opens a standard CSV file and keeps things important to me """
        def __init__(self,file_name,**options):
            defaults={'directory':None,'file_name':None,'skiprows':1}
            self.options={}
            for key,value in defaults.items():
                self.options[key]=value
            for key,value in options.items():
                self.options[key]=value
            CSVData.__init__(self,file_name,**self.options)
            self.max=max(self.y_data)
            self.min=min(self.y_data)
    new_data=MyData(os.path.join(os.getcwd(),'Python_Introduction_Files','Example_File_1.txt'))
    new_data.max

Out[22]:

    0.620929768806

Now the class MyData has all of the functions and variables in CSVData in addition to the new ones!!

back to top

### Basic Plotting

Basic plots that emulate early MATLAB plots are easy to make in python using the library matplotlib.

In [23]:

    # Basic plotting with Matplotlib
    import matplotlib.pyplot as plt

    # Make a data set
    x=[i for i in range(-100,100)]
    y=[j**2 for j in x]

    # Basic Plot
    plt.plot(x,y)
    plt.show()

back to top

### Intermediate Plotting

If making a publication quality plot, many changes to suit the given audience are necessary such as font size, markers, etc. In addition, it would be nice to have the plot inside the notebook for reference; this is achieved with the notebook "magic". There are a lot of functionalities hidden in jupyter (and its predecessor ipython) that can be accessed using magics. A magic is started by putting something like `%matplotlib notebook` at the beginning of a cell. In this case it will cause the plot to live in the notebook and be interactive.

#### Making an Interactive Plot¶

Note: To undo an interactive plot usually requires a kernel restart.

In [24]:

    %matplotlib notebook
    x=[i for i in range(-100,100)]
    y=[j**2 for j in x]
    plt.plot(x,y)
    plt.show()

In [25]:

    # The magic stays until the kernel is restarted, so now is a good opportunity to add a plot_title
    plt.title('Parabolic')
    plt.show()

In [26]:

    # This will spruce up the title with a little latex
    plt.title('Parabolic Curve '+r'$f(x)=x^2$')

    # Adding x and y labels
    plt.xlabel('X Value')
    plt.ylabel('Y Value')

Out[26]:

    Text(20.8472,0.5,'Y Value')

In [27]:

    # Changing the line width
    line=plt.plot(x,y)
    plt.setp(line,color='r', linewidth=2.0)

Out[27]:

    [None, None]

In [28]:

    # Switch from a line to markers
    plt.plot(x,y,'ro')

Out[28]:

    [<matplotlib.lines.Line2D at 0x79e5be0>]

In [ ]:

    # Changing the fontsize
    plt.xlabel('X Value',fontsize=24)

In [29]:

    # Adding annotations in the coordinates system of the plot
    x,y=-50,7000
    plt.text(x, y, 'A New text Annotation at({0:3.2f},{1:3.2f})'.format(x,y))

Out[29]:

    Text(-50,7000,'A New text Annotation at(-50.00,7000.00)')

In [30]:

    # To kill the current plot
    plt.close()

back to top

### Using Jupyter's Markdown Cells to Document Code

The jupyter environment comes with a handy way of creating notes and documenting code - the markdown cell. A markdown cell handles traditional markdown (a HTML short hand), most html tags, and latex. This allows the user to make their code pretty.

In this section the user will learn:

1. How to create simple markdown cells with formatted text inside
2. How to add html for more complex formatting
3. How to highlight code for demonstration, but not execution
4. How to output simple formulae using latex (note: sometimes this is broken in Internet Explorer, use another browser such as Chrome)

#### Creating a Simple Markdown cell with Formatted Text Inside¶

1. Create a new cell, by executing the previous cell or Insert Cell
2. Change the type of cell from Code to Markdown by escape+m or by changing the selection in the tool bar
3. In the new cell enter `# My Heading`, execute the cell by pressing shift+enter. This will produce something like: # My Heading
4. The markdown shorthand has the ability to apply a large number of simple styles such as bold, italic, tables, links, and the equivalent of the HTML heading tags. A more complete listing of the possibilities for markdown can be found at :https://daringfireball.net/projects/markdown/syntax a more jupyter focused tutorial can be found here.

Most used shorthand rules:

- Headings start with `#` . The level is given by the number of `#'s` , like `## Heading 2` # Heading 1 ## Heading 2 ### Heading 3 #### Heading 4
- Number lists are default for things that start with numbers like `1. Numbered list element 1`
  1. Numbered list element 1
  2. Numbered list element 2
- Unordered or bulleted lists start with a symbol, such as `+`
  `+ Bullet 1`
  - Bullet 1
  - Bullet 2
- Bold is two underscores before and after the word or phrase, like `__my bolded text__` and will be displayed as my bolded text
- Italics is an asterix before and after, like `*italics is awesome*` and will be displayed as italics is awesome
- Tables can be made with underscores and pipes such as `| Tables | Are | Cool |` | Tables | Are | Cool |, I usually use the html version though.

#### Adding HTML for more Complex Formatting¶

In addition to handling the markdown shorthand syntax, jupyter's markdown cells handle most HTML tags properly. These include the ability to add the style attribute. For example, to change a heading to be red and in a different font type `<h1 style="color:red;font-family:serif"> My Heading</h1>`, execute the cell and something similar to the heading below should appear.

# My Heading

or to do a styled table such as:
`<table style="color:blue;"><tr><td style="border: 5px solid green;">Table row 1 column 1 </td><td style="border: 5px solid blue;">Table row 1 column 2 </td></tr></table>`

Table row 1 column 1 | Table row 1 column 2

this works for a range of tags that include :`<h1></h1>...<h6></h6><p></p><table></table><img />` does not work for javascript or svg. To embed them requires the use of the rich output methods in jupyter.

#### Code Highlighting for Code not Executed¶

Markdown cells have the ability to include a chunk of code in number of languages, provided they are properly identified with highlighting. To do this, start by escaping the test with three ` (backquote, the character under ~ on most keyboards) followed by a language name. So it looks like ```python ```

For example:
```python
# python comment
```
is rendered like

    # python comment

This works with HTML, javscript, python and a few others by changing the word directly after the three backquotes.
HTML:

    <h3 style="color:red"> Table </h3>
    <table> </table>

Javascript:

    // A javascript comment
    function myJavascriptFunction(input){
    var longUnreadableName=null;
    document.getElementById('fake-id').innerHTML=input;
    }

Visual Basic:

    vb
    dim Variable as integer

#### Latex for Equations and Type Setting¶

Jupyter markdown cells also understand latex. To do this, wrap expressions with two \$'s <br/> <br/>i.e.: <br/> ```$$your latex here$$``` for equations on their own line <br/> ```$your latex here$``` for inline equations <br/> This is sometimes broken in Internet Explorer, but works fine in most other browsers. <br/><br/>For example:<br/> the sum of xi from i=0 to i=100 <br/> `$\sum\limits{i=100}^{0}x{i}$` <br/>renders as: $\sum\limits_{i=100}^{0}x_i$ <br/> or a gaussian is `$$f(x)=ae^{-{(x-x_0)}^2\over{2\sigma^2}}$$`: $$f(x)=ae^{-{(x-x_0)}^2\over{2\sigma^2}}$$

back to top

### Basic Fitting

#### Fitting Using scipy¶

There is more than one way to do a fit in python. This tutorial will use the scipy library and its optimizing package to do a fit. Scipy can be found at https://www.scipy.org/

In [31]:

    # Import scipy and some of its important modules
    import scipy
    import scipy.optimize
    import scipy.stats
    # Import numpy for linspace and random numbers
    import numpy as np
    # Import python random numbers
    import random

    # Simulate data for fit
    start=-10
    stop=10
    num=1000
    xdata=np.linspace(start,stop,num)
    ydata=.5*scipy.exp(-(xdata-.1)**2/(2.0*3.0**2))
    ydata_with_noise=[.5*scipy.exp(-(x-.1)**2/(2.0*3.0**2))+np.random.random() for x in xdata]
    ydata_with_noise_2=[.5*scipy.exp(-(x-.1)**2/(2.0*3.0**2))+random.gauss(0,.03) for x in xdata]

    # Define fitting functions
    def lorentzian_function(a,x):
        "a[0]=amplitude,a[1]=center,a[2]=FWHM"
        return a[0]*1/(1+(x-a[1])**2/(a[2]**2))

    def gaussian_function(a,x):
        " a[0]=amplitude, a[1]=center, a[2]=std deviation"
        return a[0]*scipy.exp(-(x-a[1])**2/(2.0*a[2]**2))

    # Define the fit function that returns least square optimization
    def fit(function,xdata,ydata,a0):
        fit_dictionary={}
        error_function=lambda a, xdata, ydata:function(a,xdata)-ydata
        a,succes=scipy.optimize.leastsq(error_function, a0,args=(xdata,ydata))
        return a

    # Make a horrible guess
    a0=[1,1,1]
    # Call fit function on the data
    result=fit(gaussian_function,xdata,ydata_with_noise_2,a0)
    #result=fit(lorentzian_function,xdata,ydata_with_noise_2,a0)

    # Plot the results

    # Data plot
    plt.plot(xdata,ydata_with_noise_2,'bo')
    # Fit plot
    plt.plot(xdata,[gaussian_function(result,x) for x in xdata],'r-',linewidth=3)
    # Some plot type setting
    plt.title('Fit of data')
    options={'fontsize':24}
    # Text showing the numeric values with latex
    plt.text(-8, .5, r'$%3.2fe^{\frac{-{(x-%3.2f)}^2}{2*%3.2f^2}}$'%(result[0],result[1],result[2]),**options)
    plt.show()

back to top

### Analysis of a Fit

##### First Calculate the Residuals¶

Probably the most basic analysis of a fit is that of computing how the fit varies from the data. These data points are typically called residuals.

In [32]:

    plt.close()
    def calculate_residuals(fit_function,a,xdata,ydata):
        """Given the fit function, a parameter vector, xdata, and ydata returns the residuals as [x,y] pairs"""
        output_x=xdata
        output_y=[fit_function(a,x)-ydata[index] for index,x in enumerate(xdata)]
        return [output_x,output_y]
    [x,y]=calculate_residuals(gaussian_function,result,xdata,ydata_with_noise_2)
    plt.plot(x,y)
    #calculate_residuals(gaussian_function,result,xdata,ydata_with_noise_2)

Out[32]:

    [<matplotlib.lines.Line2D at 0x8f82710>]

#### Look at the distribution of residuals. How does the data vary from the model (fit)?¶

In [33]:

    # To do a histogram of the residuals
    plt.close()
    plt.hist(y)

Out[33]:

    (array([  3.,   8.,  52., 131., 248., 255., 196.,  91.,  14.,   2.]),
     array([-0.10914945, -0.08819918, -0.06724891, -0.04629863, -0.02534836,
            -0.00439809,  0.01655218,  0.03750245,  0.05845272,  0.07940299,
             0.10035327]),
     <a list of 10 Patch objects>)

In [34]:

    # To find the mean of the residuals and the standard deviation
    import numpy as np
    mean_y=np.mean(y)
    std_y=np.std(y)
    print("The residuals mean is {0} \n The residuals standard deviation is {1}".format(mean_y,std_y))

    The residuals mean is -0.00029961434077281
     The residuals standard deviation is 0.0300043523327786

#### It is Useful to Look at all the Information at the Same Time¶

In [35]:

    plt.close()
    # It is often helpful to see all of the above together at the same time
    def fit_and_plot(fit_function,xdata,ydata,a0=[1,1,1]):
        """fit and plot, fits xdata, ydata  with fit function and returns a plot of the fit overlaid with the data and
        a plot of the residuals, including a histogram"""
        fit_results=fit(fit_function,xdata,ydata,a0)
        # To do 2 plots in 2 rows with shared axis
        fig, (ax0, ax1) = plt.subplots(nrows=2, sharex=True)
        ax0.plot(xdata,ydata,'bo',label='data')
        ax0.plot(xdata,[fit_function(fit_results,x) for x in xdata],'r-',linewidth=3,label='fit')
        ax0.legend(loc='upper right', shadow=True)
        ax0.set_title('Fit of data using {0}'.format(fit_function.__name__))
        [x,y]=calculate_residuals(fit_function,fit_results,xdata,ydata)
        ax1.plot(x,y)
        ax1.set_title('Residuals')
        plt.show()
    fit_and_plot(gaussian_function,xdata,ydata_with_noise_2)

#### When the data does not Conform to the Model it is Common to see Structured Residuals¶

In [36]:

    plt.close()
    # Try the same data with a different fit function
    fit_and_plot(lorentzian_function,xdata,ydata_with_noise_2)

In [37]:

    plt.close()
    a0=[1 for i in range(3)]
    [x,y]=calculate_residuals(lorentzian_function,fit(lorentzian_function,
                                                      xdata,ydata_with_noise_2,a0),xdata,ydata_with_noise_2)
    plt.hist(y)
    plt.show()
    mean_y=np.mean(y)
    std_y=np.std(y)
    print("The residuals mean is {0} \nThe residuals standard deviation is {1}".format(mean_y,std_y))

    The residuals mean is 0.0125866621119829
    The residuals standard deviation is 0.044804042998745866

In [38]:

    %matplotlib inline

    from ipywidgets import *
    import numpy,scipy,random

    t=numpy.arange(0.0,1.0,0.01)
    start=-10
    stop=10
    num=1000
    xdata=numpy.linspace(start,stop,num)
    ydata=.5*scipy.exp(-(xdata-.1)**2/(2.0*3.0**2))
    ydata_with_noise=[.5*scipy.exp(-(x-.1)**2/(2.0*3.0**2))+numpy.random.random() for x in xdata]
    ydata_with_noise_2=[.5*scipy.exp(-(x-.1)**2/(2.0*3.0**2))+random.gauss(0,.03) for x in xdata]

    interact(lambda noise_sigma:fit_and_plot(lorentzian_function,
                                   xdata,
                                   [.5*scipy.exp(-(x-.1)**2/(2.0*3.0**2))+random.gauss(0,noise_sigma) for x in xdata]),
             noise_sigma=(0,.3,.01))

Out[38]:

    <function __main__.<lambda>(noise_sigma)>

In [39]:

    interact(lambda noise_sigma,noise_mu:fit_and_plot(gaussian_function,
                                   xdata,
                                   [.5*scipy.exp(-(x-.1)**2/(2.0*3.0**2))+random.gauss(noise_mu,noise_sigma) for x in xdata]),
             noise_sigma=(0,1,.01),noise_mu=(0,1,.01))

Out[39]:

    <function __main__.<lambda>(noise_sigma, noise_mu)>

back to top

### Basic Instrument Control

### Visa instrument control requires a visa driver for your OS. The standard one is made by National Instruments, but there are others.¶

1. Install NIVisa or on Mac OS https://pyvisa.readthedocs.io/en/stable/getting_nivisa.html#getting-nivisa
   - NIVISA requires registration, but it is free.
   - It allows communication through several methods, including GPIB.
   - This is not a python package, but is needed for pyvisa, pyvisa-py looks to be a potential alternative.
2. Install pyvisa (from the OS command prompt type `pip install pyvisa` or from jupyter try:

       import os
       os.system('pip install pyvisa')

In [ ]:

    # Test the installation
    import visa
    rm = visa.ResourceManager()
    print(rm.list_resources())

In [ ]:

    # To send a gpib command, change the address to the device
    my_instrument = rm.open_resource('GPIB0::14::INSTR')
    print(my_instrument.query('*IDN?'))

back to top

- ### More Information

## Other Resources¶

This tutorial only: https://pyvisa.readthedocs.io/en/stable/getting_nivisa.html#getting-nivisa covers a fraction of available resources for python, for more information see the links below.

### Python Based Packages Like Anaconda¶

1. Enthought https://www.enthought.com/products/canopy/
2. Python(x,y)https://python-xy.github.io/

### Jupyter Examples¶

1. http://nbviewer.jupyter.org/github/ipython/ipython/blob/4.0.x/examples/IPython%20Kernel/Index.html

### Macine Learning in Python¶

1. http://scikit-learn.org/stable/index.html

### Python Web Frameworks¶

1. https://www.djangoproject.com/
2. http://flask.pocoo.org/
3. http://www.tornadoweb.org/en/stable/

### Python Integrated Development Enviornments¶

1. https://pythonhosted.org/spyder/
2. https://www.jetbrains.com/pycharm/