AsciiDataTable_Example

# AsciiDataTable Example¶

The AsciiDataTable class is meant to deal with data in the form of a table with a header and footer. It has the ability to change formating, load a file, and do simple manipulations.

Some key elements of the AsciiDataTable class:

1. It can be created by opening a file with the right options. To preserve the options of a given file use the save_schema method and the read_schema function.
2. It can save, or print.
3. It has a large amount of formatting flexability

To make a new data table:

    new_table=AsciiDataTable()
    #or
    new_table=AsciiDataTable(header=["My Header","Line2" ],column_names=["a","b","c"],data=[[1,2,3]])

To open an existing table:

    #in default format (csv with no #)
    existing_table=AsciiDataTable("My_Table.txt")
    # for an existing format with a saved schema
    schema=read_schema("My_Schema")
    existing_table=AsciiDataTable("My_Table.txt",**schema)

In [1]:

    # As an example
    import pyMez.Code.DataHandlers.GeneralModels as GeneralModels

    Importing pyMez, this should take roughly 30 seconds
    The module smithplot was not found,please put it on the python path
    The module smithplot was not found,please put it on the python path

In [2]:

    #Lets make a table and put it into the AsciiDataTable
    new_table=GeneralModels.AsciiDataTable()

In [3]:

    # The data is a list of lists with the first dimension # of rows and the second dimension # of columns
    table_data=[[i+j*3 for i in range(3)]for j in range(3)]
    print table_data

    [[0, 1, 2], [3, 4, 5], [6, 7, 8]]

In [4]:

    #input the data directly
    new_table.data=table_data

In [5]:

    # assign the column names
    new_table.column_names=['x1','x2','x3']

In [6]:

    # when a string related command is called it uses build string to output the string print(new_table) also works
    print new_table.build_string()

    x1,x2,x3
    0,1,2
    3,4,5
    6,7,8

In [7]:

    # the attribute options stores the schema for the data
    new_table.options['column_names_begin_token']='#'

In [8]:

    print new_table.build_string()

    #x1,x2,x3
    0,1,2
    3,4,5
    6,7,8

In [9]:

    # changing a formating option doesn't change the data
    new_table.options['column_names_begin_token']='['
    new_table.options['column_names_end_token']=']'

In [10]:

    print new_table.build_string()

    [x1,x2,x3]
    0,1,2
    3,4,5
    6,7,8

In [11]:

    # There is the ability to control the output format in many ways
    new_table.options['data_begin_token']='*'*80+'\n'
    new_table.options['data_end_token']="\n"+'*'*80

In [12]:

    print new_table.build_string()

    [x1,x2,x3]
    ********************************************************************************
    0,1,2
    3,4,5
    6,7,8
    ********************************************************************************

In [13]:

    # the header is where we would store metadata
    new_table.header=['My First Header Line','My Second Header Line']

In [14]:

    print new_table

    My First Header Line
    My Second Header Line
    [x1,x2,x3]
    ********************************************************************************
    0,1,2
    3,4,5
    6,7,8
    ********************************************************************************

In [15]:

    new_table.options['comment_begin']='#'
    new_table.options['comment_end']='\n'

In [16]:

    print new_table

    #My First Header Line
    #My Second Header Line

    [x1,x2,x3]
    ********************************************************************************
    0,1,2
    3,4,5
    6,7,8
    ********************************************************************************

In [17]:

    # get_options returns all options printed in a nice form
    new_table.get_options()

    inline_comments = None
    data_begin_line = 4
    column_names_begin_line = 3
    footer_line_types = None
    general_descriptor = Table
    header_end_token = None
    header_end_line = 2
    column_names_end_line = 4
    data_table_element_separator =

    escape_character = None
    data = None
    inline_comment_begin = None
    column_descriptions = None
    inline_comment_end = None
    data_begin_token = ********************************************************************************

    comment_end =

    header_begin_line = 0
    row_end_token = None
    column_names_end_token = ]
    block_comment_begin = None
    header_begin_token = None
    metadata = None
    specific_descriptor = Data
    empty_value = None
    column_names_begin_token = [
    footer_end_token = None
    footer_begin_token = None
    column_names = None
    metadata_key_value_delimiter = None
    column_types = None
    comment_begin = #
    treat_header_as_comment = None
    column_units = None
    column_names_delimiter = ,
    treat_footer_as_comment = None
    extension = txt
    header = None
    footer = None
    data_end_line = None
    block_comment_end = None
    footer_end_line = None
    data_delimiter = ,
    footer_begin_line = None
    data_end_token =
    ********************************************************************************
    row_begin_token = None
    directory = None
    header_line_types = None
    row_formatter_string = None
    metadata_delimiter = None

In [18]:

    # when we created the file it autonamed itself. This is the place the file will save to using new_table.save()
    new_table.path

Out[18]:

    'Data_Table_20170429_001.txt'

In [19]:

    # Each formatting option can be turned off
    new_table.options['data_begin_token']=None
    new_table.options['data_end_token']=None

In [20]:

    print new_table

    #My First Header Line
    #My Second Header Line

    [x1,x2,x3]
    0,1,2
    3,4,5
    6,7,8

In [21]:

    # We can build the table this way or using
    # AsciiDataTable(header=new_table.header,column_names=new_table.column_names)
    new_table2=GeneralModels.AsciiDataTable()
    new_table2.header=new_table.header
    new_table2.column_names=new_table.column_names
    new_table2.options=new_table.options

In [22]:

    print new_table2

    #My First Header Line
    #My Second Header Line

    [x1,x2,x3]

In [23]:

    new_table2.data=[[2*(i+j*3) for i in range(3)]for j in range(3)]

In [24]:

    print new_table2

    #My First Header Line
    #My Second Header Line

    [x1,x2,x3]
    0,2,4
    6,8,10
    12,14,16

In [25]:

    # Adding the tables, if column names are the same
    new_table+new_table2

Out[25]:

    <Code.DataHandlers.GeneralModels.AsciiDataTable instance at 0x00000000112179C8>

In [26]:

    print new_table
    print new_table2

    #My First Header Line
    #My Second Header Line

    [x1,x2,x3]
    0,1,2
    3,4,5
    6,7,8
    0,2,4
    6,8,10
    12,14,16
    #My First Header Line
    #My Second Header Line

    [x1,x2,x3]
    0,2,4
    6,8,10
    12,14,16

In [27]:

    print new_table

    #My First Header Line
    #My Second Header Line

    [x1,x2,x3]
    0,1,2
    3,4,5
    6,7,8
    0,2,4
    6,8,10
    12,14,16

In [28]:

    print new_table2.build_string()

    #My First Header Line
    #My Second Header Line

    [x1,x2,x3]
    0,2,4
    6,8,10
    12,14,16

In [29]:

    str(new_table2).count("\n")

Out[29]:

    6

In [30]:

    test_string=str(new_table2)

In [31]:

    test_string

Out[31]:

    '#My First Header Line\n#My Second Header Line\n\n[x1,x2,x3]\n0,2,4\n6,8,10\n12,14,16'

In [32]:

    test_list=test_string.split('\n')

In [33]:

    test_list

Out[33]:

    ['#My First Header Line',
     '#My Second Header Line',
     '',
     '[x1,x2,x3]',
     '0,2,4',
     '6,8,10',
     '12,14,16']

In [34]:

    print new_table

    #My First Header Line
    #My Second Header Line

    [x1,x2,x3]
    0,1,2
    3,4,5
    6,7,8
    0,2,4
    6,8,10
    12,14,16

In [35]:

    new_table.options['inline_comment_begin']='(*'
    new_table.options['inline_comment_end']='*)'
    new_table.inline_comments=[["Hello, This  is an inline comment",0,3]]
    print new_table

    #My(*Hello, This  is an inline comment*) First Header Line
    #My Second Header Line

    [x1,x2,x3]
    0,1,2
    3,4,5
    6,7,8
    0,2,4
    6,8,10
    12,14,16

In [36]:

    import random

In [37]:

    data_delimiters=[",",".",":",";","-"," ","\t","::","~"," | "]
    begin_end_pairs=[["<",">"],["(*","*)"],["[","]"],["/*","*/"],["(",")"],["<","/>"],["#",""]]
    random_int=random.randint(0,len(data_delimiters)-1)
    random_list=[random.randint(0,len(data_delimiters)-1) for i in range(20)]
    print random_list

    [0, 7, 6, 8, 0, 3, 4, 1, 3, 9, 5, 8, 3, 5, 8, 1, 4, 6, 6, 8]

In [38]:

    random_delimiter_list_1=[data_delimiters[random.randint(0,len(data_delimiters)-1)] for i in range(20)]
    random_delimiter_list_2=[data_delimiters[random.randint(0,len(data_delimiters)-1)] for i in range(20)]
    random_pairs_1=[begin_end_pairs[random.randint(0,6)] for i in range(20)]
    random_pairs_2=[begin_end_pairs[random.randint(0,6)] for i in range(20)]

In [39]:

    print begin_end_pairs
    new_table.inline_comments=[["Hello, This  is an inline comment",0,-1]]

    [['<', '>'], ['(*', '*)'], ['[', ']'], ['/*', '*/'], ['(', ')'], ['<', '/>'], ['#', '']]

In [40]:

    # An example of all the formats randomized
    for index in range(len(random_delimiter_list_1)):
        new_table.options["data_delimiter"]=random_delimiter_list_1[index]
        new_table.options["column_names_delimiter"]=random_delimiter_list_2[index]
        new_table.options["comment_begin"]=random_pairs_1[index][0]
        new_table.options["comment_end"]=random_pairs_1[index][1]+"\n"
        new_table.options["column_names_begin_token"]=random_pairs_2[index][0]
        new_table.options["column_names_end_token"]=random_pairs_2[index][1]
        print "\n{0}\n".format(new_table)

    (*My First Header Line*)(*Hello, This  is an inline comment*)
    (*My Second Header Line*)

    /*x1:x2:x3*/
    0 1 2
    3 4 5
    6 7 8
    0 2 4
    6 8 10
    12 14 16

    [My First Header Line](*Hello, This  is an inline comment*)
    [My Second Header Line]

    /*x1.x2.x3*/
    0-1-2
    3-4-5
    6-7-8
    0-2-4
    6-8-10
    12-14-16

    /*My First Header Line*/(*Hello, This  is an inline comment*)
    /*My Second Header Line*/

    (*x1;x2;x3*)
    0::1::2
    3::4::5
    6::7::8
    0::2::4
    6::8::10
    12::14::16

    [My First Header Line](*Hello, This  is an inline comment*)
    [My Second Header Line]

    (x1::x2::x3)
    0	1	2
    3	4	5
    6	7	8
    0	2	4
    6	8	10
    12	14	16

    <My First Header Line>(*Hello, This  is an inline comment*)
    <My Second Header Line>

    <x1::x2::x3/>
    0,1,2
    3,4,5
    6,7,8
    0,2,4
    6,8,10
    12,14,16

    [My First Header Line](*Hello, This  is an inline comment*)
    [My Second Header Line]

    <x1 | x2 | x3>
    0.1.2
    3.4.5
    6.7.8
    0.2.4
    6.8.10
    12.14.16

    /*My First Header Line*/(*Hello, This  is an inline comment*)
    /*My Second Header Line*/

    [x1:x2:x3]
    0:1:2
    3:4:5
    6:7:8
    0:2:4
    6:8:10
    12:14:16

    [My First Header Line](*Hello, This  is an inline comment*)
    [My Second Header Line]

    <x1.x2.x3>
    0	1	2
    3	4	5
    6	7	8
    0	2	4
    6	8	10
    12	14	16

    /*My First Header Line*/(*Hello, This  is an inline comment*)
    /*My Second Header Line*/

    (x1.x2.x3)
    0::1::2
    3::4::5
    6::7::8
    0::2::4
    6::8::10
    12::14::16

    #My First Header Line(*Hello, This  is an inline comment*)
    #My Second Header Line

    <x1,x2,x3/>
    0 1 2
    3 4 5
    6 7 8
    0 2 4
    6 8 10
    12 14 16

    (*My First Header Line*)(*Hello, This  is an inline comment*)
    (*My Second Header Line*)

    #x1.x2.x3
    0.1.2
    3.4.5
    6.7.8
    0.2.4
    6.8.10
    12.14.16

    (My First Header Line)(*Hello, This  is an inline comment*)
    (My Second Header Line)

    [x1:x2:x3]
    0:1:2
    3:4:5
    6:7:8
    0:2:4
    6:8:10
    12:14:16

    /*My First Header Line*/(*Hello, This  is an inline comment*)
    /*My Second Header Line*/

    [x1~x2~x3]
    0 1 2
    3 4 5
    6 7 8
    0 2 4
    6 8 10
    12 14 16

    <My First Header Line>(*Hello, This  is an inline comment*)
    <My Second Header Line>

    <x1 x2 x3/>
    0::1::2
    3::4::5
    6::7::8
    0::2::4
    6::8::10
    12::14::16

    <My First Header Line>(*Hello, This  is an inline comment*)
    <My Second Header Line>

    (x1 x2 x3)
    0.1.2
    3.4.5
    6.7.8
    0.2.4
    6.8.10
    12.14.16

    /*My First Header Line*/(*Hello, This  is an inline comment*)
    /*My Second Header Line*/

    <x1 | x2 | x3/>
    0~1~2
    3~4~5
    6~7~8
    0~2~4
    6~8~10
    12~14~16

    #My First Header Line(*Hello, This  is an inline comment*)
    #My Second Header Line

    (*x1 x2 x3*)
    0~1~2
    3~4~5
    6~7~8
    0~2~4
    6~8~10
    12~14~16

    [My First Header Line](*Hello, This  is an inline comment*)
    [My Second Header Line]

    [x1~x2~x3]
    0.1.2
    3.4.5
    6.7.8
    0.2.4
    6.8.10
    12.14.16

    (My First Header Line)(*Hello, This  is an inline comment*)
    (My Second Header Line)

    (*x1;x2;x3*)
    0 | 1 | 2
    3 | 4 | 5
    6 | 7 | 8
    0 | 2 | 4
    6 | 8 | 10
    12 | 14 | 16

    (My First Header Line)(*Hello, This  is an inline comment*)
    (My Second Header Line)

    (x1-x2-x3)
    0-1-2
    3-4-5
    6-7-8
    0-2-4
    6-8-10
    12-14-16

In [41]:

    GeneralModels.show_structure_script()

    Printing the string representation of the table
    --------------------------------------------------------------------------------
    {header_begin_token}
    self.header[0]
    self.header[1]{inline_comment_begin}inline_comments[0][0]{inline_comment_end}
    {comment_begin}self.header[2]{comment_end}

    {block_comment_begin}
    self.header[3]
    self.header{4]
    {block_comment_end}
    {header_end_token}
    {data_table_element_separator}
    {column_names_begin_token}column_names[0]{column_names_delimiter}column_names[1]{column_names_delimiter}column_names[2]{column_names_end_token}
    {data_table_element_separator}
    {data_begin_token}
    {row_begin_token}data[0][0]{data_delimiter}data[1][0]{data_delimiter}data[2][0]{row_end_token}
    {row_begin_token}data[0][1]{data_delimiter}data[1][1]{data_delimiter}data[2][1]{row_end_token}
    {data_end_token}
    {data_table_element_separator}
    {footer_begin_token}
    {block_comment_begin}
    self.footer[0]
    self.footer[1]
    {block_comment_end}
    {footer_end_token}
    Printing the lines representation of the table with line numbers
    --------------------------------------------------------------------------------
    0 {header_begin_token}
    1 self.header[0]
    2 self.header[1]{inline_comment_begin}inline_comments[0][0]{inline_comment_end}
    3 {comment_begin}self.header[2]{comment_end}
    4
    5 {block_comment_begin}
    6 self.header[3]
    7 self.header{4]
    8 {block_comment_end}
    9 {header_end_token}
    10 {data_table_element_separator}
    11 {column_names_begin_token}column_names[0]{column_names_delimiter}column_names[1]{column_names_delimiter}column_names[2]{column_names_end_token}
    12 {data_table_element_separator}
    13 {data_begin_token}
    14 {row_begin_token}data[0][0]{data_delimiter}data[1][0]{data_delimiter}data[2][0]{row_end_token}
    15 {row_begin_token}data[0][1]{data_delimiter}data[1][1]{data_delimiter}data[2][1]{row_end_token}
    16 {data_end_token}
    17 {data_table_element_separator}
    18 {footer_begin_token}
    19 {block_comment_begin}
    20 self.footer[0]
    21 self.footer[1]
    22 {block_comment_end}
    23 {footer_end_token}
    --------------------------------------------------------------------------------
    The result of self.lines[0:10] is :
    {header_begin_token}
    self.header[0]
    self.header[1]{inline_comment_begin}inline_comments[0][0]{inline_comment_end}
    {comment_begin}self.header[2]{comment_end}

    {block_comment_begin}
    self.header[3]
    self.header{4]
    {block_comment_end}
    {header_end_token}
    --------------------------------------------------------------------------------
    --------------------------------------------------------------------------------
    The result of self.lines[11:12] is :
    {column_names_begin_token}column_names[0]{column_names_delimiter}column_names[1]{column_names_delimiter}column_names[2]{column_names_end_token}
    --------------------------------------------------------------------------------
    --------------------------------------------------------------------------------
    The result of self.lines[13:17] is :
    {data_begin_token}
    {row_begin_token}data[0][0]{data_delimiter}data[1][0]{data_delimiter}data[2][0]{row_end_token}
    {row_begin_token}data[0][1]{data_delimiter}data[1][1]{data_delimiter}data[2][1]{row_end_token}
    {data_end_token}
    --------------------------------------------------------------------------------
    --------------------------------------------------------------------------------
    The result of self.lines[18:None] is :
    {footer_begin_token}
    {block_comment_begin}
    self.footer[0]
    self.footer[1]
    {block_comment_end}
    {footer_end_token}
    --------------------------------------------------------------------------------