Graph_Model_Transformation_Table_Example

# Graph Transformation Example¶

Data in a table form can be transformed into numerous different formats using the class pyMez.Code.DataHandlers.GraphModels.TableGraph. The abstract data model is a table with column names.

## Examples of Formats¶

Excel File
Csv File
Hdf File
Html File
Json File
Matlab File
Open Office File
XML File

## Import needed modules¶

In [1]:

    from pyMez.Code.DataHandlers.GraphModels import *

    Importing pyMez, this should take roughly 30 seconds
    Importing Code.DataHandlers.GeneralModels
    It took 0.51203 s to import Code.DataHandlers.GeneralModels
    Importing Code.DataHandlers.HTMLModels
    It took 0.105006 s to import Code.DataHandlers.HTMLModels
    Importing Code.DataHandlers.NISTModels
    The module smithplot was not found,please put it on the python path
    It took 0.785044 s to import Code.DataHandlers.NISTModels
    Importing Code.DataHandlers.TouchstoneModels
    It took 0.0 s to import Code.DataHandlers.TouchstoneModels
    Importing Code.DataHandlers.XMLModels
    It took 0.094006 s to import Code.DataHandlers.XMLModels
    Importing Code.DataHandlers.ZipModels
    It took 0.008 s to import Code.DataHandlers.ZipModels
    Importing Code.InstrumentControl.Experiments
    It took 2.62215 s to import Code.InstrumentControl.Experiments
    Importing Code.InstrumentControl.Instruments
    It took 0.0 s to import Code.InstrumentControl.Instruments
    Importing Code.Utils.Names
    It took 0.0 s to import Code.Utils.Names
    It took 4.126236 s to import all of the active modules

    C:\Users\sandersa\AppData\Local\Continuum\anaconda3\lib\site-packages\odo\backends\pandas.py:102: FutureWarning: pandas.tslib is deprecated and will be removed in a future version.
    You can access NaTType as type(pandas.NaT)
      @convert.register((pd.Timestamp, pd.Timedelta), (pd.tslib.NaTType, type(None)))

In [2]:

    # create the graph
    graph=TableGraph()

In [3]:

    # now we have access to the folowing data formats
    graph.node_names

Out[3]:

    ['DataFrame',
     'AsciiDataTable',
     'HdfFile',
     'XmlDataTable',
     'ExcelFile',
     'OdsFile',
     'HtmlString',
     'JsonFile',
     'JsonString',
     'CsvFile',
     'MatFile',
     'XmlFile',
     'HtmlFile']

In [4]:

    # to visualize the graph use the show method
    graph.show()

Out[4]:

    <Figure size 432x288 with 0 Axes>

    <Figure size 432x288 with 0 Axes>

In [5]:

    # if we want to open an ascii data table or subclass we open the schema and then open the file
    schema=read_schema(os.path.join(os.getcwd(),"./TableGraph_Files/Line_4909_WR15_Wave_Parameters_20180313_003.schema"))
    ascii_table=AsciiDataTable(os.path.join(os.getcwd(),"./TableGraph_Files/Line_4909_WR15_Wave_Parameters_20180313_003.w2p"),
                               **schema)

In [6]:

    graph.set_state(node_name="AsciiDataTable",node_data=ascii_table)

In [7]:

    # now if you want a pandas data frame just move to that node
    graph.move_to_node("DataFrame")

In [8]:

    #now dataframe is just
    pandas_dataframe=graph.data

In [9]:

    pandas_dataframe[0:10]

Out[9]:

| Frequency | reA1_D1 | imA1_D1 | reB1_D1 | imB1_D1 | reA2_D1 | imA2_D1 | reB2_D1 | imB2_D1 | reA1_D2 | imA1_D2 | reB1_D2 | imB1_D2 | reA2_D2 | imA2_D2 | reB2_D2 | imB2_D2
--- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | ---
0 | 50.0 | -0.06771637116 | -0.01816907875 | -0.002698191406 | 0.0007358189392 | 0.002504944754 | 0.006581974472 | 0.004641698273 | -0.02466258165 | -0.002775838721 | 0.0007365005672 | -0.009040250908 | -0.01372005375 | -0.0304376245 | 0.03904758232 | -0.0006660961421 | -0.001239688192
1 | 50.1 | 0.02243850646 | 0.06665811006 | -0.002309633574 | -0.001366194728 | -0.007025498308 | 0.003140471545 | 0.003017988775 | 0.02523530152 | 0.002766433187 | 0.0008072947322 | -0.01298566794 | 0.009946449155 | -0.006614970446 | 0.04995893607 | 0.0005611848447 | 0.0007264105887
2 | 50.2 | 0.04527021965 | -0.05256141552 | -0.00173388365 | -0.001927861015 | -0.002513645387 | -0.007837803592 | -0.007693914526 | -0.02422755665 | -0.0004392237569 | 0.002661782361 | 0.002922748697 | -0.01585399076 | 0.02744889107 | -0.04267520686 | 0.000952701442 | 0.0002679299738
3 | 50.3 | -0.068064164 | -0.006680145887 | -0.0005841158336 | -0.002532520237 | 0.008350680994 | -0.001583162367 | 0.01300682537 | 0.0216939389 | -0.002352720107 | -0.0004776609296 | 0.006113562282 | 0.01449721952 | -0.04672627515 | 0.01879209235 | 0.0002190834271 | 0.001004918029
4 | 50.4 | 0.01993176055 | 0.06522905575 | 0.001363952887 | -0.002947389348 | -0.0007865336407 | 0.008783444812 | -0.01435682532 | -0.02113154479 | 0.001078082821 | -0.001850086426 | -0.0104808844 | -0.01154704467 | 0.04611253162 | 0.02143472309 | 0.0002404384847 | 0.0006860393303
5 | 50.5 | 0.06120319388 | -0.02933126305 | 0.002600965115 | 0.0002420115068 | -0.00765577376 | -0.004734756304 | 0.01142720969 | 0.02331471344 | 0.0009795936786 | 0.001590920773 | 0.01322155897 | 0.008355509256 | -0.01542597473 | -0.04949449405 | -0.001176449841 | 0.0005619314235
6 | 50.6 | -0.04549624588 | -0.05036273752 | 0.002401206648 | 0.001194642981 | 0.00717132721 | -0.005577364131 | -0.01149386376 | -0.02369062349 | -0.001749184772 | -0.0001509449615 | -0.01486666319 | -0.004184518099 | -0.02583977377 | 0.04489485325 | -2.531147057e-05 | -0.0003908249213
7 | 50.7 | -0.03622071059 | 0.05817229729 | -3.623704135e-05 | 0.003032823443 | 0.002033359668 | 0.008625612246 | 0.009857377368 | 0.02480800621 | 0.001344404504 | -0.001363835 | 0.01510341316 | 0.002440900883 | 0.05179807238 | 0.0003036752765 | -0.0007968675442 | -0.0007567387715
8 | 50.8 | 0.0630271502 | 0.02902860699 | -0.001880705228 | 0.001355544502 | -0.007952566598 | -0.003059360583 | -0.003294191447 | -0.02718061392 | 0.0001941419538 | 0.00223786501 | -0.01526013303 | -0.001378560896 | -0.02279080746 | -0.04731256854 | 0.001281713799 | -0.0005104444253
9 | 50.9 | 0.02000265252 | -0.06661988479 | -0.002146230544 | -0.0004442927581 | 0.006057403486 | -0.004937491383 | -0.001895092701 | 0.02750249379 | -0.002240929551 | -0.00148763921 | 0.01513982274 | -9.430191224e-05 | -0.02918889338 | 0.04371010376 | 0.0007133279978 | 0.000487065628

In [10]:

    # if we want to transform to json string
    graph.move_to_node("JsonString")

In [11]:

    json=graph.data

In [12]:

    # if we would like to transform it to all possible data types -the xslx and ods require a confirmation
    # first change the working directory and now all files will be put in the folder
    os.chdir("./TableGraph_Files")
    visit_all_nodes(graph)

    C:\Users\sandersa\AppData\Local\Continuum\anaconda3\lib\site-packages\pyMez\Code\DataHandlers\Translations.py:572: FutureWarning: Method .as_matrix will be removed in a future version. Use .values instead.
      conversion_options["data"]=pandas_data_frame.as_matrix().tolist()[:]

    data_nodes is [<DOM Element: Data at 0xd1b12a8>]

    C:\Users\sandersa\AppData\Local\Continuum\anaconda3\lib\site-packages\pyMez\Code\DataHandlers\Translations.py:572: FutureWarning: Method .as_matrix will be removed in a future version. Use .values instead.
      conversion_options["data"]=pandas_data_frame.as_matrix().tolist()[:]
    C:\Users\sandersa\AppData\Local\Continuum\anaconda3\lib\site-packages\scipy\io\matlab\miobase.py:414: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.
      arr[empties] = ' '

    data_nodes is [<DOM Element: Data at 0xd61be88>]
    data_nodes is [<DOM Element: Data at 0x805bb90>]