{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "9965e19a-97cf-45c0-941e-699602ec60fe",
   "metadata": {},
   "source": [
    "This tutorial deals with __reading a SEG-Y data set__ in various different ways and analyzing the available data. Please make sure you have downloaded the test data as outlined in the file \"examples/data/README\".\n",
    "\n",
    "Let's start by importing some packages (including the __seisio__ package itself) and setting up the logging config."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "2852c71c-ae09-4fa4-b1a0-9683cbeeda13",
   "metadata": {},
   "outputs": [],
   "source": [
    "import logging\n",
    "import matplotlib.pyplot as plt\n",
    "from matplotlib import cm\n",
    "import numpy as np\n",
    "\n",
    "import sys\n",
    "sys.path.insert(1, '../')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "4f65db4f",
   "metadata": {},
   "outputs": [],
   "source": [
    "import seisio"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "54d491e2",
   "metadata": {},
   "outputs": [],
   "source": [
    "logging.basicConfig(level=logging.INFO, format='%(levelname)s: %(message)s', force=True)\n",
    "log = logging.getLogger(\"main\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "422834cb-996c-4474-b4f0-8ba8fb815089",
   "metadata": {},
   "source": [
    "We would like to read the file called Line_001.sgy in the data directory. So let's just try the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "9e1c85b2-2720-4f45-8652-13e51fb9ec84",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO: Input file: data/Line_001.sgy\n",
      "INFO: SEG-Y textual file header encoding looks to be 'EBCDIC' (best guess).\n",
      "INFO: Input file endianess looks to be '>' (best guess).\n",
      "INFO: SEG-Y revision: original SEG-Y conforming to 1975 standard.\n",
      "INFO: Data sample format: 4-byte IBM floating-point.\n",
      "WARNING: SEG-Y fixed-trace-length flag is not set in binary header.\n",
      "WARNING: As SEG-Y major revision is 0, assuming fixed-length traces.\n",
      "INFO: Number of additional textual header records: 0.\n",
      "INFO: Number of trailer stanza records: 0.\n",
      "INFO: SEG-Y trace header extension 1 is not present.\n",
      "INFO: Number of user-defined trace headers: 0\n",
      "INFO: Byte offset of first trace relative to start of file: 3600 bytes.\n",
      "INFO: Number of samples per data trace: 1501.\n",
      "INFO: Sampling interval: 2000 (unit as per SEG-Y standard).\n",
      "INFO: Number of data traces in file: 71284.\n",
      "INFO: Delay (on first trace): 0 (unit as per SEG-Y standard).\n"
     ]
    }
   ],
   "source": [
    "infile = \"data/Line_001.sgy\"\n",
    "sio = seisio.input(infile)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "9de0fbce-34cf-48dd-b7fe-df24279d13e2",
   "metadata": {},
   "source": [
    "The standard INFO output already provides quite a lot of information about the file. Apparently, there are 71284 traces, each trace has 1501 samples, and the sampling interval is 2 milliseconds as per SEG-Y standard. The data themselves are stored as 4-byte (32-bit) IBM floating-point values. The endianess of the file and the encoding of the primary SEG-Y textual header have been detected automatically. The same holds for the number of additional header records, the presence of header extension 1, the number of user-defined trace headers, and the number of trailer records. As expected with SEG-Y rev. 0, none of these features are present here as they aren't allowed by the original SEG-Y standard from 1975. There are options available to override the automatic detections if necessary in case of any problems. Default binary and trace header definition tables have been used, but a user could also provide own definition tables (in JSON format) in case the data had an unusual header layout or we wanted to rename/remap certain headers or pick up additional header words (mnemonics) from certain byte locations.\n",
    "\n",
    "Note: The file is not kept open all the time, i.e., we do not need a context manager here to clean up in case something goes wrong. The file is opened only when an I/O operation needs to take place.\n",
    "\n",
    "As soon as the seisio object is established, we can get certain properties:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "3e4c3a8c",
   "metadata": {},
   "outputs": [],
   "source": [
    "file_endianess = sio.endianess       # file endianess, little or big endian\n",
    "file_name = sio.file                 # file name\n",
    "file_size = sio.fsize                # file size in bytes\n",
    "trace_header_size = sio.thsize       # size in bytes of a single trace header\n",
    "trace_header_keys = sio.mnemonics    # list of trace header mnemonics\n",
    "trace_size = sio.trsize              # size in bytes of a single complete trace\n",
    "nsamples = sio.nsamples              # number of samples per trace; or sio.ns for short\n",
    "ntraces = sio.ntraces                # total number of traces in file; or sio.nt for short\n",
    "sampling_interval = sio.vsi          # sampling interval (usually in microsec) along vertical axis\n",
    "sampling_times = sio.vaxis           # vector of sampling times (vertical axis)\n",
    "recording_delay = sio.delay          # delay recording time of first trace (usually in millisec)\n",
    "ntxtrecord = sio.ntxtrec             # number of additional textual header records\n",
    "ntxtrailer = sio.ntxtrail            # number of textual trailer records\n",
    "nthuser = sio.nthuser                # number of user-defined trace headers\n",
    "trace_header_ext1 = sio.thext1       # flag whether trace header extension 1 is present\n",
    "textual_file_header = sio.txthead    # primary SEG-Y textual file header (decoded if necessary)\n",
    "binary_file_header = sio.binhead     # SEG-Y binary file header"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "07fdf435-e913-4463-b105-1cd0cd4ad647",
   "metadata": {},
   "source": [
    "We can get and log the content of the primary SEG-Y textual header at the same time:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "8331ecae",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO: SEG-Y textual file header:\n",
      "INFO: -------- BEGIN --------\n",
      "C 1 CLIENT                        COMPANY                       CREW NO         \n",
      "C 2 LINE:  LINE_001           AREA                        MAP ID                \n",
      "C 3 REEL NO           DAY-START OF REEL     YEAR      OBSERVER                  \n",
      "C 4 INSTRUMENT: MFG            MODEL            SERIAL NO                       \n",
      "C 5 DATA TRACES/RECORD: 282  AUXILIARY TRACES/RECORD:  2    CDP FOLD            \n",
      "C 6 SAMPLE INTERNAL:  4MS     SAMPLES/TRACE: 750  BITS/IN      BYTES/SAMPLE 4   \n",
      "C 7 RECORDING FORMAT        FORMAT THIS REEL: SEGY   MEASUREMENT SYSTEM         \n",
      "C 8 SAMPLE CODE: FLOATING PT     FIXED PT     FIXED PT-GAIN     CORRELATED      \n",
      "C 9 GAIN  TYPE: FIXED     BINARY     FLOATING POINT     OTHER                   \n",
      "C10 FILTERS: ALIAS     HZ  NOTCH     HZ  BAND    -     HZ  SLOPE    -    DB/OCT \n",
      "C11 SOURCE: TYPE            NUMBER/POINT        POINT INTERVAL                  \n",
      "C12     PATTERN:                           LENGTH        WIDTH                  \n",
      "C13 SWEEP: START     HZ  END     HZ  LENGTH      MS  CHANNEL NO     TYPE        \n",
      "C14 TAPER: START LENGTH       MS  END LENGTH       MS  TYPE                     \n",
      "C15 SPREAD: OFFSET        MAX DISTANCE        GROUP INTERVAL                    \n",
      "C16 GEOPHONES: PER GROUP     SPACING     FREQUENCY     MFG          MODEL       \n",
      "C17     PATTERN:                           LENGTH        WIDTH                  \n",
      "C18 TRACES SORTED BY: RECORD     CDP     OTHER                                  \n",
      "C19 AMPLITUDE RECOVEY: NONE      SPHERICAL DIV       AGC    OTHER               \n",
      "C20 MAP PROJECTION                      ZONE ID       COORDINATE UNITS          \n",
      "C21 PROCESSING:                                                                 \n",
      "C22 PROCESSING:                                                                 \n",
      "C23                                                                             \n",
      "C24                                                                             \n",
      "C25                                                                             \n",
      "C26                                                                             \n",
      "C27                                                                             \n",
      "C28                                                                             \n",
      "C29                                                                             \n",
      "C30                                                                             \n",
      "C31                                                                             \n",
      "C32                                                                             \n",
      "C33                                                                             \n",
      "C34                                                                             \n",
      "C35                                                                             \n",
      "C36                                                                             \n",
      "C37                                                                             \n",
      "C38                                                                             \n",
      "C39                                                                             \n",
      "C40 END EBCDIC                                                                  \n",
      "INFO: --------- END ---------\n"
     ]
    }
   ],
   "source": [
    "txthead = sio.log_txthead()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "301e3729-64e0-49ab-9ed6-f067879dfd68",
   "metadata": {},
   "source": [
    "Obviously, whoever created this SEG-Y file didn't put a lot of information in the primary textual header (EBCDIC), and some of the information is even wrong (such as the sampling interval). Unfortunately, that's fairly common when dealing with SEG-Y data.\n",
    "\n",
    "We can also get and log the binary file header at the same time. When logging such a header, it's default to not print entries with value zero. In other words, the list here only shows binary header entries with non-zero values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "5e4b91f6",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO: SEG-Y binary file header:\n",
      "+--------+---------+----------------------------------------------------------+\n",
      "|        |   value | description                                              |\n",
      "|--------+---------+----------------------------------------------------------|\n",
      "| nt     |   34463 | Number of traces per ensemble                            |\n",
      "| dt     |    2000 | Sample interval                                          |\n",
      "| dtorig |    2000 | Sample interval of original field recording              |\n",
      "| ns     |    1501 | Number of samples per trace                              |\n",
      "| nsorig |    8193 | Number of samples per trace for original field recording |\n",
      "| format |       1 | Data sample format code                                  |\n",
      "| tsort  |       1 | Trace sorting code                                       |\n",
      "| msys   |       1 | Measurement system                                       |\n",
      "+--------+---------+----------------------------------------------------------+\n"
     ]
    }
   ],
   "source": [
    "binhead = sio.log_binhead()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "391ae187-9282-43da-bc70-017cd6fbc914",
   "metadata": {},
   "source": [
    "Let's read all the treace headers from disk so we can have a look at them more closely. There are other methods available to only read certain subsets of trace headers. However, here we just read all of the in one go:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "c1ed162c",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO: Reading all 71284 trace headers from disk...\n",
      "INFO: Reading all headers took 0.048 seconds.\n"
     ]
    }
   ],
   "source": [
    "trace_headers = sio.read_all_headers()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "da9a30b3-801e-4893-8ab3-1b17fceb47ec",
   "metadata": {},
   "source": [
    "Now we can determine some trace header statistics to get a good overview of the data set at hand. As we have already read the trace headers from disk, we can make those headers available to the *log_thstat()* method, otherwise the method would read the trace headers from disk itself."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "a2e6e75f",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO: Calculating trace header statistics.\n",
      "INFO: Summary of trace header statistics (zeros excluded):\n",
      "+--------+--------+--------+---------------+---------------+-----------+-----------+\n",
      "|        |    min |    max |          mean |           std |       25% |       75% |\n",
      "|--------+--------+--------+---------------+---------------+-----------+-----------|\n",
      "| tracl  |      1 |  71284 |  35642.5      | 20578.1       |  17821.8  |  53463.2  |\n",
      "| tracr  |      1 |  71284 |  35642.5      | 20578.1       |  17821.8  |  53463.2  |\n",
      "| fldr   |    231 |    481 |    356        |    72.4574    |    293    |    419    |\n",
      "| tracf  |     -1 |    282 |    140.496    |    81.9899    |     69.75 |    211.25 |\n",
      "| ep     |     32 |    282 |    157        |    72.4574    |     94    |    220    |\n",
      "| cdpt   |      1 |    284 |    142.5      |    81.9838    |     71.75 |    213.25 |\n",
      "| trid   |      0 |      1 |      0.992958 |     0.0836227 |      1    |      1    |\n",
      "| nhs    |      1 |      1 |      1        |     0         |      1    |      1    |\n",
      "| scalel | -10000 | -10000 | -10000        |     0         | -10000    | -10000    |\n",
      "| scalco | -10000 | -10000 | -10000        |     0         | -10000    | -10000    |\n",
      "| counit |      3 |      3 |      3        |     0         |      3    |      3    |\n",
      "| ns     |   1501 |   1501 |   1501        |     0         |   1501    |   1501    |\n",
      "| dt     |   2000 |   2000 |   2000        |     0         |   2000    |   2000    |\n",
      "| gain   |      3 |      3 |      3        |     0         |      3    |      3    |\n",
      "| igc    |      1 |      1 |      1        |     0         |      1    |      1    |\n",
      "| afilf  |    207 |    207 |    207        |     0         |    207    |    207    |\n",
      "| afils  |    298 |    298 |    298        |     0         |    298    |    298    |\n",
      "| hcf    |    207 |    207 |    207        |     0         |    207    |    207    |\n",
      "| hcs    |    298 |    298 |    298        |     0         |    298    |    298    |\n",
      "| year   |   1998 |   1998 |   1998        |     0         |   1998    |   1998    |\n",
      "+--------+--------+--------+---------------+---------------+-----------+-----------+\n"
     ]
    }
   ],
   "source": [
    "thstat = sio.log_thstat(traces=trace_headers)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "56f5c0da-8342-4744-bc74-13a63e67545f",
   "metadata": {},
   "source": [
    "Similar to logging the binary file header, only trace header mnemonics whose values aren't all zero are shown here to keep the list reasonably short. The output here is somewhat similar to the SU command *surange* but does not only show the minimum and maximum value but also other statistics.\n",
    "\n",
    "You might wonder what trace header mnemonics are available in the first place. We can easily query and log the (in this case standard) trace header definition setup:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "4966d28b-55ca-4671-abb0-643e5c82ebed",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO: Trace header definition:\n",
      "+------------+--------+--------+------------------------------------------------------------------+\n",
      "|            |   byte | type   | description                                                      |\n",
      "|------------+--------+--------+------------------------------------------------------------------|\n",
      "| tracl      |      0 | i      | Trace sequence number within line                                |\n",
      "| tracr      |      4 | i      | Trace sequence number within file                                |\n",
      "| fldr       |      8 | i      | Original field record number                                     |\n",
      "| tracf      |     12 | i      | Trace number within original field record                        |\n",
      "| ep         |     16 | i      | Energy source point number                                       |\n",
      "| cdp        |     20 | i      | Ensemble number                                                  |\n",
      "| cdpt       |     24 | i      | Trace number within ensemble                                     |\n",
      "| trid       |     28 | h      | Trace identification code                                        |\n",
      "| nvs        |     30 | h      | Number of vertically summed traces                               |\n",
      "| nhs        |     32 | h      | Number of horizontally summed traces                             |\n",
      "| duse       |     34 | h      | Data use                                                         |\n",
      "| offset     |     36 | i      | Distance from center of source point to center of receiver group |\n",
      "| gelev      |     40 | i      | Receiver group elevation from sea level                          |\n",
      "| selev      |     44 | i      | Surface elevation at source                                      |\n",
      "| sdepth     |     48 | i      | Source depth below surface                                       |\n",
      "| gdel       |     52 | i      | Datum elevation at receiver group                                |\n",
      "| sdel       |     56 | i      | Datum elevation at source                                        |\n",
      "| swdep      |     60 | i      | Water depth at source                                            |\n",
      "| gwdep      |     64 | i      | Water depth at receiver group                                    |\n",
      "| scalel     |     68 | h      | Scalar to be applied to elevations and depths                    |\n",
      "| scalco     |     70 | h      | Scalar to be applied to coordinates                              |\n",
      "| sx         |     72 | i      | Source coodinate x                                               |\n",
      "| sy         |     76 | i      | Source coordinate y                                              |\n",
      "| gx         |     80 | i      | Group coordinate x                                               |\n",
      "| gy         |     84 | i      | Group coordinate y                                               |\n",
      "| counit     |     88 | h      | Coordinate units                                                 |\n",
      "| wevel      |     90 | h      | Weathering velocity                                              |\n",
      "| swevel     |     92 | h      | Subewathering velocity                                           |\n",
      "| sut        |     94 | h      | Uphole time at source (ms)                                       |\n",
      "| gut        |     96 | h      | Uphole time at receiver group                                    |\n",
      "| sstat      |     98 | h      | Source static correction                                         |\n",
      "| gstat      |    100 | h      | Group static correction                                          |\n",
      "| tstat      |    102 | h      | Total static applied                                             |\n",
      "| laga       |    104 | h      | Lag time A                                                       |\n",
      "| lagb       |    106 | h      | Lag time B                                                       |\n",
      "| delrt      |    108 | h      | Delay recording time                                             |\n",
      "| muts       |    110 | h      | Mute time start                                                  |\n",
      "| mute       |    112 | h      | Mute time end                                                    |\n",
      "| ns         |    114 | H      | Number of samples                                                |\n",
      "| dt         |    116 | H      | Sample interval                                                  |\n",
      "| gain       |    118 | h      | Gain type of field instrument code                               |\n",
      "| igc        |    120 | h      | Instrument gain constant                                         |\n",
      "| igi        |    122 | h      | Instrument initial gain                                          |\n",
      "| corr       |    124 | h      | Correlated                                                       |\n",
      "| sfs        |    126 | h      | Sweep frequency start                                            |\n",
      "| sfe        |    128 | h      | Sweep frequency end                                              |\n",
      "| slen       |    130 | h      | Sweep length                                                     |\n",
      "| styp       |    132 | h      | Sweep type code                                                  |\n",
      "| stas       |    134 | h      | Sweep trace length at start                                      |\n",
      "| stae       |    136 | h      | Sweep trace length at end                                        |\n",
      "| tatyp      |    138 | h      | Taper type                                                       |\n",
      "| afilf      |    140 | h      | Alias filter frequency                                           |\n",
      "| afils      |    142 | h      | Alias filter slope                                               |\n",
      "| nofilf     |    144 | h      | Notch filter frequency                                           |\n",
      "| nofils     |    146 | h      | Notch filter slope                                               |\n",
      "| lcf        |    148 | h      | Low-cut frequency                                                |\n",
      "| hcf        |    150 | h      | High-cut frequency                                               |\n",
      "| lcs        |    152 | h      | Low-cut slope                                                    |\n",
      "| hcs        |    154 | h      | High-cut slope                                                   |\n",
      "| year       |    156 | h      | Year data recorded                                               |\n",
      "| day        |    158 | h      | Day of year                                                      |\n",
      "| hour       |    160 | h      | Hour of day                                                      |\n",
      "| minute     |    162 | h      | Minute of hour                                                   |\n",
      "| sec        |    164 | h      | Second of minute                                                 |\n",
      "| timbas     |    166 | h      | Time basis code                                                  |\n",
      "| trwf       |    168 | h      | Trace weighting factor                                           |\n",
      "| grnors     |    170 | h      | Geophone group number of roll switch                             |\n",
      "| grnofr     |    172 | h      | Geophone group number of trace one within original field record  |\n",
      "| grnlof     |    174 | h      | Geophone group number of last trace within original field record |\n",
      "| gaps       |    176 | h      | Gap size                                                         |\n",
      "| otrav      |    178 | h      | Overtravel taper code                                            |\n",
      "| cdpx       |    180 | i      | X-coordinate of ensemble                                         |\n",
      "| cdpy       |    184 | i      | Y-coordinate of ensemble                                         |\n",
      "| iline      |    188 | i      | Inline number                                                    |\n",
      "| xline      |    192 | i      | Crossline number                                                 |\n",
      "| sp         |    196 | i      | Shotpoint number                                                 |\n",
      "| spscal     |    200 | h      | Scalar to be applied to shotpoint number                         |\n",
      "| tvmu       |    202 | h      | Trace value measurement unit                                     |\n",
      "| trdman     |    204 | i      | Transduction constant mantissa                                   |\n",
      "| trdexp     |    208 | h      | Transduction constant exponent                                   |\n",
      "| trdun      |    210 | h      | Transduction unit                                                |\n",
      "| dti        |    212 | h      | Device identifier                                                |\n",
      "| timscal    |    214 | h      | Scalar to be applied to times (bytes 95-114)                     |\n",
      "| stypor     |    216 | h      | Source type and orientation                                      |\n",
      "| sedirv     |    218 | h      | Source energy direction vertical                                 |\n",
      "| sedirx     |    220 | h      | Source energy direction crossline                                |\n",
      "| sediri     |    222 | h      | Source energy direction inline                                   |\n",
      "| smman      |    224 | i      | Source measurement mantissa                                      |\n",
      "| smexp      |    228 | h      | Source measurement exponent                                      |\n",
      "| smun       |    230 | h      | Source measurement unit                                          |\n",
      "| unassigned |    232 | q      | Binary zeros or SEG00000, ASCII or EBCDIC                        |\n",
      "+------------+--------+--------+------------------------------------------------------------------+\n"
     ]
    }
   ],
   "source": [
    "sio.log_thdef()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d46080b-53d9-44d2-b253-a5bb1d0b96f9",
   "metadata": {},
   "source": [
    "Now let's read some data and explore the various ways this can be done. First, we simply read the entire data set into memory:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "b61aa814",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO: Reading entire file (71284 traces) from disk...\n",
      "INFO: Reading all traces took 0.1 seconds.\n",
      "INFO: Converting IBM floats to IEEE floats.\n"
     ]
    }
   ],
   "source": [
    "dataset = sio.read_all_traces()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "351b1dcc",
   "metadata": {},
   "source": [
    "Note: All read-functions convert the data from IBM floats to IEEE floats while keeping the original endianess. The data set could easily be swapped using `dataset.view(dataset.dtype.newbyteorder()).byteswap()`. The information about the types of the various trace header mnemonics is readily available (see `dataset.dtype`).\n",
    "\n",
    "Accessing indiviual trace headers for all traces just read from disk is as simple as:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "ae54afdd",
   "metadata": {},
   "outputs": [],
   "source": [
    "fldr = dataset[\"fldr\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1c3034bb-95d8-4d03-9430-923e230422a9",
   "metadata": {},
   "source": [
    "Just as simple is setting trace headers. This can be done by providing a vector of length `ntraces` with the values to be used, or we can provide a scalar value which will then be used for all the traces. We can also use other trace header mnemonics and perform some maths:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "a322df71-b073-4b84-a905-5c6c58a85f16",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO: Setting a variable offset using an array.\n",
      "INFO: Offsets now: [    0     1     2 ... 71281 71282 71283]\n",
      "INFO: Setting a constant offset using a scalar.\n",
      "INFO: Offsets now: [0 0 0 ... 0 0 0]\n",
      "INFO: Setting 'ep' as twice 'fldr'.\n",
      "INFO: 'fldr': [231 231 231 ... 481 481 481]\n",
      "INFO: 'ep': [462 462 462 ... 962 962 962]\n"
     ]
    }
   ],
   "source": [
    "log.info(\"Setting a variable offset using an array.\")\n",
    "dataset[\"offset\"] = range(ntraces)\n",
    "log.info(\"Offsets now: %s\", dataset[\"offset\"])\n",
    "log.info(\"Setting a constant offset using a scalar.\")\n",
    "dataset[\"offset\"] = 0\n",
    "log.info(\"Offsets now: %s\", dataset[\"offset\"])\n",
    "log.info(\"Setting 'ep' as twice 'fldr'.\")\n",
    "dataset[\"ep\"] = 2 * dataset[\"fldr\"] \n",
    "log.info(\"'fldr': %s\", dataset[\"fldr\"])\n",
    "log.info(\"'ep': %s\", dataset[\"ep\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30afee68-b45f-4c64-b6e5-52c1d583cd38",
   "metadata": {},
   "source": [
    "We can also filter the data based on trace header mnemonics. In this example, we only keep traces whose `trid` trace header mnemonic is greater than zero:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "7de585b5-868c-492c-82be-4d2e07c06f1f",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO: Length of data set: original = 71284, filtered = 70782\n"
     ]
    }
   ],
   "source": [
    "data_live = dataset[dataset[\"trid\"] > 0]\n",
    "log.info(\"Length of data set: original = %d, filtered = %d\", len(dataset), len(data_live))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a93a3fa4-cd54-4ef7-8f90-53c0b7131a1d",
   "metadata": {},
   "source": [
    "We can also add a new trace header mnemonic to our data set. Let's create a rather unusual header mnemonic of type 'bool' and initialize it to 'True' for the entire data set:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "0c1cddc3",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO: First / last 'myheader' value: True / True\n"
     ]
    }
   ],
   "source": [
    "data_live_updated = seisio.tools.add_mnemonic(data_live, names=\"myheader\", data=True, dtypes=bool)\n",
    "log.info(\"First / last 'myheader' value: %s / %s\", data_live_updated[\"myheader\"][0], data_live_updated[\"myheader\"][-1])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b3854eec-e614-4f3d-9654-23ea7be79d32",
   "metadata": {},
   "source": [
    "The actual seismic data are stored under the `data` mnemonic. When you access it, you get a 2D Numpy array of shape `(ntraces, nsamples)`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "4083f63c",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO: Shape of seismic data array: (71284, 1501)\n"
     ]
    }
   ],
   "source": [
    "seis = dataset[\"data\"]\n",
    "log.info(\"Shape of seismic data array: %s\", seis.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ffeb132a-579e-45de-8a7c-8ada54410199",
   "metadata": {},
   "source": [
    "Instead of reading the whole data set, we can read individual trace numbers. The trace numbers are zero-based. Let's read five specific traces and cross-check what we've read by comparing the `tracl` header mnemonic values (one-based) with the traces we requested to read. If we subtract one from the header `tracl`, the numbers should agree:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "521c85c5",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO: Traces to read: [0, 17821, 35642, 53463, 71283]\n",
      "INFO: Reading 5 specific trace(s) from disk...\n",
      "INFO: Trace numbers we just read: [    0 17821 35642 53463 71283]\n"
     ]
    }
   ],
   "source": [
    "traces_to_read = [0, ntraces//4, ntraces//2, 3*ntraces//4, ntraces-1]\n",
    "log.info(\"Traces to read: %s\", traces_to_read)\n",
    "traces = sio.read_traces(*traces_to_read)\n",
    "log.info(\"Trace numbers we just read: %s\", traces[\"tracl\"]-1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "161b9ff9-905d-4124-ac66-432845ca22e5",
   "metadata": {},
   "source": [
    "Using the *read_batch_of_traces()* method we can read consecutive traces starting from a particular position in the file. In this example, let's read 200 traces starting at trace 99 (the 100th trace in the file). If we again check the `tracl` header mnemonic afterwards, we would expect the first trace to have a value of 100, and the last trace to have a value of 299:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "e7c48894",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO: Reading 200 trace(s) from disk starting at trace 99...\n",
      "INFO: First / last 'tracl' value: 100 / 299\n"
     ]
    }
   ],
   "source": [
    "traces_batch = sio.read_batch_of_traces(start=99, ntraces=200)\n",
    "log.info(\"First / last 'tracl' value: %d / %d\", traces_batch[\"tracl\"][0], traces_batch[\"tracl\"][-1])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5fa0ecdd-9b8a-4ac3-a44b-7553c5fb58ea",
   "metadata": {},
   "source": [
    "We can read multiple blocks of consecutive traces, for instance four blocks of two traces each, starting at trace 0, with a stride of 1000 traces (between the beginnings of each block). Again, we log the `tracl` header mnemonic afterwards to check that we have indeed read the correct traces that were requested:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "1647a21a",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO: Reading 4 block(s) of 2 trace(s) from disk, starting at 0 with stride 1000...\n",
      "INFO: 'tracl' headers: [   1    2 1001 1002 2001 2002 3001 3002]\n"
     ]
    }
   ],
   "source": [
    "traces_mbatch = sio.read_multibatch_of_traces(start=0, block=2, stride=1000, count=4)\n",
    "log.info(\"'tracl' headers: %s\", traces_mbatch[\"tracl\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fd405442-faa0-4b5c-9084-d48fe846715b",
   "metadata": {},
   "source": [
    "One of the simplest ways to just loop through the file and reading a constant number of traces at the time is the *batches()* method that yields a subset of the data each time:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "468c7ef6",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO: Reading 10000 trace(s) from disk starting at trace 0...\n",
      "INFO: Batch #0, number of traces: 10000\n",
      "INFO: Reading 10000 trace(s) from disk starting at trace 10000...\n",
      "INFO: Batch #1, number of traces: 10000\n",
      "INFO: Reading 10000 trace(s) from disk starting at trace 20000...\n",
      "INFO: Batch #2, number of traces: 10000\n",
      "INFO: Reading 10000 trace(s) from disk starting at trace 30000...\n",
      "INFO: Batch #3, number of traces: 10000\n",
      "INFO: Reading 10000 trace(s) from disk starting at trace 40000...\n",
      "INFO: Batch #4, number of traces: 10000\n",
      "INFO: Reading 10000 trace(s) from disk starting at trace 50000...\n",
      "INFO: Batch #5, number of traces: 10000\n",
      "INFO: Reading 10000 trace(s) from disk starting at trace 60000...\n",
      "INFO: Batch #6, number of traces: 10000\n",
      "INFO: Reading 1284 trace(s) from disk starting at trace 70000...\n",
      "INFO: Batch #7, number of traces: 1284\n"
     ]
    }
   ],
   "source": [
    "for i, tr in enumerate(sio.batches(batch_size=10000)):\n",
    "    log.info(\"Batch #%d, number of traces: %d\", i, len(tr))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "85ca19cb-14fc-41a9-b522-ee75f790d945",
   "metadata": {},
   "source": [
    "The last batch of traces is obviously smaller than the requested batch size as there weren't enough traces left in the file to provide 10,000 traces.\n",
    "\n",
    "We can also loop through the file one trace at the time but keep in mind that this isn't very efficient. Here, we log the `tracl` header mnemonic every 10,000th trace:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "f49a99a8",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO: Trace #0, header 'tracl': 1\n",
      "INFO: Trace #10000, header 'tracl': 10001\n",
      "INFO: Trace #20000, header 'tracl': 20001\n",
      "INFO: Trace #30000, header 'tracl': 30001\n",
      "INFO: Trace #40000, header 'tracl': 40001\n",
      "INFO: Trace #50000, header 'tracl': 50001\n",
      "INFO: Trace #60000, header 'tracl': 60001\n",
      "INFO: Trace #70000, header 'tracl': 70001\n"
     ]
    }
   ],
   "source": [
    "for i, tr in enumerate(sio.traces(silent=True)):\n",
    "    if i % 10000 == 0:\n",
    "        log.info(\"Trace #%d, header 'tracl': %d\", i, tr[\"tracl\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "247e79f1-c65c-407a-ac4b-af8b809a188b",
   "metadata": {},
   "source": [
    "In particular with prestack data we would probably like to read the data as ensembles (traces that share a certain header mnemonic value). To do so, we have to create a lookup index. We can use a filter function to remove traces with `trid` header values of 0 ('unknown' traces). Let's group ensembles by `fldr` (field records, i.e., shots) and sort the traces within each ensemble by header mnemonic `tracf` (trace number within field record) in descending order:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "5bebac40",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO: Ensemble lookup index has filter applied.\n",
      "INFO: Created lookup index for ['fldr'] (order '>').\n",
      "INFO: Each ensemble is sorted by ['tracf'] (order '<').\n",
      "INFO: Number of ensembles: 251\n"
     ]
    }
   ],
   "source": [
    "sio.create_index(headers=trace_headers, group_by=\"fldr\", sort_by=\"tracf\", sort_order=\"<\", filt=(lambda x: x['trid'] > 0))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "94b9de38-8989-4b23-b63b-ed61eddb6c5f",
   "metadata": {},
   "source": [
    "There are 251 ensembles (once the filter function is applied). We can now get various properties related to the ensembles:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "e7e55f22",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO: Maximum number of traces within all the ensembles: 282\n"
     ]
    }
   ],
   "source": [
    "nensembles = sio.nensembles        # number of ensembles, or sio.ne for short\n",
    "ntraces_per_ensemble = sio.nte     # vector containing number of traces per ensemble\n",
    "max_ntraces = sio.maxnte           # size (no. of traces) of largest ensemble\n",
    "ensemble_keys = sio.ensemble_keys  # keys to identify the different ensembles\n",
    "log.info(\"Maximum number of traces within all the ensembles: %d\", max_ntraces)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0a6bb2f5-68b2-4635-903c-1d0a17be25c9",
   "metadata": {},
   "source": [
    "Apparently, the largest ensemble has 282 traces. We could now loop through the ensembles ourselves using the ensemble keys (see above), but it's much more convenient to use the following way which will provide us with one ensemble at the time in the order we requested when creating the index:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "8b70b100",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO: Ensemble #0, 'fldr' 231, first / last 'tracf' value: 282 / 1\n",
      "INFO: Ensemble #50, 'fldr' 281, first / last 'tracf' value: 282 / 1\n",
      "INFO: Ensemble #100, 'fldr' 331, first / last 'tracf' value: 282 / 1\n",
      "INFO: Ensemble #150, 'fldr' 381, first / last 'tracf' value: 282 / 1\n",
      "INFO: Ensemble #200, 'fldr' 431, first / last 'tracf' value: 282 / 1\n",
      "INFO: Ensemble #250, 'fldr' 481, first / last 'tracf' value: 282 / 1\n"
     ]
    }
   ],
   "source": [
    "for i, ens in enumerate(sio.ensembles(silent=True)):\n",
    "    if i % 50 == 0:\n",
    "        log.info(\"Ensemble #%d, 'fldr' %d, first / last 'tracf' value: %d / %d\", i, ens[\"fldr\"][0], ens[\"tracf\"][0], ens[\"tracf\"][-1])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2d29b5f7-deb8-4837-9c04-005fbb14a1aa",
   "metadata": {},
   "source": [
    "As can be seen, the ensembles start at the smallest `fldr` (shot number) and increase, while the traces within each ensemble are sorted by decreasing `tracf` header mnemonic, as requested. We could also use more than one trace header menmonic to group the data, or more than one trace header mnemonic to sort the data.\n",
    "\n",
    "As final part of this tutorial, let's create a simple plot of the last ensemble we have just read:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "02718c5d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 424816 characters]",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(1, 1)\n",
    "data = ens[\"data\"]\n",
    "vmm = np.percentile(data, 97.0)\n",
    "norm = cm.colors.Normalize(vmax=vmm, vmin=-vmm)\n",
    "ax.imshow(data.T, cmap=\"seismic\", origin=\"upper\", aspect=\"auto\", norm=norm,\n",
    "          extent=[ens[\"tracf\"][0], ens[\"tracf\"][-1], sampling_times[-1], sampling_times[0]])\n",
    "ax.set_title(f\"Shot #{ens['fldr'][0]}\")\n",
    "ax.set_xlabel(\"Trace number within field record\")\n",
    "ax.set_ylabel(\"Time in s\")\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a4bcbb9c-37e6-4146-9ff9-f70d67ce3824",
   "metadata": {},
   "source": [
    "That's it for now."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b9f41601-7ddc-4984-9b5f-c2dd729300d1",
   "metadata": {},
   "source": [
    "Note: We provide a Python package called __seisplot__ to provide a convenient way to display seismic data as image or wiggle plots."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7eaed4cc-b5fe-4eed-bc54-73496514c954",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "jupytext": {
   "cell_metadata_filter": "-all",
   "executable": "/usr/bin/env python3",
   "main_language": "python",
   "notebook_metadata_filter": "-all"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.19"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
