vd(1)

NAME
VisiData - a terminal utility for exploring and arranging tabular data

SYNOPSIS
vd [options] [input ...]
vd [options] --play cmdlog [-w waitsecs] [--batch] [-o output] [field=value ...]

DESCRIPTION
VisiData is a multipurpose tool for exploring, cleaning, editing, and restructuring data. Rows can be selected, filtered, and grouped; columns can be rearranged, transformed, and derived via regex or Python expressions; workflows can be saved, documented, and replayed. VisiData is built on the vdtui platform.

  REPLAY MODE
replays a saved cmdlog within the interface waits seconds between commands replays in batch mode (with no interface) saves final visible sheet to file as .tsv
--replay-movement=bool
    toggles --play to move cursor cell-by-cell
field=value
    replaces "{field}" in cmdlog contents with value

  Commands During Replay
^U  pauses/resumes replay
Tab executes next row in replaying sheet
^K  cancels current replay

  GLOBAL COMMANDS
In most cases, commands that affect 'selected rows' will affect all rows if no rows are selected.

  Keystrokes For The Cautious
F1   z?
    views this man page
gF1  gz?
    views all commands available on current sheet
^Q  aborts program immediately
^C  cancels user input or aborts all async threads on current sheet
q   quits current sheet
gq  quits all sheets (clean exit)

  Cursor Movement
Arrow PgUp Home
    moves as expected
h   j   k   l
    moves left/down/up/right
gh  gj  gk  gl
    moves all the way to the left/bottom/top/right
G  gg
    moves all the way to the bottom/top
^B  ^F
    scrolls one page back/forward

/   ? regex
    searches for regex forward/backward in current column
g/  g? regex
    searches for regex forward/backward over all visible columns
n   N
    moves to next/previous match from last search

<   >
    moves up/down the current column to the next value
{   }
    moves up/down the current column to the next selected row

c regex
    moves to the next column with name matching regex
r regex
    moves to the next row with key matching regex
zc  zr number
    moves to given column/row number

H   J   K   L
    slides current row/column left/down/up/right
gH  gJ  gK  gL
    slides current row/column all the way to the left/bottom/top/right of sheet

zh  zj  zk  zl
    scrolls one left/down/up/right
zt  zz  zb
    scrolls current row to top/center/bottom of screen

  Column Manipulation
_ (underscore)
    adjusts width of current column
g_  adjusts width of all visible columns
z_ number
    adjusts width of current column to number

- (hyphen)
    hides current column (to unhide, go to Columns sheet and edit its width)
z-  reduces width of current column by half

!   pins current column on the left as a key column
~  #  %  $  @
    sets type of current column to untyped/int/float/currency/date
^   edits name of current column
g^  sets names of all unnamed visible columns to contents of selected rows (or current row)
z^  sets name of current column to contents of current cell
gz^ sets name of current column to combined contents of current column for selected rows (or current row)

= expr
    creates new column from Python expr, with column names as variables
g= expr
    sets current column for selected rows to result of Python expr
gz= expr
    sets current column for selected rows to the items in result of Python sequence expr
z= expr
    sets current cell to result of evaluated Python expr on current row
' (tick)
    adds a frozen copy of current column with all cells evaluated
g'  opens a frozen copy of current sheet with all visible columns evaluated
z'  gz'
    adds/resets cache for current/all visible column(s)

: regex
    adds new columns from regex split; number of columns determined by example row at cursor
; regex
    adds new columns from capture groups of regex; also requires example row
* regex/subst
    replaces regex with subst, which may include backreferences (\1 etc)

  Row Selection
s   t   u
    selects/toggles/unselects current row
gs  gt  gu
    selects/toggles/unselects all rows
|   \ regex
    selects/unselects rows matching regex in current column
g|  g\ regex
    selects/unselects rows matching regex in any visible column
, (comma)
    selects rows matching current cell in current column
g,  selects rows matching this row in all visible columns

  Row Sorting/Filtering
[   ]
    sorts ascending/descending by current column
g[  g]
    sorts ascending/descending by all key columns
"   opens duplicate sheet with only selected rows
g"  opens duplicate sheet with all rows
gz" opens duplicate sheet with deepcopy of selected rows

  Editing Rows and Cells
a   appends a blank row
ga number
    appends number blank rows
d   gd
    deletes current/all selected row(s) and moves to clipboard
y   gy
    yanks (copies) current/all selected row(s) to clipboard
zy  yanks (copies) current cell to clipboard
p    P
    pastes clipboard rows after/before current row
zp  gzp
    sets contents of current column for current/all selected row(s) to last clipboard value
f   fills null cells in current column with contents of non-null cells up the current column
e text
    edits contents of current cell
ge text
    sets contents of current column for selected rows to text
zd   Del
    sets contents of current cell to None
gzd  gDel
    sets contents of cells in current column to None for selected rows

  Commands While Editing Input
Enter  ^C
    accepts/aborts input
^O  opens external $EDITOR to edit contents
^R  reloads initial value
^A  ^E
    moves to beginning/end of line
^B  ^F
    moves back/forward one character
^H  ^D
    deletes previous/current character
^T  transposes previous and current characters
^U  ^K
    clears from cursor to beginning/end of line
Backspace  Del
    deletes previous/current character
Insert
    toggles insert mode
Up  Down
    sets contents to previous/next in history
Tab  Shift-Tab
    autocompletes input (when available)

  Data Toolkit
o input
    opens input in VisiData
^S filename
    saves current sheet to filename in format determined by extension (default .tsv)
^D filename.vd
    saves CommandLog to filename.vd file
A number
    opens new blank sheet with number columns
R number
    opens duplicate sheet with a random population subset of number rows

+ aggregator
    adds aggregator to current column (see Frequency Table)
z+ aggregator
    displays result of aggregator over values in selected rows for current column

  Data Visualization
. (dot)
    plots current numeric column vs key columns. Numeric key column is used for the x-axis; categorical key column values determine color.
g.  plots a graph of all visible numeric columns vs key columns.

If rows on the current sheet represent plottable coordinates (as in .shp or vector .mbtiles sources), . plots the current row, and g. plots all selected rows (or all rows if none selected).

  Canvas-specific Commands
+   -
    increases/decreases zoomlevel, centered on cursor
_ (underscore)
    zooms to fit full extent
s   t   u
    selects/toggles/unselects rows on source sheet contained within canvas cursor
gs  gt  gu
    selects/toggles/unselects rows visible on screen
Enter
    opens sheet of source rows contained within canvas cursor
gEnter
    opens sheet of source rows visible on screen
1 - 9
    toggles display of layers
^L  redraws all pixels on canvas
w   toggles show_graph_labels option
mouse scrollwheel
    zooms in/out of canvas
left click-drag
    sets canvas cursor
right click-drag
    scrolls canvas

  Other Commands
V   views contents of current cell in a new TextSheet
v   toggles visibility (text wrap on TextSheet, legends/axes on Graph)
^^ (Ctrl-^)
    jumps to previous sheet (swaps with current sheet)
^E  views traceback for most recent error
g^E views traceback for most recent errors
z^E views traceback for error in current cell
^L  refreshes screen
^G  shows cursor position and bounds of current sheet on status line
^V  shows version information on status line
^R  reloads current sheet
z^R clears cache for current column
^Z  suspends VisiData process
^P  opens Status History

^Y  z^Y  g^Y
    opens current row/cell/sheet as Python object
^X expr
    evaluates Python expr and opens result as python object
z^X expr
    evaluates Python expr on current row and shows result on status line
g^X stmt
    executes Python stmt in the global scope

  Internal Sheets List
Metasheets
.   Columns Sheet (Shift-C) edit column properties
.   Sheets Sheet (Shift-S) jump between sheets or join them together
.   Options Sheet (Shift-O) edit all configurable options
.   Commandlog (Shift-D) modify and save commands for replay
.   Error Sheet (^E) view last error
.   Status History (^P) view history of status messages
.   Threads Sheet (^T) view, cancel, and profile asynchronous threads

Derived Sheets
.   Frequency Table (Shift-F) group rows by column value, with aggregations of other columns
.   Describe Sheet (Shift-I) view summary statistics for each column
.   Pivot Table (Shift-W) group rows by key and summarize current column
.   Melted Sheet (Shift-M) unpivot non-key columns into variable/value columns

  METASHEETS

  Columns Sheet (Shift-C)
Properties of columns on the source sheet can be changed with standard editing commands (e ge g= Del) on the Columns Sheet. Multiple aggregators can be set by listing them (separated by spaces) in the aggregators column. The 'g' commands affect the selected rows, which are actually the literal columns on the source sheet.
(global commands)
gC  opens Columns Sheet with all columns from all sheets
(sheet-specific commands)
&   adds column from concatenating selected source columns
!   pins current column on the left as a key column on source sheet
g!  pins selected columns on the left as key columns on source sheet
g+  adds aggregator to selected source columns
g_ (underscore)
    adjusts widths of selected columns on source sheet
g- (hyphen)
    hides selected columns on source sheet
~  #  %  $  @
    sets type of current column on source sheet to str/int/float/currency/date
g~ g# g% g$ g@
    sets type of selected columns on source sheet to str/int/float/currency/date
z~ gz~
    sets type of current/selected column(s) on source sheet to anytype
Enter
    opens a Frequency Table sheet grouped on column referenced in current row

  Sheets Sheet (Shift-S)
(sheet-specific commands)
Enter
    jumps to sheet referenced in current row
gC  opens Columns Sheet with all columns from selected sheets
& jointype
    merges selected sheets with visible columns from all, keeping rows according to jointype:
.   inner keeps only rows which match keys on all sheets
.   left keeps all rows from first selected sheet
.   full keeps all rows from all sheets (union)
.   diff keeps only rows NOT in all sheets
.   append keeps all rows from all sheets (concatenation)

  Options Sheet (Shift-O)
(sheet-specific commands)
Enter
    edits option

  CommandLog (Shift-D)
(sheet-specific commands)
x   replays command in current row
gx  replays contents of entire CommandLog
^C  aborts replay

  DERIVED SHEETS

  Frequency Table (Shift-F)
A Frequency Table groups rows by one or more columns, and includes summary columns for those with aggregators.
(global commands)
gF  opens Frequency Table, grouped by all key columns on source sheet
zF  opens one-line summary for selected rows
(sheet-specific commands)
s   t   u
    selects/toggles/unselects these entries in source sheet
Enter
    opens sheet of source rows which are grouped in current cell

  Describe Sheet (Shift-I)
(sheet-specific commands)
zs  zu
    selects/unselects rows on source sheet which are being described in current cell
!   pins current column on the left as a key column on source sheet
Enter
    opens a Frequency Table sheet grouped on column referenced in current row
zEnter
    opens copy of source sheet with rows described in current cell

  Pivot Table (Shift-W)
Set key column(s) and aggregators on column(s) before pressing Shift-W on the column to pivot.
(sheet-specific commands)
Enter
    opens sheet of source rows aggregated in current pivot row
zEnter
    opens sheet of source rows aggregated in current pivot cell

  Melted Sheet (Shift-M)
Opens melted sheet (unpivot), with all non-key columns reduced to Variable-Value rows.

  Python Object Sheet (^X ^Y g^Y z^Y)
(sheet-specific commands)
Enter
    dives further into Python object
e   edits contents of current cell
v   toggles whether methods and hidden properties are shown

COMMANDLINE OPTIONS
tsv uses loader for filetype instead of file extension True overwrites existing files without confirmation
--diff=base
    None adds colorizer for all sheets against base
--encoding=str               utf-8
    as passed to codecs.open
--encoding-errors=str        surrogateescape
    as passed to codecs.open
--regex-flags=str            I
    flags to pass to re.compile() [AILMSUX]
--default-width=int          20
    default column width
--wrap=bool                  False
    wrap text to fit window width on TextSheet
--cmd-after-edit=str         j
    command keystroke to execute after successful edit
--cmdlog-longname=bool       False
    Use command longname in cmdlog if available
--none-is-null=bool          True
    if Python None counts as null
--empty-is-null=bool         False
    if empty string counts as null
--false-is-null=bool         False
    if Python False counts as null
--zero-is-null=bool          False
    if integer 0 counts as null
--force-valid-colnames=bool  False
    clean column names to be valid Python identifiers
--debug=bool                 False
    exit on error and display stacktrace
--curses-timeout=int         100
    curses timeout in ms
--force-256-colors=bool      False
    use 256 colors even if curses reports fewer
--use-default-colors=bool    False
    set curses to use default terminal colors
--note-pending=str           ⌛
    note to display for pending cells
--note-format-exc=str       ?
    cell note for an exception during type conversion or formatting
--note-getter-exc=str       !
    cell note for an exception during computation
--scroll-incr=int            3
    amount to scroll with scrollwheel
--skip=int                   0
    skip first N lines of text input
--profile-threads=bool       True
    profile async threads
--min-memory-mb=int          0
    minimum memory to continue loading and async processing
--confirm-overwrite=bool     True
    whether to prompt for overwrite confirmation on save
--header=int                 1
    parse first N rows of .csv/.tsv as column names
--delimiter=str
    delimiter to use for tsv filetype
--filetype=str
    specify file type
--pyobj-show-hidden=bool     False
    show methods and _private properties
--replay-wait=float          0.0
    time to wait between replayed commands, in seconds
--replay-movement=bool       False
    insert movements during replay
--rowkey-prefix=str          キ
    string prefix for rowkey in the cmdlog
--regex-maxsplit=int         0
    maxsplit to pass to regex.split
--show-graph-labels=bool     True
    show axes and legend on graph
--plot-colors=str
    list of distinct colors to use for plotting distinct objects
--zoom-incr=float            2.0
    amount to multiply current zoomlevel by when zooming
--csv-dialect=str            excel
    dialect passed to csv.reader
--csv-delimiter=str         ,
    delimiter passed to csv.reader
--csv-quotechar=str          "
    quotechar passed to csv.reader
--csv-skipinitialspace=bool  True
    skipinitialspace passed to csv.reader
--fixed-rows=int             1000
    number of rows to check for fixed width columns

  DISPLAY OPTIONS
Display options can only be set via the Options Sheet or a config file (see FILES).

disp_none
    visible contents of a cell whose value is None
disp_date_fmt       %Y-%m-%d
    default fmtstr to strftime for date values
disp_truncator      …
    indicator that the contents are only partially visible
disp_oddspace       ·
    displayable character for odd whitespace
disp_unprintable   .
    substitute character for unprintables
disp_column_sep     |
    separator between columns
disp_keycol_sep     ‖
    separator between keys and rest of columns
disp_status_fmt     {sheet.name}|
    status line prefix
disp_status_sep      |
    separator between statuses
disp_edit_fill      _
    edit field fill character
disp_more_left      <
    header note indicating more columns to the left
disp_more_right     >
    header note indicating more columns to the right
disp_error_val      ¿
    displayed contents for computation exception
disp_ambig_width    1
    width to use for unicode chars marked ambiguous
color_default       normal
    the default color
color_default_hdr   bold underline
    color of the column headers
color_current_row   reverse
    color of the cursor row
color_current_col   bold
    color of the cursor column
color_current_hdr   reverse underline
    color of the header for the cursor column
color_column_sep    246 blue
    color of column separators
color_key_col       81 cyan
    color of key columns
color_selected_row  215 yellow
    color of selected rows
color_status        bold
    status line color
color_edit_cell     normal
    edit cell color
disp_pending
    string to display in pending cells
color_note_pending  bold magenta
    color of note of pending cells
color_note_type     226 green
    cell note for numeric types in anytype columns
color_format_exc    48 bold yellow
    color of formatting exception note
color_getter_exc    red bold
    color of computation exception note
disp_histogram      *
    histogram element character
disp_histolen       80
    width of histogram column
disp_replay_play    ▶
    status indicator for active replay
disp_replay_pause   ‖
    status indicator for paused replay
disp_pixel_random   False
    randomly choose attr from set of pixels instead of most common
color_graph_hidden  238 blue
    color of legend for hidden attribute
color_graph_axis    bold
    color for graph axis labels

EXAMPLES
    vd foo.tsv

    vd -f sqlite bar.db

    vd postgres://username:password@hostname:port/database

    vd --play tests/pivot.vd --replay-wait 1 --output tests/pivot.tsv

    ls -l | vd -f fixed --skip 1 --header 0

    vd --diff foo.tsv bar.tsv

FILES
At the start of every session, VisiData looks for $HOME/.visidatarc, and calls Python exec() on its contents if it exists. For example:

       options.min_memory_mb=100  # stop processing without 100MB free
       globalCommand('0', 'gh')   # alias '0' to move to first column, like vim
       def median(values):
           L = sorted(values)
           return L[len(L)//2]
       aggregator('median', median)

SUPPORTED SOURCES
These are the supported sources:

tsv (tab-separated value)
Plain and simple. VisiData writes tsv format by default. See the --delimiter option.

csv (comma-separated value)
.csv files are a scourge upon the earth, and still regrettably common.
See the --csv-dialect, --csv-delimiter, --csv-quotechar, and --csv-skipinitialspace options.
Accepted dialects are excel-tab, unix, and excel.

fixed (fixed width text)
Columns are autodetected from the first 1000 rows (adjustable with --fixed-rows).

json (single object) and jsonl (one object per line).
Cells containing lists (e.g. [3]) or dicts ({3}) can be opened into a new sheet with with z^Y.

The following URL schemes are supported:
http (requires requests); can be used as transport for with another filetype
postgres (requires psycopg2)

The following sources may include multiple tables. The initial sheet is the table directory; Enter loads the entire table into memory.

sqlite
xlsx (requires openpyxl)
xls (requires xlrd)
hdf5 (requires h5py)
shp (requires pyshp)
mbtiles (requires mapbox-vector-tile)
html (requires lxml)
In addition, .zip and .gz files are decompressed on-the-fly.

AUTHOR
VisiData was made by Saul Pwanson <vd@saul.pw>.