API Reference¶
intake_elasticsearch.ESSeqPlugin() |
Plugin for ElasticSearch to sequence reader |
intake_elasticsearch.ESTablePlugin() |
Plugin for ElasticSearch to pandas reader |
intake_elasticsearch.elasticsearch_table.ElasticSearchTableSource(…) |
Data source which executes arbitrary queries on ElasticSearch |
intake_elasticsearch.elasticsearch_seq.ElasticSearchSeqSource(…) |
Data source which executes arbitrary queries on ElasticSearch |
-
class
intake_elasticsearch.ESSeqPlugin[source]¶ Plugin for ElasticSearch to sequence reader
Methods
open(query, **kwargs)Create ElasticSearchSource instance separate_base_kwargs -
open(query, **kwargs)[source]¶ Create ElasticSearchSource instance
- Parameters:
- query : str
- Query string (lucene syntax or JSON text)
- qargs: dict
- Set of modifiers to apply to the query (https://elasticsearch-py.readthedocs.io/en/master/api.html#elasticsearch.Elasticsearch.search)
- kwargs (dict):
- Additional parameters to pass to ElasticSearch init.
-
-
class
intake_elasticsearch.ESTablePlugin[source]¶ Plugin for ElasticSearch to pandas reader
Methods
open(query, **kwargs)Create ElasticSearchSource instance separate_base_kwargs
-
class
intake_elasticsearch.elasticsearch_table.ElasticSearchTableSource(*args, **kwargs)[source]¶ Data source which executes arbitrary queries on ElasticSearch
This is the tabular reader: will return dataframes. Nested return items will become dict-like objects in the output.
Parameters: - query: str
Query to execute. Can either be in Lucene single-line format, or a JSON structured query (presented as text)
- qargs: dict
Further parameters to pass to the query, such as set of indexes to consider, filtering, ordering. See http://elasticsearch-py.readthedocs.io/en/master/api.html#elasticsearch.Elasticsearch.search
- es_kwargs: dict
Settings for the ES connection, e.g., a simple local connection may be
{'host': 'localhost', 'port': 9200}. Other keywords to the Plugin that end up here and are material:- scroll: str
how long the query is live for, default
'100m'- size: int
the paging size when downloading, default 1000.
- metadata: dict
Extra information for this source.
Attributes: - plot
Methods
close()Close open resources corresponding to this data source. discover()Open resource and populate the source attributes. read()Load entire dataset into a container and return it read_chunked()Return iterator over container fragments of data source read_partition(i)Return a (offset_tuple, container) corresponding to i-th partition. to_dask()Make single-partition lazy dask data-frame
-
class
intake_elasticsearch.elasticsearch_seq.ElasticSearchSeqSource(query, qargs, es_kwargs, metadata)[source]¶ Data source which executes arbitrary queries on ElasticSearch
This is the tabular reader: will return dataframes. Nested return items will become dict-like objects in the output.
Parameters: - query: str
Query to execute. Can either be in Lucene single-line format, or a JSON structured query (presented as text)
- qargs: dict
Further parameters to pass to the query, such as set of indexes to consider, filtering, ordering. See http://elasticsearch-py.readthedocs.io/en/master/api.html#elasticsearch.Elasticsearch.search
- es_kwargs: dict
Settings for the ES connection, e.g., a simple local connection may be
{'host': 'localhost', 'port': 9200}. Other keywords to the Plugin that end up here and are material:- scroll: str
how long the query is live for, default
'100m'- size: int
the paging size when downloading, default 1000.
- metadata: dict
Extra information for this source.
Attributes: - plot
Methods
close()Close open resources corresponding to this data source. discover()Open resource and populate the source attributes. read()Load entire dataset into a container and return it read_chunked()Return iterator over container fragments of data source read_partition(i)Return a (offset_tuple, container) corresponding to i-th partition. to_dask()Return a dask container for this data source