Python Library Documentation: module cosmosnosqlfreighter

NAME
    cosmosnosqlfreighter

CLASSES
    AgeFreighter(builtins.object)
        CosmosNoSQLFreighter
    
    class CosmosNoSQLFreighter(AgeFreighter)
     |  CosmosNoSQLFreighter() -> None
     |  
     |  A freighter class for loading data from Cosmos NoSQL databases.
     |  
     |  Method resolution order:
     |      CosmosNoSQLFreighter
     |      AgeFreighter
     |      builtins.object
     |  
     |  Methods defined here:
     |  
     |  async __aenter__(self) -> 'CosmosNoSQLFreighter'
     |      Enter the context manager.
     |  
     |  async __aexit__(self, exc_type: Any, exc: Any, tb: Any) -> None
     |      Exit the context manager.
     |  
     |  __init__(self) -> None
     |      Initialize the AgeFreighter object.
     |  
     |  async load(self, cosmos_endpoint: str = '', cosmos_key: str = '', cosmos_database: str = '', cosmos_container: str = '', id_map: Dict[Any, Any] = {}, graph_name: str = '', chunk_size: int = 128, direct_loading: bool = False, create_graph: bool = False, use_copy: bool = True, **kwargs: Any) -> None
     |      Load data from a Cosmos NoSQL database.
     |      
     |      Args:
     |          cosmos_endpoint (str): The Cosmos endpoint.
     |          cosmos_key (str): The Cosmos key.
     |          cosmos_database (str): The Cosmos database.
     |          cosmos_container (str): The Cosmos container.
     |          id_map (dict): The ID mapping dictionary.
     |          graph_name (str): The graph name to load the data into.
     |          chunk_size (int): Size of the chunks for processing.
     |          direct_loading (bool): Whether to load the data directly.
     |          create_graph (bool): Whether to create a new graph.
     |          use_copy (bool): Whether to use COPY protocol for loading.
     |          **kwargs: Additional keyword arguments, e.g. progress monitor.
     |      
     |      Returns:
     |          None
     |  
     |  ----------------------------------------------------------------------
     |  Static methods defined here:
     |  
     |  fetch_documents(cosmos_endpoint: str = '', cosmos_key: str = '', cosmos_database: str = '', cosmos_container: str = '', page_size: int = 12800) -> Dict[str, Dict[str, str]]
     |      Fetch documents from a Cosmos NoSQL database and store each group in temporary files.
     |      
     |      Args:
     |          cosmos_endpoint (str): The Cosmos endpoint.
     |          cosmos_key (str): The Cosmos key.
     |          cosmos_database (str): The Cosmos database.
     |          cosmos_container (str): The Cosmos container.
     |          page_size (int): The maximum number of items per page read.
     |      
     |      Returns:
     |          dict: A dictionary with keys "vertex" and "edge". Each key maps labels (or types)
     |                to file paths containing the corresponding documents.
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from AgeFreighter:
     |  
     |  async close(self) -> None
     |      Close the connection pool.
     |      
     |      Returns:
     |          None
     |  
     |  async connect(self, dsn: str = '', max_connections: int = 64, min_connections: int = 4, log_level=None, **kwargs) -> None
     |      Open a connection pool.
     |      
     |      Args:
     |          dsn (str): The data source name.
     |          max_connections (int): The maximum number of connections.
     |          min_connections (int): The minimum number of connections.
     |          log_level: The log level.
     |          **kwargs: Additional keyword arguments.
     |      
     |      Returns:
     |          None
     |  
     |  async copyChunk(self, chunk: pandas.core.frame.DataFrame, first_id: int = 0, graph_name: str = '', label: str = '', id_maps: dict = {}, is_edge: bool = False) -> None
     |      Copy a chunk of data to the PostgreSQL database.
     |      
     |      Args:
     |          chunk (pd.DataFrame): The chunk of data to copy.
     |          first_id (int): The first ID.
     |          graph_name (str): The name of the graph.
     |          label (str): The label of the data.
     |          id_maps (dict): The ID maps.
     |          is_edge (bool): Whether the data is an edge.
     |      
     |      Returns:
     |          None
     |  
     |  async copyEdges(self, edges: pandas.core.frame.DataFrame, edge_type: str = '', edge_props: list = [], chunk_size: int = 0) -> None
     |      Create edges in the PostgreSQL database via the COPY protocol.
     |      
     |      Args:
     |          edges (pd.DataFrame): The edges to create.
     |          edge_type (str): The type of the edges.
     |          edge_props (list): The properties of the edges.
     |          chunk_size (int): The size of the chunks to create.
     |      
     |      Returns:
     |          None
     |  
     |  async copyVertices(self, vertices: pandas.core.frame.DataFrame, vertex_label: str = '', chunk_size: int = 0) -> None
     |      Create vertices in the PostgreSQL database via the COPY protocol.
     |      
     |      Args:
     |          vertices (pd.DataFrame): The vertices to create.
     |          label (str): The label of the vertices.
     |          chunk_size (int): The size of the chunks to create.
     |      
     |      Returns:
     |          None
     |  
     |  createEdgeCypher(self, row: pandas.core.series.Series, edge_type: str = '', edge_props: list = []) -> str
     |      Create a Cypher query to create an edge.
     |      
     |      Args:
     |          row (pd.core.series.Series): The row to create the edge from.
     |          edge_type (str): The type of the edge.
     |          edge_props (list): The properties of the edge.
     |      
     |      Returns:
     |          str: The Cypher query to create the edge.
     |  
     |  async createEdges(self, edges: pandas.core.frame.DataFrame, edge_type: str = '', edge_props: list = [], chunk_size: int = 3, direct_loading: bool = False, use_copy: bool = False) -> None
     |      Create edges in the PostgreSQL database.
     |      
     |      Args:
     |          edges (pd.DataFrame): The edges to create.
     |          edge_type (str): The type of the edges.
     |          edge_props (list): The properties of the edges.
     |          chunk_size (int): The size of the chunks to create.
     |          direct_loading (bool): Whether to load the edges directly.
     |          use_copy (bool): Whether to use the COPY protocol to load the edges.
     |      
     |      Returns:
     |          None
     |  
     |  async createEdgesCypher(self, edges: pandas.core.frame.DataFrame, edge_type: str = '', edge_props: list = [], chunk_size: int = 0) -> None
     |      Create edges in the PostgreSQL database via Cypher.
     |      
     |      Args:
     |          edges (pd.DataFrame): The edges to create.
     |          edge_type (str): The type of the edges.
     |          edge_props (list): The properties of the edges.
     |          chunk_size (int): The size of the chunks to create.
     |      
     |      Returns:
     |          None
     |  
     |  async createEdgesDirectly(self, edges: pandas.core.frame.DataFrame, edge_type: str = '', edge_props: list = [], chunk_size: int = 0) -> None
     |      Create edges in the PostgreSQL database directly.
     |      
     |      Args:
     |          edges (pd.DataFrame): The edges to create.
     |          edge_type (str): The type of the edges.
     |          edge_props (list): The properties of the edges.
     |          chunk_size (int): The size of the chunks to create.
     |      
     |      Returns:
     |          None
     |  
     |  async createGraphFromDataFrame(self, graph_name: str, src: pandas.core.frame.DataFrame, existing_node_ids: list = [], first_chunk: bool = True, start_v_label: str = '', start_id: str = '', start_props: list[str] = [], edge_type: str = '', edge_props: list[str] = [], end_v_label: str = '', end_id: str = '', end_props: list[str] = [], chunk_size: int = 3, direct_loading: bool = False, create_graph: bool = False, use_copy: bool = False) -> None
     |      Create a graph from DataFrame
     |      
     |      Args:
     |          graph_name (str): The name of the graph to load the data into.
     |          src (pd.DataFrame): The DataFrame to load the data from.
     |          existing_node_ids (list): The existing node IDs.
     |          first_chunk (bool): Whether it is the first chunk.
     |          start_v_label (str): The label of the start vertex.
     |          start_id (str): The ID of the start vertex.
     |          start_props (list): The properties of the start vertex.
     |          edge_type (str): The type of the edge.
     |          edge_props (list): The properties of the edge.
     |          end_v_label (str): The label of the end vertex.
     |          end_id (str): The ID of the end vertex.
     |          end_props (list): The properties of the end vertex.
     |          chunk_size (int): The size of the chunks to create.
     |          direct_loading (bool): Whether to load the data directly.
     |          create_graph (bool): Whether to create the graph.
     |          use_copy (bool): Whether to use the COPY protocol to load the data.
     |      
     |      Returns:
     |          None
     |  
     |  async createLabelType(self, label_type: str = '', value: str = '') -> None
     |      Create a label type in the PostgreSQL database.
     |      
     |      Args:
     |          label_type (str): The type of the label to create. It can be either "vertex" or "edge".
     |          value (str): The value of the label to create.
     |      
     |      Returns:
     |          None
     |  
     |  createValuesDirectly(self, row: pandas.core.series.Series, edge_props: list = [], id_maps: dict = {}) -> str
     |      Create values directly.
     |      
     |      Args:
     |          row (pd.core.series.Series): The row to create the values from.
     |          edge_props (list): The properties of the edges.
     |          id_maps (dict): The ID maps.
     |      
     |      Returns:
     |          str: The values.
     |  
     |  async createVertices(self, vertices: pandas.core.frame.DataFrame, vertex_label: str = '', chunk_size: int = 3, direct_loading: bool = False, use_copy: bool = False) -> None
     |      Create vertices in the PostgreSQL database.
     |      
     |      Args:
     |          vertices (pd.DataFrame): The vertices to create.
     |          vertex_label (str): The label of the vertices.
     |          chunk_size (int): The size of the chunks to create.
     |          direct_loading (bool): Whether to load the vertices directly.
     |          use_copy (bool): Whether to use the COPY protocol to load the vertices.
     |      
     |      Returns:
     |          None
     |  
     |  async createVerticesCypher(self, vertices: pandas.core.frame.DataFrame, label: str = '', chunk_size: int = 0) -> None
     |      Create vertices in the PostgreSQL database via Cypher.
     |      
     |      Args:
     |          vertices (pd.DataFrame): The vertices to create.
     |          label (str): The label of the vertices.
     |          chunk_size (int): The size of the chunks to create.
     |      
     |      Returns:
     |          None
     |      
     |      Note:
     |          psycopg can not handle Cypher's UNWIND as of 9th Dec 2024.
     |  
     |  async createVerticesDirectly(self, vertices: pandas.core.frame.DataFrame, label: str = '', chunk_size: int = 0) -> None
     |      Create vertices in the PostgreSQL database directly.
     |      
     |      Args:
     |          vertices (pd.DataFrame): The vertices to create.
     |          label (str): The label of the vertices.
     |          chunk_size (int): The size of the chunks to create.
     |      
     |      Returns:
     |          None
     |  
     |  async executeQuery(self, query: str = '') -> None
     |      Execute a query with an async connection pool.
     |      
     |      Args:
     |          query (str): The query to execute.
     |      
     |      Returns:
     |          None
     |  
     |  async executeWithTasks(self, target: Callable, args: list = []) -> None
     |      Execute queries with tasks.
     |      
     |      Args:
     |          target (Callable): The target function to execute.
     |          args (list): The arguments to pass to the target function.
     |      
     |      Returns:
     |          None
     |  
     |  async getFirstId(self, graph_name: str = '', label_type: str = '') -> int
     |      Get the first id for a vertex or edge.
     |      
     |      Args:
     |          graph_name (str): The name of the graph to get the first id for.
     |          label_type (str): The type or the label to get the first id for.
     |      
     |      Returns:
     |          int: The first id.
     |  
     |  async getIdMaps(self, edges: pandas.core.frame.DataFrame) -> dict
     |      Get the idmaps between entry_id and id(graphid).
     |      
     |      Args:
     |          edges (pd.DataFrame): The edges to create.
     |      
     |      Returns:
     |          dict: The ID maps.
     |  
     |  processChunkDirectly(self, chunk: pandas.core.frame.DataFrame, query: str = '', edge_props: list = [], id_maps: dict = {}) -> list
     |      Process a chunk of data directly.
     |      
     |      Args:
     |          chunk (pd.DataFrame): The chunk of data to process.
     |          query (str): The query to execute.
     |          edge_props (list): The properties of the edges.
     |          id_maps (dict): The ID maps.
     |      
     |      Returns:
     |          list: The processed chunk of data.
     |  
     |  async setUpGraph(self, graph_name: str = '', create_graph: bool = False) -> None
     |      Set up the graph in the PostgreSQL database.
     |      
     |      Args:
     |          graph_name (str): The name of the graph to set up.
     |          create_graph (bool): Whether to create the graph.
     |      
     |      Returns:
     |          None
     |  
     |  showProgress(self, type: str = '', i: int = 0, total: int = 0) -> None
     |      Show the progress of the loading.
     |      
     |      Args:
     |          type (str): The type of the loading.
     |          i (int): The current index.
     |          total (int): The total number of items.
     |      
     |      Returns:
     |          None
     |  
     |  ----------------------------------------------------------------------
     |  Static methods inherited from AgeFreighter:
     |  
     |  checkKeys(keys: list = [], elements: list = [], error_msg: str = '') -> None
     |      Check if the keys of the CSV file match the elements.
     |      
     |      Args:
     |          keys (list): The keys of the CSV file.
     |          elements (list): The elements to check.
     |          error_msg (str): The error message to display.
     |      
     |      Returns:
     |          None
     |  
     |  quotedGraphName(graph_name: str = '') -> str
     |      Quote the graph name.
     |      
     |      Args:
     |          graph_name (str): The name of the graph.
     |      
     |      Returns:
     |          str: The quoted graph name
     |  
     |  ----------------------------------------------------------------------
     |  Data descriptors inherited from AgeFreighter:
     |  
     |  __dict__
     |      dictionary for instance variables (if defined)
     |  
     |  __weakref__
     |      list of weak references to the object (if defined)

DATA
    Any = typing.Any
        Special type indicating an unconstrained type.
        
        - Any is compatible with every type.
        - Any assumed to have all methods.
        - All values assumed to be instances of Any.
        
        Note that all the above statements are true from the point of view of
        static type checkers. At runtime, Any should not be used with instance
        or class checks.
    
    Dict = typing.Dict
        A generic version of dict.
    
    List = typing.List
        A generic version of list.
    
    log = <Logger cosmosnosqlfreighter (WARNING)>

FILE
    cosmosnosqlfreighter.py

