# Data Retrieval

## Getting Data
Getting data is one of the most basic tasks in PrismStudio. It allows us to retrieve the desired financial information for analysis and further processing. This guide will walk you through the process of obtaining data using PrismStudio's functionalities.

### Single data

To get started, we need to define a financial component as a variable in Python. In this example, we will retrieve daily closing price by using the [prismstudio.market.close](<#prismstudio.market.close>) function:

```python
>>> close = ps.market.close()
```

Once you have defined the financial component, there are two methods to retrieve the data: using the [prismstudio.get_data](<#prismstudio.get_data>) function or directly calling the **`get_data`** method on the financial item. In both cases, you need to specify the desired parameters, such as the universe, start date, end date, shown ID, and name.

Here is an example of using the [prismstudio.get_data](<#prismstudio.get_data>) function:

```python
>>> close_df = ps.get_data(
        component=close,
        universe='KOSPI 200 Index',
        startdate='2015-01-01',
        enddate='2020-12-31',
        shownid=['ticker']
    )
```

Alternatively, you can directly call the **`get_data`** method on the financial item:

```python
>>> close_df = close.get_data(
        universe='KOSPI 200 Index',
        startdate='2015-01-01',
        enddate='2020-12-31',
        shownid=['ticker'],
        name=['my_close']
    )
```

The result of the data query will be directly delivered to your Python environment and stored in the **`close_df`** variable as a pandas DataFrame. This allows you to access, analyze, and manipulate the data using familiar pandas functions for the downstream tasks.

Note that [prismstudio.get_data](<#prismstudio.get_data>) function and the **`get_data`** method have the same functionality and parameters.

### Multiple data

In addition to retrieving single data components, PrismStudio also allows you to retrieve multiple data components simultaneously. This can be useful when you need to obtain multiple sets of data for analysis or model building. The following example demonstrates how to retrieve multiple data components in a single query.

Let's consider a scenario where we want to retrieve the closing prices and 1-day percentage changes for a specific universe of stocks called **`my_snp_500`** between the dates **`2020-01-01`** and **`2020-12-31`**. We also want to include additional information such as company names and ISIN codes for each stock.


**Step 1: Define the Data Components**

First, we need to define the data components we want to retrieve. In this example, we'll use the following components:

```python
>>> c = ps.market.price_close()  # Closing price component (data component)
>>> r = c.n_periods_pct_change(n=1)  # 1-day percentage change component (function component)
```

The **`c`** component represents closing prices, and the **`r`** component calculates the 1-day percentage change based on the closing prices.


**Step 2: Retrieve Single Data (Optional)**

Before retrieving multiple data components, we can also choose to retrieve a single data component for a specific query. Here's an example of retrieving data for the **`r`** component using the **`get_data`** method:

```python
>>> r.get_data(universe='my_snp_500', startdate='2020-01-01', enddate='2020-12-31', name=["daily_return"], shownid=['Company Name', 'ISIN'])
```

This query retrieves the 1-day percentage change data for the specified universe and time period, and includes the company names and ISIN codes in the resulting DataFrame. With in the returned dataframe the data value column is named as **`daily_return`**.


**Step 3: Retrieve Multiple Data**

To retrieve multiple data components in a single query, we can use the [prismstudio.get_data](<#prismstudio.get_data>) function. Here's an example of retrieving both the closing prices (**`c`**) and 1-day percentage changes (**`r`**) for the specified universe and time period:

```python
>>> ps.get_data(
        component=[c, r],
        universe='my_snp_500',
        startdate='2020-01-01',
        enddate='2020-12-31',
        name=["close", "daily_return"],
        shownid=['Company Name', 'ISIN']
    )
```

In this query, we pass a list of data components **`[c, r]`** as the **`component`** parameter. We also provide the universe, start date, end date, name, and shown ID parameters to specify the query details.

```{tip}
When retrieving the data for both components simultaneously, PrismStudio recognizes the dependency and only pulls the necessary data once if there is an overlap. This approach reduces the I/O load and enhances the speed of data retrieval.
```


In the given example, the **`r`** component is derived from the **`c`** component, as it calculates the 1-day percentage change based on the closing prices. PrismStudio recognizes the overlap between **`r`** and **`c`** and only pulls the daily closing price once.

The result will be in a list of DataFrames that includes both the closing prices and 1-day percentage changes, along with the company names and ISIN codes.

## Exporting Data
### Data Export

In PrismStudio, you can easily export data using the [prismstudio.export_data](<#prismstudio.export_data>) function, enabling you to save data of specified components results in your cloud space for future use. While utilizing export data typically involves downloading the result, which returns zipped parquet files, you also have the option to directly retrieve the exported data into your Python environment using the **`retrieve_data`** function.

Here's a step-by-step guide on how to export data using Prism:

**Step 1. Define Data:**

Before exporting data, ensure you have defined the necessary data using prismstudio. In the provided code snippet, the **`close`** and **`open`** prices for a market are retrieved using [prismstudio.market.close](<#prismstudio.market.close>) and [prismstudio.market.open](<#prismstudio.market.open>).

```python
>>> close = ps.market.close()
>>> open = ps.market.open()
```

**Step 2. Prepare Export Data Query:**
To export the data, you need to create an export data query. In the provided code, the [prismstudio.export_data](<#prismstudio.export_data>) function is used to define the export query. It takes the following arguments:

```python
ed = prismstudio.export_data([close, open], "KRX_300", "2022-01-01")
```

- Data: Provide the data to be exported as a list. In this case, it includes **`close`** and **`open`**.
- Universe: Specify the universe or group of securities from which the data is extracted. In this example, it is "KRX_300".
- Start Date: Set the start date for the data to be exported. The provided date is **`2022-01-01`**.
- End Date: Since no enddate is given, it defaults to when the export data is run.

**Step 3. Run Export Data Query:**
The export data query is executed using the [prismstudio._ExportData.run](<#prismstudio._ExportData.run>) function on the export data object (**`ed`**). The [prismstudio._ExportData.run](<#prismstudio._ExportData.run>) function takes two parameters:

- Filepath: Specify the filepath where the exported data will be saved. In the given code, the data will be saved in a file named **`open_close`**.
- Filename: Provide a list of file names for the exported data within the zip file. In this case, the filenames are specified as **`close`**, **`open`**.

```python
>>> ed.run("open_close", ["close", "open"])

export_data is added to worker queue!
{'status': 'Pending',
'message': 'export_data is added to worker queue!',
'result': [{'resulttype': 'jobid', 'resultvalue': 465}]}
```

After executing the export data query, Prism will initiate the export job and provide information about the job status. In the provided code snippet, the response contains a dictionary with the following information:

- Status: Indicates the current status of the export data job. It starts as "Pending" and will change to "Completed" once the job is finished.
- Message: Provides a message indicating that the export data job has been added to the worker queue, which includes a job ID (**`jobid`**) for the export data job.
- Check Export Data Job Status: To ensure the export data job is completed, you can check the job status using the job ID. You can monitor the job status periodically until it changes to "Completed".

### Access Exported Data

**Retrieving Exported Data into Python Workspace**

Once you have exported the data from Prism and saved it in a file, you can access and load the exported data into your Python workspace. The [prismstudio.retrieve_datafiles](<#prismstudio.retrieve_datafiles>) function allows you to retrieve the exported data as a dictionary of DataFrames.

```python
>>> ps.retrieve_datafiles('open_close')

Exported Data test_single_q has components: ['close', 'open']
{'close':
         listingid       date         Close
0         20108704 2022-01-03  16188.903172
1         20108704 2022-01-04  16477.132902
2         20108704 2022-01-05  16092.826595
3         20108704 2022-01-06  15468.328847
4         20108704 2022-01-07  15804.596865
...            ...        ...           ...
666583  1816050654 2023-03-20  31700.000000
666584  1816050654 2023-03-21  31050.000000
666585  1816050654 2023-03-22  31800.000000
666586  1816050654 2023-03-23  34400.000000
666587  1816050654 2023-03-24  35750.000000
,
'open':
         listingid        date          Open
0         20108704  2022-01-03  16092.826595
1         20108704  2022-01-04  16284.979749
2         20108704  2022-01-05  16477.132902
3         20108704  2022-01-06  15852.635154
4         20108704  2022-01-07  15564.405424
...            ...         ...           ...
664804  1816050654  2023-03-20  30300.000000
664805  1816050654  2023-03-21  33050.000000
664806  1816050654  2023-03-22  31900.000000
664807  1816050654  2023-03-23  31350.000000
664808  1816050654  2023-03-24  35000.000000
}
```

In the provided code snippet, the [prismstudio.retrieve_datafiles](<#prismstudio.retrieve_datafiles>) function is used to retrieve the exported data. Pass the name of the exported data file, in this case **`open_close`**, as the argument to the function.

**Downloading Exported Data**

To download exported data from the web UI, you have two methods available. The first method is to use the "Finder" feature, while the second method involves using the "Job Manager" to locate the associated job ID for downloading the desired file. Please note that downloading files is supported exclusively through the web browser.

**Method 1: Using [prismstudio.finder](<#prismstudio.finder>) to Locate the File in the Data Files Tab and Download**

1. Launch the PrismStudio web UI by running the following code: [prismstudio.finder()](<#prismstudio.finder>). This will open the web UI in your default browser.
2. In the web UI, navigate to the "Data Files" tab. This tab is usually located at the top of the page. Click on the "Data Files" tab to access the data files management page.

![Untitled](../_static/english_guide/Untitled.png)

1. Once you have located the desired file, right-click on it to open the menu. From the righ-click menu, select the "Download" option. This will initiate the download process.

![Untitled](../_static/english_guide/Untitled1.png)

**Method 2: Using [prismstudio.job_manager](<#prismstudio.job_manager>) to Locate the Associated Job in the "ALL" Task Tab and "Data Export" Task Tab**

1. Launch the PrismStudio web UI by running the following code: [prismstudio.job_manager](<#prismstudio.job_manager>). This will open the web UI in your default browser.
2. On the job manager page, you will find a list of all the tasks that have been executed. Look for the "Data Export" task tab, which specifically displays the export data tasks.
3. In the "Data Export" task tab, locate the specific job that corresponds to the export data task you initiated. This can be done by searching for the job using filters such as date, task type, or any other available criteria.

![Untitled](../_static/english_guide/Untitled2.png)

1. Look for the download option, which is typically represented by an icon such as a download arrow or a "Download" button. Click on the download option to initiate the download process.

![Untitled](../_static/english_guide/Untitled3.png)

Please note that currently, the exported data file format in PrismStudio is parquet. In later releases, PrismStudio plans to support additional file formats for data export, providing more flexibility for users in choosing the file format that best suits their needs.