ydata-profiling_example

`filoma` recommends using `ydata-profiling` to generate a report for a given dataframe. Below is an example of how to do this in a Jupyter notebook.

In [1]:

    !uv pip install ydata-profiling ipywidgets ipympl setuptools --upgrade

    Using Python 3.11.11 environment at: /home/kalfasy/repos/filoma/.venv
    Resolved 70 packages in 469ms
    Resolved 70 packages in 469ms
    ⠙ Preparing packages... (0/43)
    ⠙ Preparing packages... (0/43)---     0 B/4.75 MiB
    ⠙ Preparing packages... (0/43)--- 16.00 KiB/4.75 MiB
    ⠙ Preparing packages... (0/43)--- 32.00 KiB/4.75 MiB
    ⠙ Preparing packages... (0/43)--- 48.00 KiB/4.75 MiB
    ⠙ Preparing packages... (0/43)--- 62.20 KiB/4.75 MiB
    ⠙ Preparing packages... (0/43)--- 78.20 KiB/4.75 MiB
    ⠙ Preparing packages... (0/43)--- 94.20 KiB/4.75 MiB
    ⠙ Preparing packages... (0/43)--- 110.20 KiB/4.75 MiB
    ⠙ Preparing packages... (0/43)--- 126.20 KiB/4.75 MiB
    ⠙ Preparing packages... (0/43)--- 142.20 KiB/4.75 MiB
    ⠙ Preparing packages... (0/43)--- 158.20 KiB/4.75 MiB
    ⠙ Preparing packages... (0/43)--- 174.20 KiB/4.75 MiB
    ⠙ Preparing packages... (0/43)--- 190.20 KiB/4.75 MiB
    ⠙ Preparing packages... (0/43)--- 206.20 KiB/4.75 MiB
    ⠙ Preparing packages... (0/43)--- 222.20 KiB/4.75 MiB
    ⠙ Preparing packages... (0/43)------- 238.20 KiB/4.75 MiB
    ⠙ Preparing packages... (0/43)---     0 B/4.75 MiB
    ⠙ Preparing packages... (0/43)--- 16.00 KiB/4.75 MiB
    ⠙ Preparing packages... (0/43)--- 32.00 KiB/4.75 MiB
    ⠙ Preparing packages... (0/43)--- 48.00 KiB/4.75 MiB
    ⠙ Preparing packages... (0/43)--- 62.20 KiB/4.75 MiB
    ⠙ Preparing packages... (0/43)--- 78.20 KiB/4.75 MiB
    ⠙ Preparing packages... (0/43)--- 94.20 KiB/4.75 MiB
    ⠙ Preparing packages... (0/43)--- 110.20 KiB/4.75 MiB
    ⠙ Preparing packages... (0/43)--- 126.20 KiB/4.75 MiB
    ⠙ Preparing packages... (0/43)--- 142.20 KiB/4.75 MiB
    ⠙ Preparing packages... (0/43)--- 158.20 KiB/4.75 MiB
    ⠙ Preparing packages... (0/43)--- 174.20 KiB/4.75 MiB
    ⠙ Preparing packages... (0/43)--- 190.20 KiB/4.75 MiB
    ⠙ Preparing packages... (0/43)--- 206.20 KiB/4.75 MiB
    ⠙ Preparing packages... (0/43)--- 222.20 KiB/4.75 MiB
    ⠙ Preparing packages... (0/43)--- 238.20 KiB/4.75 MiB
    ⠙ Preparing packages... (0/43)--- 254.20 KiB/4.75 MiB
    ⠙ Preparing packages... (0/43)--- 270.20 KiB/4.75 MiB
    ⠙ Preparing packages... (0/43)--- 718.20 KiB/4.75 MiB
    ⠙ Preparing packages... (0/43)--- 254.20 KiB/4.75 MiB
    ⠙ Preparing packages... (0/43)--- 270.20 KiB/4.75 MiB
    ⠙ Preparing packages... (0/43)--- 718.20 KiB/4.75 MiB
    ⠙ Preparing packages... (0/43)--- 1.28 MiB/4.75 MiB
    ⠹ Preparing packages... (42/43)-- 1.28 MiB/4.75 MiB
    ⠹ Preparing packages... (42/43)-- 1.79 MiB/4.75 MiB
    ⠹ Preparing packages... (42/43)-- 1.79 MiB/4.75 MiB
    ⠹ Preparing packages... (42/43)-- 2.36 MiB/4.75 MiB
    ⠹ Preparing packages... (42/43)-- 2.36 MiB/4.75 MiB
    ⠹ Preparing packages... (42/43)-- 2.92 MiB/4.75 MiB
    ⠹ Preparing packages... (42/43)-- 2.92 MiB/4.75 MiB
    ⠹ Preparing packages... (42/43)-- 3.48 MiB/4.75 MiB
    ⠸ Preparing packages... (42/43)-- 3.48 MiB/4.75 MiB
    ⠸ Preparing packages... (42/43)-- 4.02 MiB/4.75 MiB
    ⠸ Preparing packages... (42/43)-- 4.02 MiB/4.75 MiB
    ⠸ Preparing packages... (42/43)m- 4.56 MiB/4.75 MiB
    Prepared 43 packages in 466ms
    Uninstalled 10 packages in 40ms
    Prepared 43 packages in 466ms
    Uninstalled 10 packages in 40ms
    Installed 43 packages in 51ms
     + annotated-types==0.7.0
     - certifi==2025.6.15
     + certifi==2025.8.3
     - charset-normalizer==3.4.2
     + charset-normalizer==3.4.3
     + contourpy==1.3.3
     + cycler==0.12.1
     + dacite==1.9.2
     - executing==2.2.0
     + executing==2.2.1
     + fonttools==4.60.0
     + htmlmin==0.1.12
     + imagehash==4.3.1
     + ipympl==0.9.7
     - ipython==9.4.0
     + ipython==9.5.0
     + ipywidgets==8.1.7
     + joblib==1.5.2
     + jupyterlab-widgets==3.0.15
     + kiwisolver==1.4.9
     + llvmlite==0.44.0
     + matplotlib==3.10.0
     + multimethod==1.12
     + networkx==3.5
     + numba==0.61.0
     - numpy==2.3.1
     + numpy==2.1.3
     - parso==0.8.4
     + parso==0.8.5
     + patsy==1.0.1
     + phik==0.12.5
     - prompt-toolkit==3.0.51
     + prompt-toolkit==3.0.52
     + puremagic==1.30
     + pydantic==2.11.9
     + pydantic-core==2.33.2
     + pyparsing==3.2.4
     + pywavelets==1.9.0
     - requests==2.32.4
     + requests==2.32.5
     - scipy==1.16.2
     + scipy==1.15.3
     + seaborn==0.13.2
     + statsmodels==0.14.5
     + tqdm==4.67.1
     + typeguard==4.4.4
     - typing-extensions==4.14.1
     + typing-extensions==4.15.0
     + typing-inspection==0.4.1
     + visions==0.8.1
     + widgetsnbextension==4.0.14
     + wordcloud==1.9.4
     + ydata-profiling==4.16.1
    Installed 43 packages in 51ms
     + annotated-types==0.7.0
     - certifi==2025.6.15
     + certifi==2025.8.3
     - charset-normalizer==3.4.2
     + charset-normalizer==3.4.3
     + contourpy==1.3.3
     + cycler==0.12.1
     + dacite==1.9.2
     - executing==2.2.0
     + executing==2.2.1
     + fonttools==4.60.0
     + htmlmin==0.1.12
     + imagehash==4.3.1
     + ipympl==0.9.7
     - ipython==9.4.0
     + ipython==9.5.0
     + ipywidgets==8.1.7
     + joblib==1.5.2
     + jupyterlab-widgets==3.0.15
     + kiwisolver==1.4.9
     + llvmlite==0.44.0
     + matplotlib==3.10.0
     + multimethod==1.12
     + networkx==3.5
     + numba==0.61.0
     - numpy==2.3.1
     + numpy==2.1.3
     - parso==0.8.4
     + parso==0.8.5
     + patsy==1.0.1
     + phik==0.12.5
     - prompt-toolkit==3.0.51
     + prompt-toolkit==3.0.52
     + puremagic==1.30
     + pydantic==2.11.9
     + pydantic-core==2.33.2
     + pyparsing==3.2.4
     + pywavelets==1.9.0
     - requests==2.32.4
     + requests==2.32.5
     - scipy==1.16.2
     + scipy==1.15.3
     + seaborn==0.13.2
     + statsmodels==0.14.5
     + tqdm==4.67.1
     + typeguard==4.4.4
     - typing-extensions==4.14.1
     + typing-extensions==4.15.0
     + typing-inspection==0.4.1
     + visions==0.8.1
     + widgetsnbextension==4.0.14
     + wordcloud==1.9.4
     + ydata-profiling==4.16.1

In [2]:

    %reset -f
    %load_ext autoreload
    %autoreload 2

In [3]:

    import filoma

    dfw = filoma.probe_to_df("../")
    dfw = dfw.filter(dfw["suffix"] == ".py")

    2025-09-20 21:42:29.318 | DEBUG    | filoma.directories.directory_profiler:__init__:352 - Interactive environment detected, disabling progress bars to avoid conflicts
    2025-09-20 21:42:29.318 | INFO     | filoma.directories.directory_profiler:probe:439 - Starting directory analysis of '../' using 🦀 Rust (Parallel) implementation
    2025-09-20 21:42:29.318 | INFO     | filoma.directories.directory_profiler:probe:439 - Starting directory analysis of '../' using 🦀 Rust (Parallel) implementation
    2025-09-20 21:42:30.518 | SUCCESS  | filoma.directories.directory_profiler:probe:455 - Directory analysis completed in 1.20s - Found 84,737 items (78,623 files, 6,114 folders) using 🦀 Rust (Parallel)
    2025-09-20 21:42:30.518 | SUCCESS  | filoma.directories.directory_profiler:probe:455 - Directory analysis completed in 1.20s - Found 84,737 items (78,623 files, 6,114 folders) using 🦀 Rust (Parallel)

In [5]:

    from ydata_profiling import ProfileReport

    profile = ProfileReport(dfw.to_pandas(), title="Filoma Report");

In [6]:

    profile

    Summarize dataset:   0%|          | 0/5 [00:00<?, ?it/s]

    100%|██████████| 18/18 [00:00<00:00, 4054.43it/s]

    Generate report structure:   0%|          | 0/1 [00:00<?, ?it/s]

    Render HTML:   0%|          | 0/1 [00:00<?, ?it/s]

Out[6]:

In [ ]: