# virt_s3

* [virt_s3 package](virt_s3.md)
  * [Module contents](virt_s3.md#module-virt_s3)
    * [`get_custom_logger()`](virt_s3.md#virt_s3get_custom_loggernamevirts3-log_level10)
    * [`PredictConnectionStoreParams`](virt_s3.md#class-virt_s3predictconnectionstoreparamsuser_id-store_name-store_owner)
    * [`S3Params`](virt_s3.md#class-virt_s3s3paramsbucket_name-endpoint_url-aws_access_key_idnone-aws_secret_access_keynone-region_namenone-profile_namenone-connection_store_paramsnone)
    * [`SessionManager`](virt_s3.md#class-virt_s3sessionmanagerparamsnone-clientnone-keep_openfalse-session_client_kwargs)
    * [`TransferConfig`](virt_s3.md#class-virt_s3transferconfigmultipart_threshold8388608-max_concurrency10-multipart_chunksize8388608-num_download_attempts5-max_io_queue100-io_chunksize262144-use_threadstrue-max_bandwidthnone-preferred_transfer_clientauto)
    * [`LocalFSParams`](virt_s3.md#class-virt_s3localfsparamsuse_local_fs-root_dir-user-bucket_name)
    * [`ImageFormatType`](virt_s3.md#class-virt_s3imageformattypevalue-namesnone--modulenone-qualnamenone-typenone-start1-boundarynone)
    * [`get_default_params()`](virt_s3.md#virt_s3get_default_paramsuse_localfalse-use_s3false)
    * [`get_session_client()`](virt_s3.md#virt_s3get_session_clientparamsnone-kwargs)
    * [`create_bucket()`](virt_s3.md#virt_s3create_bucketbucket_name-paramsnone-clientnone)
    * [`get_file_chunked()`](virt_s3.md#virt_s3get_file_chunkedfpath-paramsnone-clientnone-kwargs)
    * [`get_file()`](virt_s3.md#virt_s3get_filefpath-paramsnone-clientnone-bytes_iofalse-kwargs)
    * [`get_image()`](virt_s3.md#virt_s3get_imagefpath-paramsnone-clientnone-img_formatnone-bytes_iofalse-kwargs)
    * [`get_files_generator()`](virt_s3.md#virt_s3get_files_generatorfpath_li-paramsnone-clientnone-bytes_iofalse-kwargs)
    * [`get_files_batch()`](virt_s3.md#virt_s3get_files_batchfpath_li-paramsnone-clientnone-max_concurrency0-bytes_iofalse-kwargs)
    * [`list_dirs()`](virt_s3.md#virt_s3list_dirsdir_path-paramsnone-clientnone-delimiter-kwargs)
    * [`get_valid_file_paths()`](virt_s3.md#virt_s3get_valid_file_pathsdir_path-ignoreds_store-filter_extensions-paramsnone-clientnone-kwargs)
    * [`file_exists()`](virt_s3.md#virt_s3file_existsfpath-paramsnone-clientnone-kwargs)
    * [`upload_data()`](virt_s3.md#virt_s3upload_datadata-fpath-paramsnone-clientnone-kwargs)
    * [`delete_file()`](virt_s3.md#virt_s3delete_filefpath-paramsnone-clientnone-kwargs)
    * [`delete_files_by_dir()`](virt_s3.md#virt_s3delete_files_by_dirdir_path-paramsnone-clientnone-kwargs)
    * [`archive_zip_as_buffer()`](virt_s3.md#virt_s3archive_zip_as_bufferdata_bytes_dict)
    * [`extract_compressed_file()`](virt_s3.md#virt_s3extract_compressed_filefpath-extracted_dir_prefixnone-paramsnone-clientnone-kwargs)
    * [`read_parquet_file_df()`](virt_s3.md#virt_s3read_parquet_file_dffpath-paramsnone-clientnone-engineauto-columnsnone-filtersnone-use_nullable_dtypesfalse-download_kwargs)
    * [`write_parquet_file_df()`](virt_s3.md#virt_s3write_parquet_file_dfdf-save_fpath-paramsnone-clientnone-engineauto-compressionnone-indexnone-partition_colsnone-upload_kwargs)
  * [Submodules](virt_s3.md#submodules)
  * [virt_s3.s3 module](virt_s3.md#module-virt_s3.s3)
    * [`PredictConnectionStoreParams`](virt_s3.md#class-virt_s3s3predictconnectionstoreparamsuser_id-store_name-store_owner)
    * [`S3Params`](virt_s3.md#class-virt_s3s3s3paramsbucket_name-endpoint_url-aws_access_key_idnone-aws_secret_access_keynone-region_namenone-profile_namenone-connection_store_paramsnone)
    * [`S3SessionManager`](virt_s3.md#class-virt_s3s3s3sessionmanagers3_paramsnone-s3_clientnone-keep_openfalse-session_client_kwargs)
    * [`PerformanceMetrics`](virt_s3.md#class-virt_s3s3performancemetricsmbps00)
    * [`PerformanceRecord`](virt_s3.md#class-virt_s3s3performancerecord_type-metricssimple--transfer--count0-chosen_funcnone)
    * [`UploadDownloadPerformanceManager`](virt_s3.md#class-virt_s3s3uploaddownloadperformancemanager)
      * [`UploadDownloadPerformanceManager.reset()`](virt_s3.md#reset_type)
      * [`UploadDownloadPerformanceManager.average_metrics()`](virt_s3.md#average_metricsmetrics_li)
      * [`UploadDownloadPerformanceManager.get_metrics()`](virt_s3.md#get_metrics_type-func_str)
      * [`UploadDownloadPerformanceManager.choose_function()`](virt_s3.md#choose_function_type)
      * [`UploadDownloadPerformanceManager.update_performance_record()`](virt_s3.md#update_performance_record_type-bytes_transferred-transfer_time)
    * [`get_s3_default_params()`](virt_s3.md#virt_s3s3get_s3_default_params)
    * [`get_s3_session_client()`](virt_s3.md#virt_s3s3get_s3_session_clients3_params-kwargs)
    * [`create_s3_bucket()`](virt_s3.md#virt_s3s3create_s3_bucketbucket_name-s3_clientnone-s3_paramsnone-kwargs)
    * [`get_s3_file_chunked()`](virt_s3.md#virt_s3s3get_s3_file_chunkeds3_fpath-s3_clientnone-s3_paramsnone-chunk_size1048576-progress_callbacknone-bytes_iofalse-kwargs)
    * [`get_s3_file_simple()`](virt_s3.md#virt_s3s3get_s3_file_simples3_fpath-s3_clientnone-s3_paramsnone-bytes_iofalse-kwargs)
    * [`get_s3_file_transfer()`](virt_s3.md#virt_s3s3get_s3_file_transfers3_fpath-s3_clientnone-s3_paramsnone-progress_callbacknone-transfer_configboto3s3transfertransferconfig-object-bytes_iofalse-kwargs)
    * [`get_s3_file()`](virt_s3.md#virt_s3s3get_s3_files3_fpath-s3_clientnone-s3_paramsnone-progress_callbacknone-transfer_configboto3s3transfertransferconfig-object-bytes_iofalse-kwargs)
    * [`get_s3_image()`](virt_s3.md#virt_s3s3get_s3_images3_fpath-s3_clientnone-s3_paramsnone-img_formatnone-bytes_iofalse-kwargs)
    * [`get_s3_files_generator()`](virt_s3.md#virt_s3s3get_s3_files_generatorkey_list-s3_clientnone-s3_paramsnone-bytes_iofalse-kwargs)
    * [`get_s3_files_batch()`](virt_s3.md#virt_s3s3get_s3_files_batchkey_list-s3_clientnone-s3_paramsnone-max_concurrency0-bytes_iofalse-kwargs)
    * [`list_s3_folders()`](virt_s3.md#virt_s3s3list_s3_foldersprefix-s3_clientnone-s3_paramsnone-delimiter-kwargs)
    * [`get_valid_s3_keys()`](virt_s3.md#virt_s3s3get_valid_s3_keysprefix-ignoreds_store-filter_extensions-s3_clientnone-s3_paramsnone-kwargs)
    * [`key_exists_s3()`](virt_s3.md#virt_s3s3key_exists_s3key-s3_clientnone-s3_paramsnone-kwargs)
    * [`upload_to_s3_simple()`](virt_s3.md#virt_s3s3upload_to_s3_simpledata-save_key-s3_clientnone-s3_paramsnone-kwargs)
    * [`upload_to_s3_transfer()`](virt_s3.md#virt_s3s3upload_to_s3_transferdata-save_key-s3_clientnone-s3_paramsnone-progress_callbacknone-transfer_configboto3s3transfertransferconfig-object-kwargs)
    * [`upload_to_s3()`](virt_s3.md#virt_s3s3upload_to_s3data-save_key-s3_clientnone-s3_paramsnone-progress_callbacknone-transfer_configboto3s3transfertransferconfig-object-kwargs)
    * [`delete_s3_object()`](virt_s3.md#virt_s3s3upload_to_s3data-save_key-s3_clientnone-s3_paramsnone-progress_callbacknone-transfer_configboto3s3transfertransferconfig-object-kwargs)
    * [`delete_s3_objects_by_prefix()`](virt_s3.md#virt_s3s3delete_s3_objects_by_prefixprefix-s3_clientnone-s3_paramsnone-kwargs)
    * [`extract_compressed_file_s3()`](virt_s3.md#virt_s3s3extract_compressed_file_s3key-extracted_dir_prefixnone-s3_clientnone-s3_paramsnone-kwargs)
  * [virt_s3.fs module](virt_s3.md#module-virt_s3.fs)
    * [`LocalFSParams`](virt_s3.md#class-virt_s3fslocalfsparamsuse_local_fs-root_dir-user-bucket_name)
    * [`LocalFSSessionManager`](virt_s3.md#class-virt_s3fslocalfssessionmanagerlocal_fs_paramsnone-clientnone-keep_openfalse-session_client_kwargs)
    * [`get_local_fs_params()`](virt_s3.md#virt_s3fsget_local_fs_params)
    * [`fix_fs_path()`](virt_s3.md#virt_s3fsfix_fs_pathfpath-local_fs_params)
    * [`create_fs_bucket()`](virt_s3.md#virt_s3fscreate_fs_bucketbucket_path-local_fs_paramsnone-kwargs)
    * [`get_fs_file_chunked()`](virt_s3.md#virt_s3fsget_fs_file_chunkedfpath-local_fs_paramsnone-chunk_size1048576-bytes_iofalse-kwargs)
    * [`get_fs_file()`](virt_s3.md#virt_s3fsget_fs_filefpath-local_fs_paramsnone-bytes_iofalse-kwargs)
    * [`get_fs_image()`](virt_s3.md#virt_s3fsget_fs_imagefpath-img_formatnone-local_fs_paramsnone-bytes_iofalse-kwargs)
    * [`get_fs_files_generator()`](virt_s3.md#virt_s3fsget_fs_files_generatorfpath_list-local_fs_paramsnone-bytes_iofalse-kwargs)
    * [`get_fs_files_batch()`](virt_s3.md#virt_s3fsget_fs_files_batchfpath_list-local_fs_paramsnone-max_concurrency0-bytes_iofalse-kwargs)
    * [`list_fs_folders()`](virt_s3.md#virt_s3fslist_fs_foldersdir_path-local_fs_paramsnone-kwargs)
    * [`get_valid_fs_fpaths()`](virt_s3.md#virt_s3fsget_valid_fs_fpathsdir_path-ignoreds_store-filter_extensions-local_fs_paramsnone-kwargs)
    * [`fpath_exists_fs()`](virt_s3.md#virt_s3fsfpath_exists_fsfpath-local_fs_paramsnone-kwargs)
    * [`upload_to_fs()`](virt_s3.md#virt_s3fsupload_to_fsdata-save_path-local_fs_paramsnone-kwargs)
    * [`delete_fs_file()`](virt_s3.md#virt_s3fsdelete_fs_filefpath-local_fs_paramsnone-kwargs)
    * [`delete_fs_files_by_dir()`](virt_s3.md#virt_s3fsdelete_fs_files_by_dirdir_path-local_fs_paramsnone-kwargs)
    * [`extract_compressed_file_fs()`](virt_s3.md#virt_s3fsextract_compressed_file_fsfpath-extracted_dir_prefixnone-local_fs_paramsnone-kwargs)
  * [virt_s3.utils module](virt_s3.md#module-virt_s3.utils)
    * [`ImageFormatType`](virt_s3.md#class-virt_s3utilsimageformattypevalue-namesnone--modulenone-qualnamenone-typenone-start1-boundarynone)
    * [`get_custom_logger()`](virt_s3.md#virt_s3utilsget_custom_loggernamevirts3-log_level10)
    * [`archive_zip_as_buffer()`](virt_s3.md#virt_s3utilsarchive_zip_as_bufferdata_bytes_dict)
    * [`format_bytes()`](virt_s3.md#virt_s3.utils.format_bytes)
  * [virt_s3.extras module](virt_s3.md#module-virt_s3.extras)
    * [`read_parquet_file_df()`](virt_s3.md#virt_s3extrasread_parquet_file_dffpath-paramsnone-clientnone-engineauto-columnsnone-filtersnone-use_nullable_dtypesfalse-download_kwargs)
    * [`write_parquet_file_df()`](virt_s3.md#virt_s3extraswrite_parquet_file_dfdf-save_fpath-paramsnone-clientnone-engineauto-compressionnone-indexnone-partition_colsnone-upload_kwargs)
    * [`FileValidator`](virt_s3.md#class-virt_s3extrasfilevalidatorfile_extensionsjson-csv-parquetsz-params)
      * [`FileValidator.validate_file()`](virt_s3.md#abstract-validate_filefpath-args-paramsnone-clientnone-kwargs)
      * [`FileValidator.run()`](virt_s3.md#runfpath_li-root_pathnone-ignore)
    * [`JSONFileValidator`](virt_s3.md#class-virt_s3extrasjsonfilevalidatorfile_extensionsjson-params)
    * [`CSVFileValidator`](virt_s3.md#class-virt_s3extrascsvfilevalidatorfile_extensionscsv-params)
    * [`ParquetFileValidator`](virt_s3.md#class-virt_s3extrasparquetfilevalidatorfile_extensionsparquetsz-parquet-params)


    