{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"05_TidyAddr.ipynb","provenance":[],"collapsed_sections":[],"toc_visible":true,"authorship_tag":"ABX9TyNs65+Z0ugJaIBb08rau6F5"},"kernelspec":{"name":"python3","display_name":"Python 3"},"language_info":{"name":"python"}},"cells":[{"cell_type":"code","metadata":{"id":"5cst_coVgxhl"},"source":["# default_exp tidyaddr"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"ZpoH4bc0HkWZ"},"source":["## Notes on Security:\n","- Colabs runs in a free, secure and private vm hosted by google (a tiered-service)\n","- All data in the VM is lost when the VM Shuts down which happens a minute or so after the user leaves the colab"]},{"cell_type":"markdown","metadata":{"id":"6qJkVOCWOV9n"},"source":["## 1. Upload a csv file \n","- Execute next cell-block in order to run tidyAddr.  \n","- You will be prompted to upload a **CSV** file. It must contain only one column labeled 'address' exactly."]},{"cell_type":"code","metadata":{"id":"zu3XHyhJWQTX"},"source":["print(' ~~~~~~~~~~~ UPLOAD A FILE PLEASE ~~~~~~~~~~~')\n","import os\n","from google.colab import files\n","uploaded = files.upload()"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"6QyPxD2DAHQI"},"source":["file"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"nARZGhW4AnSY"},"source":["uploadedFilename = \"AFile_2020_addronly.csv\"\n","# d = list( uploaded.keys() )[0]"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"mJGEgQ8mOTGf"},"source":[""]},{"cell_type":"code","metadata":{"id":"HHzbcmOqI-ue"},"source":["#export\n","# Input: A CSV with a single column titled 'address' (may require pre-processing to get it like this)\n","# Output: A CSV with a single column containing text that can be split into 5 columns in xl. \n","# - Needs to be re-merged with any other columns removed before tidy addring.\n","import os\n","def runTidyAddr( filename, newfilename ):\n","  print( filename, newfilename )\n","  print(' ~~~~~~~~~~~ Getting TIDYADDR ~~~~~~~~~~~ ')\n","  ! rm tidyaddr-js -r \n","  ! git clone https://github.com/BNIA/tidyaddr-js.git\n","  print(' ~~~~~~~~~~~ Installing TIDYADDR ~~~~~~~~~~~') \n","  ! echo ORIGINAL_DIRECTORY && echo $(ls)\n","  ! cp *.csv tidyaddr-js/ && cd tidyaddr-js && npm install \n","  print(' ~~~~~~~~~~~ Running TIDYADDR ~~~~~~~~~~~')\n","  ! echo TIDYADDR_DIRECTORY && cd tidyaddr-js && echo $(ls)\n","  txt = \"cd tidyaddr-js && node tidyaddr.js clean-csv \" + filename + \" \" + newfilename\n","  os.system(txt)\n","\n","runTidyAddr( uploadedFilename, './'+uploadedFilename.replace(\".\", \"_tidyaddred.\") )"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"PkR1FwTzOdTv"},"source":["2. The output will be a single text column. It can be split by \";\" in excel to break it out into 5 or so different columns of data.\n","3. This output (once split into many columns) will need to be re-appended to the original dataset to have a final complete version. "]},{"cell_type":"code","metadata":{"id":"urnNY4oeagsW"},"source":["ls"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"HKQJaSsOQWAE"},"source":["## Debugging"]},{"cell_type":"markdown","metadata":{"id":"zZYQKXKzIcjg"},"source":["This following section is python code. Its not part of tidy addr but I use it to quickly inspect the dataframe in case there is any problem."]},{"cell_type":"code","metadata":{"id":"U0hBo-5Xx97q"},"source":["import pandas as pd"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"K7aKNnsiw61z"},"source":["pd.read_csv('bnia_snap_for_2019.csv')"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"17FttK6nxyfK"},"source":["df = pd.read_csv('bnia_tanf_2019_20208_by_zipcodes.csv')\n","df.head(1)"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"NbDitn8J_PNf"},"source":["ls"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"0jT8eF_d0QH1"},"source":[""],"execution_count":null,"outputs":[]}]}