{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The Science of Science\n",
    "by: Dashun Wang and Albert-Laszlo Barabasi\n",
    "\n",
    "[You can get the textbook here.](https://www.amazon.com/Science-Dashun-Wang/dp/1108716954/ref=asc_df_1108716954/?tag=hyprod-20&linkCode=df0&hvadid=459526655425&hvpos=&hvnetw=g&hvrand=10075848530578766295&hvpone=&hvptwo=&hvqmt=&hvdev=c&hvdvcmdl=&hvlocint=&hvlocphy=9002059&hvtargid=pla-967727027885&psc=1)\n",
    "\n",
    "The companion notebooks by Alex Gates."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To start, we need to download the data and preprocess it.  \n",
    "These steps only need to be run once, when you first download the data.  \n",
    "\n",
    "Note: We just learned Microsoft will discontinue their support for MAG as of Dec 2021.  As other data become available, we will update this code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# load the pyscisci package\n",
    "\n",
    "import pyscisci.all as pyscisci"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# you should download the MAG data from Microsoft's website:\n",
    "# https://www.microsoft.com/en-us/research/project/microsoft-academic-graph/\n",
    "\n",
    "# set this path to where the MAG database is locally stored\n",
    "path2mag = '/home/ajgates/MAG'\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a MAG object\n",
    "\n",
    "mymag = pyscisci.MAG(path2mag, keep_in_memory=False) \n",
    "# set keep_in_memory=False if you want to load the database each time its needed - good for when you \n",
    "# cant keep more than one database in memory at a time\n",
    "# otherwise keep_in_memory=True will keep each database in memory after its loaded"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# before we can start running our analysis, we have to preprocess the raw data into\n",
    "# DataFrames that are more convenient to work with\n",
    "\n",
    "# we only need to run this for the first time, but it will take awhile\n",
    "mymag.preprocess(verbose=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# MAG contains the following dataframes:\n",
    "\n",
    "# pub_df - keeps all of the publication information\n",
    "# columns : ['PublicationId', 'Year', 'JournalId', 'FamilyId',  'Doi', 'Title', 'Date', 'Volume', 'Issue', 'DocType']\n",
    "\n",
    "# author_df - keeps all of the author information\n",
    "# columns : ['AuthorId', 'FullName', 'LastName', 'FirstName', 'MiddleName']\n",
    "\n",
    "# pub2ref_df - links publications to their references or citations\n",
    "# columns : ['CitingPublicationId', 'CitedPublicationId']\n",
    "\n",
    "# paa_df - links publications, authors, and affiliations\n",
    "# columns : ['PublicationId', 'AuthorId', 'AffiliationId', 'AuthorSequence',  'OrigAuthorName', 'OrigAffiliationName']\n",
    "\n",
    "# author2pub_df - links the authors to their publications\n",
    "# columns : ['PublicationId', 'AuthorId', 'AuthorOrder']\n",
    "\n",
    "# field_df - field information\n",
    "# columns : ['FieldId', 'FieldLevel', 'NumberPublications', 'FieldName']\n",
    "\n",
    "# pub2field_df - links publications to their fields\n",
    "# columns : ['PublicationId', 'FieldId']\n",
    "\n",
    "# affiliation_df - affiliation information\n",
    "# columns : ['AffiliationId', 'NumberPublications', 'NumberCitations', 'FullName', 'GridId', 'OfficialPage', 'WikiPage', 'Latitude', 'Longitude']\n",
    "\n",
    "# journal_df - journal information\n",
    "# columns : ['JournalId', 'FullName', 'Issn', 'Publisher', 'Webpage']\n",
    "\n",
    "\n",
    "# after additional processing, these DataFrames become available\n",
    "\n",
    "# pub2refnoself_df - links publications to their references or citations with self-citations removed\n",
    "# columns : ['CitingPublicationId', 'CitedPublicationId']\n",
    "\n",
    "# impact_df - precomputed citation counts, columns will depend on which counts are computed\n",
    "# columns : ['PublicationId', 'Year', ....]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
