{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![Private Eyes are Watching You](art/private-eyes.jpg)\n",
    "\n",
    "# Fetching Summary Data \n",
    "## Introducing the Ontology2 Edition of Dbpedia\n",
    "\n",
    "In our [last episode](http://ontology2.com/notebooks/local/DBpedia_Schema_Queries.html),  I did a number of queries against the DBpedia Ontology to map out the information available.  In that notebook,  I gave myself the restriction that I would only do queries against a copy of the DBpedia\n",
    "Ontology that is stored with the notebook.\n",
    "\n",
    "Because the Ontology contains roughly 740 types and 2700 properties (more than 250 for Person alone)\n",
    "this turned out to be a serious limitation -- unless we know how much information is available for these properties,  I can't know which ones are important,  and thus make a visualization that makes sense.\n",
    "\n",
    "Gastrodon is capable of querying the [DBpedia Public SPARQL endpoint](http://dbpedia.org/sparql), but the  DBpedia Endpoint has some limitations,  particularly,  it returns at most 10,000 results for a query.  Complex queries can also time out.  Certainly I could write a series of smaller queries to compute statistics,  but then I face a balancing act between too many small queries (which will take a long time to run) and queries that get too large (and sometimes time out.)\n",
    "\n",
    "Fortunately I have a product in the AWS Marketplace,  the [Ontology2 Edition of DBpedia 2016-04](https://aws.amazon.com/marketplace/pp/B01HMUNH4Q/) which is a private SPARQL endpoint already loaded with data from DBpedia.  By starting this product,  and waiting about an hour for it to initialize,  I can run as many SPARQL queries as I like of arbitrary complexity,  and shut it down when I'm through.\n",
    "\n",
    "In this notebook,  I use this private SPARQL endpoint to count the prevalence of types,  properties,  and datatypes.  I use SPARQL Construct to save this information into an RDF graph that I'll later be able to combine with the DBpedia Ontology RDF graph to better explore the schema.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I start with the usual preliminaries,  importing Python modules and prefix definitions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The autotime extension is already loaded. To reload it, use:\n",
      "  %reload_ext autotime\n",
      "time: 4.5 ms\n"
     ]
    }
   ],
   "source": [
    "%load_ext autotime\n",
    "import sys\n",
    "from os.path import expanduser\n",
    "from gastrodon import RemoteEndpoint,QName,ttl,URIRef,inline\n",
    "import pandas as pd\n",
    "import json\n",
    "pd.options.display.width=120\n",
    "pd.options.display.max_colwidth=100"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 8 ms\n"
     ]
    }
   ],
   "source": [
    "prefixes=inline(\"\"\"\n",
    "    @prefix dbo: <http://dbpedia.org/ontology/> .\n",
    "    @prefix summary: <http://rdf.ontology2.com/summary/> .\n",
    "\"\"\").graph"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It wouldn't be safe for me to check database connection information into Git,  so I store it in a file\n",
    "in my home directory named `~/.dbpedia/config.json`, which looks like\n",
    "\n",
    "```\n",
    "{\n",
    "    \"url\":\"http://130.21.14.234:8890/sparql-auth\",\n",
    "    \"user\":\"dba\",\n",
    "    \"passwd\":\"vKUcW1eSVkruDOtT\",\n",
    "    \"base_uri\":\"http://dbpedia.org/resource/\"\n",
    "}\n",
    "```\n",
    "\n",
    "(Note that that is not my real IP address and passwd.  If you want to reproduce this,  put in the IP address and password for your own server and save it to `~/.dbpedia/config.json`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 4 ms\n"
     ]
    }
   ],
   "source": [
    "connection_data=json.load(open(expanduser(\"~/.dbpedia/config.json\")))\n",
    "connection_data[\"prefixes\"]=prefixes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 12.5 ms\n"
     ]
    }
   ],
   "source": [
    "endpoint=RemoteEndpoint(**connection_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Counting Properties and Classes\n",
    "\n",
    "### Finding the right graphs\n",
    "\n",
    "The [Ontology2 Edition of DBpedia 2016-04](https://aws.amazon.com/marketplace/pp/B01HMUNH4Q/) is\n",
    "divided into a number of different named graphs,  one for each dataset described [here](http://wiki.dbpedia.org/downloads-2016-04#datasets).\n",
    "\n",
    "It's important to pay attention to this for two reasons.\n",
    "\n",
    "One of them is that facts can appear in the output of a SPARQL query more than once than if the query covers multiple graphs and if facts are repeated in those graphs.  This can throw off the accuracy of our counts.\n",
    "\n",
    "The other is that some queries seem to take a long time to run if they are run over all graphs;  particularly this affects queries that involve filtering over a prefix in the predicate field (ex.)\n",
    "\n",
    "```\n",
    "FILTER(STRSTARTS(STR(?p)),\"http://dbpedia.org/ontology/\")\n",
    "```\n",
    "\n",
    "Considering both of these factors,  it is wise to know which graphs the facts we want are stored in,  thus I start exploring:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cnt</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>g</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>http://downloads.dbpedia.org/2016-04/core-i18n/en/citedFacts_en.ttl.bz2</th>\n",
       "      <td>5502</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://downloads.dbpedia.org/2016-04/core-i18n/en/specific_mappingbased_properties_en.ttl.bz2</th>\n",
       "      <td>148105</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                                                                  cnt\n",
       "g                                                                                                    \n",
       "http://downloads.dbpedia.org/2016-04/core-i18n/en/citedFacts_en.ttl.bz2                          5502\n",
       "http://downloads.dbpedia.org/2016-04/core-i18n/en/specific_mappingbased_properties_en.ttl.bz2  148105"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 2.07 s\n"
     ]
    }
   ],
   "source": [
    "endpoint.select(\"\"\"\n",
    "    select ?g (COUNT(*) AS ?cnt) {\n",
    "       GRAPH ?g { ?a <http://dbpedia.org/ontology/Person/height> ?b } .\n",
    "    } GROUP BY ?g\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Thus I find one motherload of properties right away:  I save this in a variable so I can use it later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 999 µs\n"
     ]
    }
   ],
   "source": [
    "pgraph=URIRef(\"http://downloads.dbpedia.org/2016-04/core-i18n/en/specific_mappingbased_properties_en.ttl.bz2\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Looking up types,  I find a number of graphs and choose the transitive types:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cnt</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>g</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>http://downloads.dbpedia.org/2016-04/core-i18n/en/instance_types_transitive_en.ttl.bz2</th>\n",
       "      <td>1014819</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://downloads.dbpedia.org/2016-04/core-i18n/en/instance_types_en.ttl.bz2</th>\n",
       "      <td>502997</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://downloads.dbpedia.org/2016-04/core-i18n/en/instance_types_sdtyped_dbo_en.ttl.bz2</th>\n",
       "      <td>212295</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://downloads.dbpedia.org/2016-04/core-i18n/en/instance_types_lhd_dbo_en.ttl.bz2</th>\n",
       "      <td>834547</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                                                             cnt\n",
       "g                                                                                               \n",
       "http://downloads.dbpedia.org/2016-04/core-i18n/en/instance_types_transitive_en.ttl.bz2   1014819\n",
       "http://downloads.dbpedia.org/2016-04/core-i18n/en/instance_types_en.ttl.bz2               502997\n",
       "http://downloads.dbpedia.org/2016-04/core-i18n/en/instance_types_sdtyped_dbo_en.ttl.bz2   212295\n",
       "http://downloads.dbpedia.org/2016-04/core-i18n/en/instance_types_lhd_dbo_en.ttl.bz2       834547"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 332 ms\n"
     ]
    }
   ],
   "source": [
    "endpoint.select(\"\"\"\n",
    "    select ?g (COUNT(*) AS ?cnt) {\n",
    "       GRAPH ?g { ?a a dbo:Person } .\n",
    "    } GROUP BY ?g\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 1 ms\n"
     ]
    }
   ],
   "source": [
    "tgraph=URIRef(\"http://downloads.dbpedia.org/2016-04/core-i18n/en/instance_types_transitive_en.ttl.bz2\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Counting Classes\n",
    "\n",
    "It is now straightforward to pull up a list of types (classes),  noting that these are not mutually exclusive.  (You can be a `dbo:Actor` and a `dbo:Politician`)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cnt</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>type</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>dbo:WinterSportPlayer</th>\n",
       "      <td>22373</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:Project</th>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:PopulatedPlace</th>\n",
       "      <td>505557</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:Actor</th>\n",
       "      <td>1969</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:Document</th>\n",
       "      <td>23799</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:Genre</th>\n",
       "      <td>1229</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:Group</th>\n",
       "      <td>33681</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:Politician</th>\n",
       "      <td>19569</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:Station</th>\n",
       "      <td>2300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:Venue</th>\n",
       "      <td>789</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:Animal</th>\n",
       "      <td>219648</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:Comic</th>\n",
       "      <td>4097</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:GridironFootballPlayer</th>\n",
       "      <td>18908</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:MusicalArtist</th>\n",
       "      <td>480</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:RacingDriver</th>\n",
       "      <td>1765</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:Software</th>\n",
       "      <td>20419</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:Song</th>\n",
       "      <td>1155</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:EducationalInstitution</th>\n",
       "      <td>53057</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:MusicalWork</th>\n",
       "      <td>199355</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:NaturalEvent</th>\n",
       "      <td>1191</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:RaceTrack</th>\n",
       "      <td>242</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:Gene</th>\n",
       "      <td>90</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:Cartoon</th>\n",
       "      <td>6373</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:Cleric</th>\n",
       "      <td>12842</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:Database</th>\n",
       "      <td>358</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/%3Chttp://purl.org/dc/terms/Jurisdiction%3E</th>\n",
       "      <td>24409</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:Instrumentalist</th>\n",
       "      <td>151</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:LegalCase</th>\n",
       "      <td>2724</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:Name</th>\n",
       "      <td>4361</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:OrganisationMember</th>\n",
       "      <td>323111</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:Device</th>\n",
       "      <td>25748</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:Engine</th>\n",
       "      <td>18829</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:FictionalCharacter</th>\n",
       "      <td>7990</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:MotorcycleRider</th>\n",
       "      <td>701</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:Olympics</th>\n",
       "      <td>4032</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:Person</th>\n",
       "      <td>1014819</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:Plant</th>\n",
       "      <td>5062</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:Royalty</th>\n",
       "      <td>799</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:Settlement</th>\n",
       "      <td>230132</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:Species</th>\n",
       "      <td>295514</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:SportsEvent</th>\n",
       "      <td>23588</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:SportsTeam</th>\n",
       "      <td>30031</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:TimePeriod</th>\n",
       "      <td>922533</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:Wrestler</th>\n",
       "      <td>470</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:WrittenWork</th>\n",
       "      <td>62761</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:ArchitecturalStructure</th>\n",
       "      <td>188172</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:FootballLeagueSeason</th>\n",
       "      <td>3348</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:SportsTeamSeason</th>\n",
       "      <td>35360</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:GeneLocation</th>\n",
       "      <td>86</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:Athlete</th>\n",
       "      <td>298681</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:FloweringPlant</th>\n",
       "      <td>381</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:Stream</th>\n",
       "      <td>28289</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:ClericalAdministrativeRegion</th>\n",
       "      <td>3250</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:Coach</th>\n",
       "      <td>6985</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:Horse</th>\n",
       "      <td>3855</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:Location</th>\n",
       "      <td>816252</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:Region</th>\n",
       "      <td>24409</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:Satellite</th>\n",
       "      <td>2137</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:SportsManager</th>\n",
       "      <td>17654</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:Tower</th>\n",
       "      <td>1868</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>108 rows × 1 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                                             cnt\n",
       "type                                                                            \n",
       "dbo:WinterSportPlayer                                                      22373\n",
       "dbo:Project                                                                   11\n",
       "dbo:PopulatedPlace                                                        505557\n",
       "dbo:Actor                                                                   1969\n",
       "dbo:Document                                                               23799\n",
       "dbo:Genre                                                                   1229\n",
       "dbo:Group                                                                  33681\n",
       "dbo:Politician                                                             19569\n",
       "dbo:Station                                                                 2300\n",
       "dbo:Venue                                                                    789\n",
       "dbo:Animal                                                                219648\n",
       "dbo:Comic                                                                   4097\n",
       "dbo:GridironFootballPlayer                                                 18908\n",
       "dbo:MusicalArtist                                                            480\n",
       "dbo:RacingDriver                                                            1765\n",
       "dbo:Software                                                               20419\n",
       "dbo:Song                                                                    1155\n",
       "dbo:EducationalInstitution                                                 53057\n",
       "dbo:MusicalWork                                                           199355\n",
       "dbo:NaturalEvent                                                            1191\n",
       "dbo:RaceTrack                                                                242\n",
       "dbo:Gene                                                                      90\n",
       "dbo:Cartoon                                                                 6373\n",
       "dbo:Cleric                                                                 12842\n",
       "dbo:Database                                                                 358\n",
       "http://dbpedia.org/ontology/%3Chttp://purl.org/dc/terms/Jurisdiction%3E    24409\n",
       "dbo:Instrumentalist                                                          151\n",
       "dbo:LegalCase                                                               2724\n",
       "dbo:Name                                                                    4361\n",
       "dbo:OrganisationMember                                                    323111\n",
       "...                                                                          ...\n",
       "dbo:Device                                                                 25748\n",
       "dbo:Engine                                                                 18829\n",
       "dbo:FictionalCharacter                                                      7990\n",
       "dbo:MotorcycleRider                                                          701\n",
       "dbo:Olympics                                                                4032\n",
       "dbo:Person                                                               1014819\n",
       "dbo:Plant                                                                   5062\n",
       "dbo:Royalty                                                                  799\n",
       "dbo:Settlement                                                            230132\n",
       "dbo:Species                                                               295514\n",
       "dbo:SportsEvent                                                            23588\n",
       "dbo:SportsTeam                                                             30031\n",
       "dbo:TimePeriod                                                            922533\n",
       "dbo:Wrestler                                                                 470\n",
       "dbo:WrittenWork                                                            62761\n",
       "dbo:ArchitecturalStructure                                                188172\n",
       "dbo:FootballLeagueSeason                                                    3348\n",
       "dbo:SportsTeamSeason                                                       35360\n",
       "dbo:GeneLocation                                                              86\n",
       "dbo:Athlete                                                               298681\n",
       "dbo:FloweringPlant                                                           381\n",
       "dbo:Stream                                                                 28289\n",
       "dbo:ClericalAdministrativeRegion                                            3250\n",
       "dbo:Coach                                                                   6985\n",
       "dbo:Horse                                                                   3855\n",
       "dbo:Location                                                              816252\n",
       "dbo:Region                                                                 24409\n",
       "dbo:Satellite                                                               2137\n",
       "dbo:SportsManager                                                          17654\n",
       "dbo:Tower                                                                   1868\n",
       "\n",
       "[108 rows x 1 columns]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 1min 39s\n"
     ]
    }
   ],
   "source": [
    "endpoint.select(\"\"\"\n",
    "   SELECT ?type (COUNT(*) AS ?cnt) {\n",
    "       GRAPH ?_tgraph { ?s a ?type . }\n",
    "       FILTER(STRSTARTS(STR(?type),\"http://dbpedia.org/ontology/\"))\n",
    "    } GROUP BY ?type\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I can store these facts in an RDF graph (instead of a Pandas DataFrame) by using a `CONSTRUCT` query (instead of a `SELECT` query).  To capture the results of a `GROUP BY` query,  however,  I have to use a subquery -- this is because SPARQL requires that I only use variables in the `CONSTRUCT` clause,  thus I have to evaluate expressions (such as `COUNT(*)`) somewhere else.\n",
    "\n",
    "The resulting query is straightforward,  even if it looks a little awkward with all the braces:  roughly I cut and pasted the above SELECT query into a `CONSTRUCT` query that defines the facts that will be emitted."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 1min 40s\n"
     ]
    }
   ],
   "source": [
    "t_counts=endpoint.construct(\"\"\"\n",
    "   CONSTRUCT {\n",
    "      ?type summary:count ?cnt .\n",
    "   } WHERE { \n",
    "       {\n",
    "            SELECT ?type (COUNT(*) AS ?cnt) {\n",
    "                GRAPH ?_tgraph { ?s a ?type . }\n",
    "                FILTER(STRSTARTS(STR(?type),\"http://dbpedia.org/ontology/\"))\n",
    "            } GROUP BY ?type\n",
    "       } \n",
    "   }\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I can count the facts in this resulting graph (same as the number of rows in the `SELECT` query)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "108"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 3 ms\n"
     ]
    }
   ],
   "source": [
    "len(t_counts)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And here is a sample fact:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(rdflib.term.URIRef('http://dbpedia.org/ontology/Book'),\n",
       " rdflib.term.URIRef('http://rdf.ontology2.com/summary/count'),\n",
       " rdflib.term.Literal('22', datatype=rdflib.term.URIRef('http://www.w3.org/2001/XMLSchema#integer')))"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 4 ms\n"
     ]
    }
   ],
   "source": [
    "next(t_counts.__iter__())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that in the DBpedia Ontology there are a number of other facts about `dbo:Book`,  so if add the above fact to my copy of the DBpedia Ontology,  SPARQL queries will be able to pick up the count together with all the other facts."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Counting \"Specific Properties\"\n",
    "\n",
    "If I count properties in the \"specific mappingbased properties\" graph,  I find that these are\n",
    "all properties that have the Class name baked in"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cnt</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>p</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Canal/originalMaximumBoatLength</th>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Engine/length</th>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Engine/powerOutput</th>\n",
       "      <td>193</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Planet/averageSpeed</th>\n",
       "      <td>650</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Planet/density</th>\n",
       "      <td>123</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Infrastructure/length</th>\n",
       "      <td>23124</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Lake/volume</th>\n",
       "      <td>1401</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Person/weight</th>\n",
       "      <td>66144</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/PopulatedPlace/area</th>\n",
       "      <td>41718</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/PopulatedPlace/populationDensity</th>\n",
       "      <td>75629</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Software/fileSize</th>\n",
       "      <td>816</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/SpaceShuttle/distance</th>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/SpaceShuttle/timeInSpace</th>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Planet/apoapsis</th>\n",
       "      <td>3018</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Engine/displacement</th>\n",
       "      <td>335</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Canal/maximumBoatBeam</th>\n",
       "      <td>79</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/MeanOfTransportation/diameter</th>\n",
       "      <td>219</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Planet/volume</th>\n",
       "      <td>62</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/PopulatedPlace/populationUrbanDensity</th>\n",
       "      <td>162</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Stream/dischargeAverage</th>\n",
       "      <td>148</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Stream/maximumDischarge</th>\n",
       "      <td>975</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Engine/cylinderBore</th>\n",
       "      <td>400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Engine/height</th>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Engine/pistonStroke</th>\n",
       "      <td>372</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Engine/torqueOutput</th>\n",
       "      <td>67</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/GrandPrix/course</th>\n",
       "      <td>2635</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Planet/maximumTemperature</th>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/School/campusSize</th>\n",
       "      <td>1039</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Stream/discharge</th>\n",
       "      <td>2133</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Weapon/width</th>\n",
       "      <td>1416</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Weapon/weight</th>\n",
       "      <td>3387</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Planet/orbitalPeriod</th>\n",
       "      <td>3197</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Planet/surfaceArea</th>\n",
       "      <td>41</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/SpaceStation/volume</th>\n",
       "      <td>27</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Stream/minimumDischarge</th>\n",
       "      <td>915</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Automobile/fuelCapacity</th>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Canal/originalMaximumBoatBeam</th>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Engine/width</th>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Engine/weight</th>\n",
       "      <td>48</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/GrandPrix/distance</th>\n",
       "      <td>2595</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/PopulatedPlace/areaMetro</th>\n",
       "      <td>766</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Rocket/lowerEarthOrbitPayload</th>\n",
       "      <td>55</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Weapon/diameter</th>\n",
       "      <td>666</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Planet/meanTemperature</th>\n",
       "      <td>59</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Astronaut/timeInSpace</th>\n",
       "      <td>419</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Automobile/wheelbase</th>\n",
       "      <td>5331</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Lake/shoreLength</th>\n",
       "      <td>1592</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/MeanOfTransportation/weight</th>\n",
       "      <td>3173</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/MeanOfTransportation/width</th>\n",
       "      <td>5721</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Planet/meanRadius</th>\n",
       "      <td>45</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/PopulatedPlace/areaTotal</th>\n",
       "      <td>162857</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Rocket/mass</th>\n",
       "      <td>188</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Weapon/length</th>\n",
       "      <td>3415</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Work/runtime</th>\n",
       "      <td>261729</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Weapon/height</th>\n",
       "      <td>1412</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Canal/maximumBoatLength</th>\n",
       "      <td>77</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Planet/minimumTemperature</th>\n",
       "      <td>41</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/Planet/periapsis</th>\n",
       "      <td>3031</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/PopulatedPlace/areaUrban</th>\n",
       "      <td>655</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/ontology/PopulatedPlace/populationMetroDensity</th>\n",
       "      <td>246</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>69 rows × 1 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                                      cnt\n",
       "p                                                                        \n",
       "http://dbpedia.org/ontology/Canal/originalMaximumBoatLength             5\n",
       "http://dbpedia.org/ontology/Engine/length                              16\n",
       "http://dbpedia.org/ontology/Engine/powerOutput                        193\n",
       "http://dbpedia.org/ontology/Planet/averageSpeed                       650\n",
       "http://dbpedia.org/ontology/Planet/density                            123\n",
       "http://dbpedia.org/ontology/Infrastructure/length                   23124\n",
       "http://dbpedia.org/ontology/Lake/volume                              1401\n",
       "http://dbpedia.org/ontology/Person/weight                           66144\n",
       "http://dbpedia.org/ontology/PopulatedPlace/area                     41718\n",
       "http://dbpedia.org/ontology/PopulatedPlace/populationDensity        75629\n",
       "http://dbpedia.org/ontology/Software/fileSize                         816\n",
       "http://dbpedia.org/ontology/SpaceShuttle/distance                       5\n",
       "http://dbpedia.org/ontology/SpaceShuttle/timeInSpace                    7\n",
       "http://dbpedia.org/ontology/Planet/apoapsis                          3018\n",
       "http://dbpedia.org/ontology/Engine/displacement                       335\n",
       "http://dbpedia.org/ontology/Canal/maximumBoatBeam                      79\n",
       "http://dbpedia.org/ontology/MeanOfTransportation/diameter             219\n",
       "http://dbpedia.org/ontology/Planet/volume                              62\n",
       "http://dbpedia.org/ontology/PopulatedPlace/populationUrbanDensity     162\n",
       "http://dbpedia.org/ontology/Stream/dischargeAverage                   148\n",
       "http://dbpedia.org/ontology/Stream/maximumDischarge                   975\n",
       "http://dbpedia.org/ontology/Engine/cylinderBore                       400\n",
       "http://dbpedia.org/ontology/Engine/height                              12\n",
       "http://dbpedia.org/ontology/Engine/pistonStroke                       372\n",
       "http://dbpedia.org/ontology/Engine/torqueOutput                        67\n",
       "http://dbpedia.org/ontology/GrandPrix/course                         2635\n",
       "http://dbpedia.org/ontology/Planet/maximumTemperature                  45\n",
       "http://dbpedia.org/ontology/School/campusSize                        1039\n",
       "http://dbpedia.org/ontology/Stream/discharge                         2133\n",
       "http://dbpedia.org/ontology/Weapon/width                             1416\n",
       "...                                                                   ...\n",
       "http://dbpedia.org/ontology/Weapon/weight                            3387\n",
       "http://dbpedia.org/ontology/Planet/orbitalPeriod                     3197\n",
       "http://dbpedia.org/ontology/Planet/surfaceArea                         41\n",
       "http://dbpedia.org/ontology/SpaceStation/volume                        27\n",
       "http://dbpedia.org/ontology/Stream/minimumDischarge                   915\n",
       "http://dbpedia.org/ontology/Automobile/fuelCapacity                    11\n",
       "http://dbpedia.org/ontology/Canal/originalMaximumBoatBeam               7\n",
       "http://dbpedia.org/ontology/Engine/width                               12\n",
       "http://dbpedia.org/ontology/Engine/weight                              48\n",
       "http://dbpedia.org/ontology/GrandPrix/distance                       2595\n",
       "http://dbpedia.org/ontology/PopulatedPlace/areaMetro                  766\n",
       "http://dbpedia.org/ontology/Rocket/lowerEarthOrbitPayload              55\n",
       "http://dbpedia.org/ontology/Weapon/diameter                           666\n",
       "http://dbpedia.org/ontology/Planet/meanTemperature                     59\n",
       "http://dbpedia.org/ontology/Astronaut/timeInSpace                     419\n",
       "http://dbpedia.org/ontology/Automobile/wheelbase                     5331\n",
       "http://dbpedia.org/ontology/Lake/shoreLength                         1592\n",
       "http://dbpedia.org/ontology/MeanOfTransportation/weight              3173\n",
       "http://dbpedia.org/ontology/MeanOfTransportation/width               5721\n",
       "http://dbpedia.org/ontology/Planet/meanRadius                          45\n",
       "http://dbpedia.org/ontology/PopulatedPlace/areaTotal               162857\n",
       "http://dbpedia.org/ontology/Rocket/mass                               188\n",
       "http://dbpedia.org/ontology/Weapon/length                            3415\n",
       "http://dbpedia.org/ontology/Work/runtime                           261729\n",
       "http://dbpedia.org/ontology/Weapon/height                            1412\n",
       "http://dbpedia.org/ontology/Canal/maximumBoatLength                    77\n",
       "http://dbpedia.org/ontology/Planet/minimumTemperature                  41\n",
       "http://dbpedia.org/ontology/Planet/periapsis                         3031\n",
       "http://dbpedia.org/ontology/PopulatedPlace/areaUrban                  655\n",
       "http://dbpedia.org/ontology/PopulatedPlace/populationMetroDensity     246\n",
       "\n",
       "[69 rows x 1 columns]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 418 ms\n"
     ]
    }
   ],
   "source": [
    "endpoint.select(\"\"\"\n",
    "   SELECT ?p (COUNT(*) AS ?cnt) {\n",
    "       GRAPH ?_pgraph { ?s ?p ?o . }\n",
    "    } GROUP BY ?p\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 416 ms\n"
     ]
    }
   ],
   "source": [
    "sp_count=endpoint.construct(\"\"\"\n",
    "    CONSTRUCT {\n",
    "      ?p summary:count ?cnt .\n",
    "    } WHERE { {\n",
    "        SELECT ?p (COUNT(*) AS ?cnt) {\n",
    "           GRAPH ?_pgraph { ?s ?p ?o . }\n",
    "        } GROUP BY ?p\n",
    "   } }\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Other Ontology properties\n",
    "\n",
    "That begs the question of in which graphs other properties are stored.  Searching for `dbo:birthDate` I find the location of ordinary Literal properties.  (Which could be a date,  a number or a string)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cnt</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>g</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>http://downloads.dbpedia.org/2016-04/core-i18n/en/mappingbased_literals_en.ttl.bz2</th>\n",
       "      <td>819371</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://downloads.dbpedia.org/2016-04/core-i18n/en/persondata_en.ttl.bz2</th>\n",
       "      <td>730541</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://downloads.dbpedia.org/2016-04/core-i18n/en/citedFacts_en.ttl.bz2</th>\n",
       "      <td>6658</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                                                       cnt\n",
       "g                                                                                         \n",
       "http://downloads.dbpedia.org/2016-04/core-i18n/en/mappingbased_literals_en.ttl.bz2  819371\n",
       "http://downloads.dbpedia.org/2016-04/core-i18n/en/persondata_en.ttl.bz2             730541\n",
       "http://downloads.dbpedia.org/2016-04/core-i18n/en/citedFacts_en.ttl.bz2               6658"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 275 ms\n"
     ]
    }
   ],
   "source": [
    "endpoint.select(\"\"\"\n",
    "    select ?g (COUNT(*) AS ?cnt) {\n",
    "       GRAPH ?g { ?a dbo:birthDate ?b } .\n",
    "    } GROUP BY ?g\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A search for `dbo:child` turns up object properties (which point to a URI reference)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cnt</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>g</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>http://downloads.dbpedia.org/2016-04/core-i18n/en/mappingbased_objects_en.ttl.bz2</th>\n",
       "      <td>14456</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://downloads.dbpedia.org/2016-04/core-i18n/en/mappingbased_objects_disjoint_range_en.ttl.bz2</th>\n",
       "      <td>112</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://downloads.dbpedia.org/2016-04/core-i18n/en/citedFacts_en.ttl.bz2</th>\n",
       "      <td>91</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://downloads.dbpedia.org/2016-04/core-i18n/en/mappingbased_objects_uncleaned_en.ttl.bz2</th>\n",
       "      <td>14568</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                                                                    cnt\n",
       "g                                                                                                      \n",
       "http://downloads.dbpedia.org/2016-04/core-i18n/en/mappingbased_objects_en.ttl.bz2                 14456\n",
       "http://downloads.dbpedia.org/2016-04/core-i18n/en/mappingbased_objects_disjoint_range_en.ttl.bz2    112\n",
       "http://downloads.dbpedia.org/2016-04/core-i18n/en/citedFacts_en.ttl.bz2                              91\n",
       "http://downloads.dbpedia.org/2016-04/core-i18n/en/mappingbased_objects_uncleaned_en.ttl.bz2       14568"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 247 ms\n"
     ]
    }
   ],
   "source": [
    "endpoint.select(\"\"\"\n",
    "    select ?g (COUNT(*) AS ?cnt) {\n",
    "       GRAPH ?g { ?a dbo:child ?b } .\n",
    "    } GROUP BY ?g\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 1 ms\n"
     ]
    }
   ],
   "source": [
    "lgraph=URIRef(\"http://downloads.dbpedia.org/2016-04/core-i18n/en/mappingbased_literals_en.ttl.bz2\")\n",
    "ograph=URIRef(\"http://downloads.dbpedia.org/2016-04/core-i18n/en/mappingbased_objects_en.ttl.bz2\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Counting All Properties\n",
    "\n",
    "By taking a `UNION` I can count the \"specific\", object, and literal properties.  The DataFrame looks OK,  so I decide to save these counts into a graph."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cnt</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>p</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>dbo:film</th>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:headteacher</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:poleDriverCountry</th>\n",
       "      <td>84</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:numberOfClassrooms</th>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:conservationStatus</th>\n",
       "      <td>48437</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:dateOfBurial</th>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:established</th>\n",
       "      <td>2463</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:firstPublicationYear</th>\n",
       "      <td>6390</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:isniId</th>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:lastElectionDate</th>\n",
       "      <td>844</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:numberOfVineyards</th>\n",
       "      <td>60</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:plays</th>\n",
       "      <td>3720</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:testaverage</th>\n",
       "      <td>69</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:acquirementDate</th>\n",
       "      <td>5244</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:bedCount</th>\n",
       "      <td>1933</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:buildingStartYear</th>\n",
       "      <td>2251</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:ceeb</th>\n",
       "      <td>1152</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:centuryBreaks</th>\n",
       "      <td>356</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:chairmanTitle</th>\n",
       "      <td>5001</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:closingDate</th>\n",
       "      <td>1638</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:closingYear</th>\n",
       "      <td>3655</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:configuration</th>\n",
       "      <td>17234</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:diameter</th>\n",
       "      <td>1306</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:discharge</th>\n",
       "      <td>2141</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:electionDateLeader</th>\n",
       "      <td>542</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:elevationQuote</th>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:fees</th>\n",
       "      <td>159</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:firstAirDate</th>\n",
       "      <td>13743</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:firstGame</th>\n",
       "      <td>59</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:formerCallsign</th>\n",
       "      <td>19894</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:jstor</th>\n",
       "      <td>451</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:whaDraft</th>\n",
       "      <td>357</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:areaRural</th>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:productionEndDate</th>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:mgiid</th>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:classes</th>\n",
       "      <td>390</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:schoolNumber</th>\n",
       "      <td>580</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:startDate</th>\n",
       "      <td>9444</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:fansgroup</th>\n",
       "      <td>29</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:internationally</th>\n",
       "      <td>4303</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:lccn</th>\n",
       "      <td>3000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:nutsCode</th>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:rocketStages</th>\n",
       "      <td>192</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:routeTypeAbbreviation</th>\n",
       "      <td>17595</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:tournamentRecord</th>\n",
       "      <td>831</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:characterInPlay</th>\n",
       "      <td>5373</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:digitalSubChannel</th>\n",
       "      <td>4440</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:frequencyOfPublication</th>\n",
       "      <td>6082</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:recommissioningDate</th>\n",
       "      <td>925</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:successfulLaunches</th>\n",
       "      <td>258</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:originalMaximumBoatBeam</th>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:argueDate</th>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:lastFlightStartDate</th>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:volumeQuote</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:distanceToEdinburgh</th>\n",
       "      <td>140</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:issDockings</th>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:maximumDepthQuote</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:reservations</th>\n",
       "      <td>58</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:statisticValue</th>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dbo:throwingSide</th>\n",
       "      <td>655</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1438 rows × 1 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                               cnt\n",
       "p                                 \n",
       "dbo:film                        20\n",
       "dbo:headteacher                  1\n",
       "dbo:poleDriverCountry           84\n",
       "dbo:numberOfClassrooms           2\n",
       "dbo:conservationStatus       48437\n",
       "dbo:dateOfBurial                12\n",
       "dbo:established               2463\n",
       "dbo:firstPublicationYear      6390\n",
       "dbo:isniId                      50\n",
       "dbo:lastElectionDate           844\n",
       "dbo:numberOfVineyards           60\n",
       "dbo:plays                     3720\n",
       "dbo:testaverage                 69\n",
       "dbo:acquirementDate           5244\n",
       "dbo:bedCount                  1933\n",
       "dbo:buildingStartYear         2251\n",
       "dbo:ceeb                      1152\n",
       "dbo:centuryBreaks              356\n",
       "dbo:chairmanTitle             5001\n",
       "dbo:closingDate               1638\n",
       "dbo:closingYear               3655\n",
       "dbo:configuration            17234\n",
       "dbo:diameter                  1306\n",
       "dbo:discharge                 2141\n",
       "dbo:electionDateLeader         542\n",
       "dbo:elevationQuote               2\n",
       "dbo:fees                       159\n",
       "dbo:firstAirDate             13743\n",
       "dbo:firstGame                   59\n",
       "dbo:formerCallsign           19894\n",
       "...                            ...\n",
       "dbo:jstor                      451\n",
       "dbo:whaDraft                   357\n",
       "dbo:areaRural                   20\n",
       "dbo:productionEndDate           13\n",
       "dbo:mgiid                        9\n",
       "dbo:classes                    390\n",
       "dbo:schoolNumber               580\n",
       "dbo:startDate                 9444\n",
       "dbo:fansgroup                   29\n",
       "dbo:internationally           4303\n",
       "dbo:lccn                      3000\n",
       "dbo:nutsCode                    20\n",
       "dbo:rocketStages               192\n",
       "dbo:routeTypeAbbreviation    17595\n",
       "dbo:tournamentRecord           831\n",
       "dbo:characterInPlay           5373\n",
       "dbo:digitalSubChannel         4440\n",
       "dbo:frequencyOfPublication    6082\n",
       "dbo:recommissioningDate        925\n",
       "dbo:successfulLaunches         258\n",
       "dbo:originalMaximumBoatBeam      7\n",
       "dbo:argueDate                   13\n",
       "dbo:lastFlightStartDate          7\n",
       "dbo:volumeQuote                  1\n",
       "dbo:distanceToEdinburgh        140\n",
       "dbo:issDockings                  3\n",
       "dbo:maximumDepthQuote            1\n",
       "dbo:reservations                58\n",
       "dbo:statisticValue               8\n",
       "dbo:throwingSide               655\n",
       "\n",
       "[1438 rows x 1 columns]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 3.16 s\n"
     ]
    }
   ],
   "source": [
    "endpoint.select(\"\"\"\n",
    "   SELECT ?p (COUNT(*) AS ?cnt) {\n",
    "       { \n",
    "           GRAPH ?_pgraph { \n",
    "               ?s ?p ?o .      \n",
    "           }\n",
    "       } UNION { \n",
    "           GRAPH ?_ograph {\n",
    "               ?s ?p ?o .\n",
    "           }\n",
    "       } UNION {\n",
    "           GRAPH ?_lgraph {\n",
    "               ?s ?p ?o .\n",
    "           }\n",
    "       }\n",
    "    } GROUP BY ?p\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 3.09 s\n"
     ]
    }
   ],
   "source": [
    "p_counts=endpoint.construct(\"\"\"\n",
    "    CONSTRUCT {\n",
    "       ?p summary:count ?cnt .\n",
    "    } WHERE {\n",
    "        {\n",
    "            SELECT ?p (COUNT(*) AS ?cnt) {\n",
    "               { \n",
    "                   GRAPH ?_pgraph { \n",
    "                       ?s ?p ?o .      \n",
    "                   }\n",
    "               } UNION { \n",
    "                   GRAPH ?_ograph {\n",
    "                       ?s ?p ?o .\n",
    "                   }\n",
    "               } UNION {\n",
    "                   GRAPH ?_lgraph {\n",
    "                       ?s ?p ?o .\n",
    "                   }\n",
    "               }\n",
    "            } GROUP BY ?p\n",
    "        }\n",
    "    }\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1438"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 2.99 ms\n"
     ]
    }
   ],
   "source": [
    "len(p_counts)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Counting datatypes\n",
    "\n",
    "In a RDF,  a Class is a kind of type which represents a \"Thing\" in the world.  Datatypes,  on the other hand,  are types that represent literal values.  The most famous types in RDF come from the [XML Schema Datatypes](http://www.w3.org/TR/xmlschema-2/) and represent things such as integers,  dates,  and strings.\n",
    "\n",
    "RDF also allows us to define custom datatypes,  which are specified with URIs,  like most things in RDF.\n",
    "\n",
    "A `GROUP BY` query reveals the prevalence of various datatypes,  which I then dump to a graph.\n",
    "\n",
    "There still are some big questions to research such as \"does the same property turn up with different units?\"  For instance,  it is very possible that a length could be represented in kilometers,  centimeters,  feet,  or furlongs.  You won't get the right answer,  however,  if you try to add multiple lengths in different units that are all represented as floats.  Thus it may be necessary at some point to build a bridge to a package like [numericalunits](https://pypi.python.org/pypi/numericalunits) or alternately build something that canonicalizes them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cnt</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>datatype</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/kilometre</th>\n",
       "      <td>36045</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/kelvin</th>\n",
       "      <td>559</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/millimetre</th>\n",
       "      <td>59730</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/centimetre</th>\n",
       "      <td>148105</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/metre</th>\n",
       "      <td>387</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/litre</th>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/newtonMetre</th>\n",
       "      <td>67</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>xsd:nonNegativeInteger</th>\n",
       "      <td>824348</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>xsd:string</th>\n",
       "      <td>3571269</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/usDollar</th>\n",
       "      <td>56793</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/norwegianKrone</th>\n",
       "      <td>549</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/russianRouble</th>\n",
       "      <td>92</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/swissFranc</th>\n",
       "      <td>251</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/indianRupee</th>\n",
       "      <td>118</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>xsd:integer</th>\n",
       "      <td>1248777</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/tanzanianShilling</th>\n",
       "      <td>35</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/southKoreanWon</th>\n",
       "      <td>63</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/nicaraguanCórdoba</th>\n",
       "      <td>18</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/iranianRial</th>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/rwandaFranc</th>\n",
       "      <td>24</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/mauritianRupee</th>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/ukrainianHryvnia</th>\n",
       "      <td>28</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/renminbi</th>\n",
       "      <td>37</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/moldovanLeu</th>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/australianDollar</th>\n",
       "      <td>39</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/trinidadAndTobagoDollar</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/peruvianNuevoSol</th>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/gambianDalasi</th>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/bulgarianLev</th>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/maldivianRufiyaa</th>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/indonesianRupiah</th>\n",
       "      <td>26</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/qatariRial</th>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/thaiBaht</th>\n",
       "      <td>40</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/jordanianDinar</th>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/icelandKrona</th>\n",
       "      <td>35</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/lithuanianLitas</th>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/turkishLira</th>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/malawianKwacha</th>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/ghanaianCedi</th>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/hungarianForint</th>\n",
       "      <td>18</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/romanianNewLeu</th>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/bangladeshiTaka</th>\n",
       "      <td>25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/nepaleseRupee</th>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/myanmaKyat</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/sierraLeoneanLeone</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/brazilianReal</th>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/newZealandDollar</th>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/estonianKroon</th>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/latvianLats</th>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/bahrainiDinar</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/honduranLempira</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/chileanPeso</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/iraqiDinar</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/guineaFranc</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/newTaiwanDollar</th>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/papuaNewGuineanKina</th>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/israeliNewSheqel</th>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>xsd:anyURI</th>\n",
       "      <td>25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/fuelType</th>\n",
       "      <td>457</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>http://dbpedia.org/datatype/valvetrain</th>\n",
       "      <td>495</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>129 rows × 1 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                         cnt\n",
       "datatype                                                    \n",
       "http://dbpedia.org/datatype/kilometre                  36045\n",
       "http://dbpedia.org/datatype/kelvin                       559\n",
       "http://dbpedia.org/datatype/millimetre                 59730\n",
       "http://dbpedia.org/datatype/centimetre                148105\n",
       "http://dbpedia.org/datatype/metre                        387\n",
       "http://dbpedia.org/datatype/litre                         11\n",
       "http://dbpedia.org/datatype/newtonMetre                   67\n",
       "xsd:nonNegativeInteger                                824348\n",
       "xsd:string                                           3571269\n",
       "http://dbpedia.org/datatype/usDollar                   56793\n",
       "http://dbpedia.org/datatype/norwegianKrone               549\n",
       "http://dbpedia.org/datatype/russianRouble                 92\n",
       "http://dbpedia.org/datatype/swissFranc                   251\n",
       "http://dbpedia.org/datatype/indianRupee                  118\n",
       "xsd:integer                                          1248777\n",
       "http://dbpedia.org/datatype/tanzanianShilling             35\n",
       "http://dbpedia.org/datatype/southKoreanWon                63\n",
       "http://dbpedia.org/datatype/nicaraguanCórdoba             18\n",
       "http://dbpedia.org/datatype/iranianRial                    6\n",
       "http://dbpedia.org/datatype/rwandaFranc                   24\n",
       "http://dbpedia.org/datatype/mauritianRupee                 6\n",
       "http://dbpedia.org/datatype/ukrainianHryvnia              28\n",
       "http://dbpedia.org/datatype/renminbi                      37\n",
       "http://dbpedia.org/datatype/moldovanLeu                    6\n",
       "http://dbpedia.org/datatype/australianDollar              39\n",
       "http://dbpedia.org/datatype/trinidadAndTobagoDollar        1\n",
       "http://dbpedia.org/datatype/peruvianNuevoSol               5\n",
       "http://dbpedia.org/datatype/gambianDalasi                  2\n",
       "http://dbpedia.org/datatype/bulgarianLev                   5\n",
       "http://dbpedia.org/datatype/maldivianRufiyaa               3\n",
       "...                                                      ...\n",
       "http://dbpedia.org/datatype/indonesianRupiah              26\n",
       "http://dbpedia.org/datatype/qatariRial                     7\n",
       "http://dbpedia.org/datatype/thaiBaht                      40\n",
       "http://dbpedia.org/datatype/jordanianDinar                 8\n",
       "http://dbpedia.org/datatype/icelandKrona                  35\n",
       "http://dbpedia.org/datatype/lithuanianLitas               10\n",
       "http://dbpedia.org/datatype/turkishLira                    8\n",
       "http://dbpedia.org/datatype/malawianKwacha                 6\n",
       "http://dbpedia.org/datatype/ghanaianCedi                  11\n",
       "http://dbpedia.org/datatype/hungarianForint               18\n",
       "http://dbpedia.org/datatype/romanianNewLeu                15\n",
       "http://dbpedia.org/datatype/bangladeshiTaka               25\n",
       "http://dbpedia.org/datatype/nepaleseRupee                 12\n",
       "http://dbpedia.org/datatype/myanmaKyat                     1\n",
       "http://dbpedia.org/datatype/sierraLeoneanLeone             1\n",
       "http://dbpedia.org/datatype/brazilianReal                  3\n",
       "http://dbpedia.org/datatype/newZealandDollar              12\n",
       "http://dbpedia.org/datatype/estonianKroon                  2\n",
       "http://dbpedia.org/datatype/latvianLats                   11\n",
       "http://dbpedia.org/datatype/bahrainiDinar                  1\n",
       "http://dbpedia.org/datatype/honduranLempira                1\n",
       "http://dbpedia.org/datatype/chileanPeso                    1\n",
       "http://dbpedia.org/datatype/iraqiDinar                     1\n",
       "http://dbpedia.org/datatype/guineaFranc                    1\n",
       "http://dbpedia.org/datatype/newTaiwanDollar                2\n",
       "http://dbpedia.org/datatype/papuaNewGuineanKina            2\n",
       "http://dbpedia.org/datatype/israeliNewSheqel               2\n",
       "xsd:anyURI                                                25\n",
       "http://dbpedia.org/datatype/fuelType                     457\n",
       "http://dbpedia.org/datatype/valvetrain                   495\n",
       "\n",
       "[129 rows x 1 columns]"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 1min 36s\n"
     ]
    }
   ],
   "source": [
    "endpoint.select(\"\"\"\n",
    "   SELECT ?datatype (COUNT(*) AS ?cnt) {\n",
    "       { \n",
    "           GRAPH ?_pgraph { \n",
    "               ?s ?p ?o .      \n",
    "           }\n",
    "       } UNION {\n",
    "           GRAPH ?_lgraph {\n",
    "               ?s ?p ?o .\n",
    "           }\n",
    "       }\n",
    "       BIND(DATATYPE(?o) AS ?datatype)\n",
    "    } GROUP BY ?datatype\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 1min 35s\n"
     ]
    }
   ],
   "source": [
    "dt_counts=endpoint.construct(\"\"\"\n",
    "    CONSTRUCT {\n",
    "       ?datatype summary:count ?cnt .\n",
    "    } WHERE {\n",
    "       SELECT ?datatype (COUNT(*) AS ?cnt) {\n",
    "           { \n",
    "               GRAPH ?_pgraph { \n",
    "                   ?s ?p ?o .      \n",
    "               }\n",
    "           } UNION {\n",
    "               GRAPH ?_lgraph {\n",
    "                   ?s ?p ?o .\n",
    "               }\n",
    "           }\n",
    "           BIND(DATATYPE(?o) AS ?datatype)\n",
    "        } GROUP BY ?datatype\n",
    "    }\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Writing to disk\n",
    "\n",
    "RDFlib overloads the '+' operator so that we can easily merge the type,  property and datatype counts into one (modestly sized) graph."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 182 ms\n"
     ]
    }
   ],
   "source": [
    "all_counts = t_counts + p_counts + dt_counts"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I add a few prefix declarations for (human) readability,  then write the data to disk in Turtle format.  I was tempted to write it to a relative path which would put this file in its final destination.  (Underneath the `local` notebook directory,  where it could be found by notebooks) but decided against it,  since I don't want to take the chance of me (or you) trashing the project by mistake.  Instead I'll have to copy the file into place later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 507 ms\n"
     ]
    }
   ],
   "source": [
    "all_counts.bind(\"datatype\",\"http://dbpedia.org/datatype/\")\n",
    "all_counts.bind(\"dbo\",\"http://dbpedia.org/ontology/\")\n",
    "all_counts.bind(\"summary\",\"http://rdf.ontology2.com/summary/\")\n",
    "all_counts.serialize(\"/data/schema_counts.ttl\",format='ttl',encoding='utf-8')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Bonus File: Human Dimensions\n",
    "\n",
    "While I had my copy of DBpedia running,  I thought I'd gather a data set that would be worth making visualizations of.  Quite a lot of data exists in DBpedia concerning people's body dimensions,  so I decided to run a query and save the data for future use."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 51 s\n"
     ]
    }
   ],
   "source": [
    "dimensions=endpoint.select(\"\"\"\n",
    "select ?p ?height ?weight {\n",
    "    GRAPH ?_pgraph {\n",
    "        ?p <http://dbpedia.org/ontology/Person/weight> ?weight .\n",
    "        ?p <http://dbpedia.org/ontology/Person/height> ?height .\n",
    "    }\n",
    "}\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>p</th>\n",
       "      <th>height</th>\n",
       "      <th>weight</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>&lt;Alexander_Hug_(rugby_union)&gt;</td>\n",
       "      <td>188.00</td>\n",
       "      <td>91.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>&lt;Anderson_Silva&gt;</td>\n",
       "      <td>187.96</td>\n",
       "      <td>83.916000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>&lt;Andrew_Gee&gt;</td>\n",
       "      <td>183.00</td>\n",
       "      <td>102.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>&lt;Bernard_Ackah&gt;</td>\n",
       "      <td>185.42</td>\n",
       "      <td>90.720000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>&lt;Billy_Brandt&gt;</td>\n",
       "      <td>177.80</td>\n",
       "      <td>74.844000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>&lt;Bob_Beamon&gt;</td>\n",
       "      <td>191.00</td>\n",
       "      <td>70.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>&lt;Caleb_Moore&gt;</td>\n",
       "      <td>177.80</td>\n",
       "      <td>72.576000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>&lt;Charles_Hamelin&gt;</td>\n",
       "      <td>175.00</td>\n",
       "      <td>71.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>&lt;Charmaine_Sinclair&gt;</td>\n",
       "      <td>172.72</td>\n",
       "      <td>57.152639</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>&lt;Christina_Von_Eerie&gt;</td>\n",
       "      <td>162.56</td>\n",
       "      <td>55.792800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>&lt;Daniela_Hantuchová&gt;</td>\n",
       "      <td>181.00</td>\n",
       "      <td>62.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>&lt;Denice_Klarskov&gt;</td>\n",
       "      <td>170.18</td>\n",
       "      <td>50.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>&lt;Eamon_Sullivan&gt;</td>\n",
       "      <td>191.00</td>\n",
       "      <td>74.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>&lt;Folke_Jansson&gt;</td>\n",
       "      <td>187.00</td>\n",
       "      <td>80.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>&lt;Forrest_Towns&gt;</td>\n",
       "      <td>188.00</td>\n",
       "      <td>75.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>&lt;Franco_Columbu&gt;</td>\n",
       "      <td>164.00</td>\n",
       "      <td>88.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>&lt;Frank_Zane&gt;</td>\n",
       "      <td>175.26</td>\n",
       "      <td>83.916000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>&lt;Frederique_van_der_Wal&gt;</td>\n",
       "      <td>177.80</td>\n",
       "      <td>61.236000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>&lt;Félix_Sánchez&gt;</td>\n",
       "      <td>178.00</td>\n",
       "      <td>73.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>&lt;Georg_Lammers&gt;</td>\n",
       "      <td>178.00</td>\n",
       "      <td>84.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>&lt;Gloria_Leonard&gt;</td>\n",
       "      <td>172.72</td>\n",
       "      <td>70.761600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>&lt;Gory_Guerrero&gt;</td>\n",
       "      <td>175.00</td>\n",
       "      <td>95.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>&lt;Guillaume_LeBlanc&gt;</td>\n",
       "      <td>183.00</td>\n",
       "      <td>74.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>&lt;Habiba_Ghribi&gt;</td>\n",
       "      <td>174.00</td>\n",
       "      <td>49.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>&lt;Haile_Gebrselassie&gt;</td>\n",
       "      <td>165.00</td>\n",
       "      <td>56.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>&lt;Hans-Joachim_Reske&gt;</td>\n",
       "      <td>184.00</td>\n",
       "      <td>80.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>&lt;Harald_Andersson&gt;</td>\n",
       "      <td>191.00</td>\n",
       "      <td>99.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>&lt;Heinz-Joachim_Rothenburg&gt;</td>\n",
       "      <td>185.00</td>\n",
       "      <td>118.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>&lt;Ivan_Ivančić&gt;</td>\n",
       "      <td>193.04</td>\n",
       "      <td>127.915200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>&lt;Jan_Henne&gt;</td>\n",
       "      <td>152.40</td>\n",
       "      <td>63.957600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41633</th>\n",
       "      <td>&lt;Brett_McDermott__Brett_McDermott__1&gt;</td>\n",
       "      <td>180.00</td>\n",
       "      <td>93.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41634</th>\n",
       "      <td>&lt;Attila_Czene__Attila_Czene__1&gt;</td>\n",
       "      <td>185.00</td>\n",
       "      <td>76.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41635</th>\n",
       "      <td>&lt;Adam_Braidwood__Adam_Braidwood__1&gt;</td>\n",
       "      <td>193.04</td>\n",
       "      <td>122.472000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41636</th>\n",
       "      <td>&lt;Clarence_Childs__1&gt;</td>\n",
       "      <td>183.00</td>\n",
       "      <td>102.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41637</th>\n",
       "      <td>&lt;Eugena_Washington__Eugena_Washington__1&gt;</td>\n",
       "      <td>152.40</td>\n",
       "      <td>58.514400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41638</th>\n",
       "      <td>&lt;Niall_Breslin__Niall_Breslin__1&gt;</td>\n",
       "      <td>198.00</td>\n",
       "      <td>100.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41639</th>\n",
       "      <td>&lt;Matt_Ghaffari__Matt_Ghaffari__1&gt;</td>\n",
       "      <td>182.88</td>\n",
       "      <td>127.008000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41640</th>\n",
       "      <td>&lt;Paul_Kelly_(fighter)__Paul_Kelly__1&gt;</td>\n",
       "      <td>175.26</td>\n",
       "      <td>70.308000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41641</th>\n",
       "      <td>&lt;Tünde_Szabó__Tünde_Szabó__1&gt;</td>\n",
       "      <td>175.00</td>\n",
       "      <td>60.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41642</th>\n",
       "      <td>&lt;Muhammed_Lawal__Muhammed_Lawal__1&gt;</td>\n",
       "      <td>0.00</td>\n",
       "      <td>92.988000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41643</th>\n",
       "      <td>&lt;Rodney_Glunder__Rodney_Glunder__1&gt;</td>\n",
       "      <td>185.42</td>\n",
       "      <td>113.400000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41644</th>\n",
       "      <td>&lt;Garth_Wood__Garth_Wood__1&gt;</td>\n",
       "      <td>179.00</td>\n",
       "      <td>80.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41645</th>\n",
       "      <td>&lt;Achim_Albrecht__Achim_Albrecht__1&gt;</td>\n",
       "      <td>180.34</td>\n",
       "      <td>125.647200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41646</th>\n",
       "      <td>&lt;Anna_Bogomazova__Anna_Bogomazova__1&gt;</td>\n",
       "      <td>185.42</td>\n",
       "      <td>70.308000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41647</th>\n",
       "      <td>&lt;Elisabetta_Dessy__Elisabetta_Dessy__1&gt;</td>\n",
       "      <td>180.00</td>\n",
       "      <td>58.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41648</th>\n",
       "      <td>&lt;Paul_Schaus__Paul_Schaus__1&gt;</td>\n",
       "      <td>152.40</td>\n",
       "      <td>70.308000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41649</th>\n",
       "      <td>&lt;Arnold_Jackson_(British_Army_officer)__Arnold_Jackson_1912.jpg__1&gt;</td>\n",
       "      <td>176.00</td>\n",
       "      <td>67.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41650</th>\n",
       "      <td>&lt;Chris_Laidlaw__1&gt;</td>\n",
       "      <td>175.00</td>\n",
       "      <td>78.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41651</th>\n",
       "      <td>&lt;Chyna__Chyna__1&gt;</td>\n",
       "      <td>177.80</td>\n",
       "      <td>81.648000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41652</th>\n",
       "      <td>&lt;Ernie_Ladd__Ernie_Ladd__1&gt;</td>\n",
       "      <td>205.74</td>\n",
       "      <td>145.152000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41653</th>\n",
       "      <td>&lt;Herschel_Walker__Herschel_Walker__1&gt;</td>\n",
       "      <td>185.42</td>\n",
       "      <td>99.792000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41654</th>\n",
       "      <td>&lt;Ken_Shamrock__Ken_Shamrock__1&gt;</td>\n",
       "      <td>185.42</td>\n",
       "      <td>110.224800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41655</th>\n",
       "      <td>&lt;Ron_Clarke__2&gt;</td>\n",
       "      <td>183.00</td>\n",
       "      <td>72.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41656</th>\n",
       "      <td>&lt;Adam_Jones_(American_football)__Adam_Jones__1&gt;</td>\n",
       "      <td>0.00</td>\n",
       "      <td>83.916000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41657</th>\n",
       "      <td>&lt;Christi_Wolf__Christi_Wolf__1&gt;</td>\n",
       "      <td>160.02</td>\n",
       "      <td>68.040000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41658</th>\n",
       "      <td>&lt;Don_Frye__Don_Frye__1&gt;</td>\n",
       "      <td>185.00</td>\n",
       "      <td>110.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41659</th>\n",
       "      <td>&lt;Shaggy_2_Dope__Shaggy_2_Dope__1&gt;</td>\n",
       "      <td>187.96</td>\n",
       "      <td>104.328000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41660</th>\n",
       "      <td>&lt;Victoria_Zdrok__Victoria_Nika_Zdrok__1&gt;</td>\n",
       "      <td>175.26</td>\n",
       "      <td>54.432000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41661</th>\n",
       "      <td>&lt;Violent_J__Violent_J__1&gt;</td>\n",
       "      <td>190.50</td>\n",
       "      <td>127.008000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41662</th>\n",
       "      <td>&lt;Katsuyori_Shibata__Katsuyori_Shibata__1&gt;</td>\n",
       "      <td>183.00</td>\n",
       "      <td>103.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>41663 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                                         p  height      weight\n",
       "0                                            <Alexander_Hug_(rugby_union)>  188.00   91.000000\n",
       "1                                                         <Anderson_Silva>  187.96   83.916000\n",
       "2                                                             <Andrew_Gee>  183.00  102.000000\n",
       "3                                                          <Bernard_Ackah>  185.42   90.720000\n",
       "4                                                           <Billy_Brandt>  177.80   74.844000\n",
       "5                                                             <Bob_Beamon>  191.00   70.000000\n",
       "6                                                            <Caleb_Moore>  177.80   72.576000\n",
       "7                                                        <Charles_Hamelin>  175.00   71.000000\n",
       "8                                                     <Charmaine_Sinclair>  172.72   57.152639\n",
       "9                                                    <Christina_Von_Eerie>  162.56   55.792800\n",
       "10                                                    <Daniela_Hantuchová>  181.00   62.000000\n",
       "11                                                       <Denice_Klarskov>  170.18   50.000000\n",
       "12                                                        <Eamon_Sullivan>  191.00   74.000000\n",
       "13                                                         <Folke_Jansson>  187.00   80.000000\n",
       "14                                                         <Forrest_Towns>  188.00   75.000000\n",
       "15                                                        <Franco_Columbu>  164.00   88.000000\n",
       "16                                                            <Frank_Zane>  175.26   83.916000\n",
       "17                                                <Frederique_van_der_Wal>  177.80   61.236000\n",
       "18                                                         <Félix_Sánchez>  178.00   73.000000\n",
       "19                                                         <Georg_Lammers>  178.00   84.000000\n",
       "20                                                        <Gloria_Leonard>  172.72   70.761600\n",
       "21                                                         <Gory_Guerrero>  175.00   95.000000\n",
       "22                                                     <Guillaume_LeBlanc>  183.00   74.000000\n",
       "23                                                         <Habiba_Ghribi>  174.00   49.000000\n",
       "24                                                    <Haile_Gebrselassie>  165.00   56.000000\n",
       "25                                                    <Hans-Joachim_Reske>  184.00   80.000000\n",
       "26                                                      <Harald_Andersson>  191.00   99.000000\n",
       "27                                              <Heinz-Joachim_Rothenburg>  185.00  118.000000\n",
       "28                                                          <Ivan_Ivančić>  193.04  127.915200\n",
       "29                                                             <Jan_Henne>  152.40   63.957600\n",
       "...                                                                    ...     ...         ...\n",
       "41633                                <Brett_McDermott__Brett_McDermott__1>  180.00   93.000000\n",
       "41634                                      <Attila_Czene__Attila_Czene__1>  185.00   76.000000\n",
       "41635                                  <Adam_Braidwood__Adam_Braidwood__1>  193.04  122.472000\n",
       "41636                                                 <Clarence_Childs__1>  183.00  102.000000\n",
       "41637                            <Eugena_Washington__Eugena_Washington__1>  152.40   58.514400\n",
       "41638                                    <Niall_Breslin__Niall_Breslin__1>  198.00  100.000000\n",
       "41639                                    <Matt_Ghaffari__Matt_Ghaffari__1>  182.88  127.008000\n",
       "41640                                <Paul_Kelly_(fighter)__Paul_Kelly__1>  175.26   70.308000\n",
       "41641                                        <Tünde_Szabó__Tünde_Szabó__1>  175.00   60.000000\n",
       "41642                                  <Muhammed_Lawal__Muhammed_Lawal__1>    0.00   92.988000\n",
       "41643                                  <Rodney_Glunder__Rodney_Glunder__1>  185.42  113.400000\n",
       "41644                                          <Garth_Wood__Garth_Wood__1>  179.00   80.000000\n",
       "41645                                  <Achim_Albrecht__Achim_Albrecht__1>  180.34  125.647200\n",
       "41646                                <Anna_Bogomazova__Anna_Bogomazova__1>  185.42   70.308000\n",
       "41647                              <Elisabetta_Dessy__Elisabetta_Dessy__1>  180.00   58.000000\n",
       "41648                                        <Paul_Schaus__Paul_Schaus__1>  152.40   70.308000\n",
       "41649  <Arnold_Jackson_(British_Army_officer)__Arnold_Jackson_1912.jpg__1>  176.00   67.000000\n",
       "41650                                                   <Chris_Laidlaw__1>  175.00   78.000000\n",
       "41651                                                    <Chyna__Chyna__1>  177.80   81.648000\n",
       "41652                                          <Ernie_Ladd__Ernie_Ladd__1>  205.74  145.152000\n",
       "41653                                <Herschel_Walker__Herschel_Walker__1>  185.42   99.792000\n",
       "41654                                      <Ken_Shamrock__Ken_Shamrock__1>  185.42  110.224800\n",
       "41655                                                      <Ron_Clarke__2>  183.00   72.000000\n",
       "41656                      <Adam_Jones_(American_football)__Adam_Jones__1>    0.00   83.916000\n",
       "41657                                      <Christi_Wolf__Christi_Wolf__1>  160.02   68.040000\n",
       "41658                                              <Don_Frye__Don_Frye__1>  185.00  110.000000\n",
       "41659                                    <Shaggy_2_Dope__Shaggy_2_Dope__1>  187.96  104.328000\n",
       "41660                             <Victoria_Zdrok__Victoria_Nika_Zdrok__1>  175.26   54.432000\n",
       "41661                                            <Violent_J__Violent_J__1>  190.50  127.008000\n",
       "41662                            <Katsuyori_Shibata__Katsuyori_Shibata__1>  183.00  103.000000\n",
       "\n",
       "[41663 rows x 3 columns]"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 25 ms\n"
     ]
    }
   ],
   "source": [
    "dimensions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The data looks a bit messy.  Most noticeably,  I see quite a few facts which,  instead of pointing to DBpedia concepts,  point to synthetic URLs (such as `<Ron_Clarke__2>`) which are supposed to represent 'topics' such the time that a particular employee worked for a particular employer.  (See [this notebook for some discussion of the phenomenon](http://ontology2.com/notebooks/remote/Querying%20DBpedia.html)).\n",
    "\n",
    "Filtering these out will not be hard,  as these synthetic URLs all contain two consecutive underscores.\n",
    "\n",
    "I also think it's suspicious that a few people have a height of `0.0`,  which might be in the underlying data,  or might be because Gastrodon is not properly handling a missing data value.\n",
    "\n",
    "It would be certainly possible to serialize these results into an RDF graph,  but instead I write them into a CSV for simplicity."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time: 504 ms\n"
     ]
    }
   ],
   "source": [
    "dimensions.to_csv(\"/data/people_weight.csv.gz\",compression=\"gzip\",encoding=\"utf-8\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion\n",
    "\n",
    "To continue the analysis I began [here](http://ontology2.com/notebooks/local/DBpedia_Schema_Queries.html),  I needed a count of how often various classes,  properties,  and datatypes were used in DBpedia.  API limits could make getting this data from the public SPARQL endpoint challenging,  so I decided to run queries against my own\n",
    "private SPARQL endpoint powered by the [Ontology2 Edition of DBpedia](https://aws.amazon.com/marketplace/pp/B01HMUNH4Q/).\n",
    "\n",
    "After setting up connection information,  connecting to this private endpoint turned out to be as simple as connecting to a public endpoint and I was efficiently able to get the data I needed into an RDF graph,  ready to merge with the DBpedia Ontology graph to make a more meaningful analysis of the data in DBpedia towards the goal of producing interesting and attractive visualizations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
