{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Import Agribalyse and link to ecoinvent 2.2\n",
    "\n",
    "[Agribalyse](http://www.ademe.fr/en/expertise/alternative-approaches-to-production/agribalyse-program) is a French LCI database of agricultural products. It builds on top of ecoinvent 2.2. It was exported from SimaPro, so the names of ecoinvent processes are mangled, and need to be fixed back to standard ecoinvent.\n",
    "\n",
    "This notebook uses Agribalyse 1.2, released in March 2015."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from brightway2 import *"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a new project for this notebook"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "projects.set_current(\"Agribalyse\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Add biosphere flows\n",
    "\n",
    "Biosphere flow names follow the standard in ecoinvent 3.3. We will need to match these names to those in Agribalyse."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Creating default biosphere\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Writing activities to SQLite3 database:\n",
      "0%                          100%\n",
      "[######                        ] | ETA: 00:00:00"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Applying strategy: normalize_units\n",
      "Applying strategy: drop_unspecified_subcategories\n",
      "Applied 2 strategies in 0.01 seconds\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[##############################] | ETA: 00:00:00\n",
      "Total time elapsed: 00:00:00\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Title: Writing activities to SQLite3 database:\n",
      "  Started: 11/30/2016 14:34:27\n",
      "  Finished: 11/30/2016 14:34:27\n",
      "  Total time elapsed: 00:00:00\n",
      "  CPU %: 100.10\n",
      "  Memory %: 0.93\n",
      "Created database: biosphere3\n",
      "Creating default LCIA methods\n",
      "\n",
      "Applying strategy: normalize_units\n",
      "Applying strategy: set_biosphere_type\n",
      "Applying strategy: drop_unspecified_subcategories\n",
      "Applying strategy: link_iterable_by_fields\n",
      "Applied 4 strategies in 2.16 seconds\n",
      "Wrote 718 LCIA methods with 178008 characterization factors\n",
      "Creating core data migrations\n",
      "\n"
     ]
    }
   ],
   "source": [
    "bw2setup()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import ecoinvent 2.2 as background database"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting XML data from 4087 datasets\n",
      "Extracted 4087 datasets in 3.68 seconds\n",
      "Applying strategy: normalize_units\n",
      "Applying strategy: assign_only_product_as_production\n",
      "Applying strategy: clean_integer_codes\n",
      "Applying strategy: drop_unspecified_subcategories\n",
      "Applying strategy: normalize_biosphere_categories\n",
      "Applying strategy: normalize_biosphere_names\n",
      "Applying strategy: strip_biosphere_exc_locations\n",
      "Applying strategy: set_code_by_activity_hash\n",
      "Applying strategy: link_iterable_by_fields\n",
      "Applying strategy: link_technosphere_by_activity_hash\n",
      "Applied 10 strategies in 4.95 seconds\n",
      "4087 datasets\n",
      "135892 exchanges\n",
      "0 unlinked exchanges\n",
      "  \n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(4087, 135892, 0)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "path = \"/Users/cmutel/Documents/LCA Documents/Ecoinvent/2.2/processes\"\n",
    "importer = SingleOutputEcospold1Importer(path, \"ecoinvent 2.2\")\n",
    "importer.apply_strategies()\n",
    "importer.statistics()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Writing activities to SQLite3 database:\n",
      "0%                          100%\n",
      "[##############################] | ETA: 00:00:00\n",
      "Total time elapsed: 00:00:19\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Title: Writing activities to SQLite3 database:\n",
      "  Started: 11/30/2016 14:36:13\n",
      "  Finished: 11/30/2016 14:36:33\n",
      "  Total time elapsed: 00:00:19\n",
      "  CPU %: 97.10\n",
      "  Memory %: 1.91\n",
      "Created database: ecoinvent 2.2\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Brightway2 SQLiteBackend: ecoinvent 2.2"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "importer.write_database()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load Agribalyse data\n",
    "\n",
    "This notebook uses the ecospold 1 version of Agribalyse, but the SimaPro CSV version should be quite similar, it would just use a different `Importer` class.\n",
    "\n",
    "We only need to give the directory, the `Importer` will find the XML file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting XML data from 1 datasets\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Warning: divide by zero encountered in log\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracted 822 datasets in 11.90 seconds\n",
      "Applying strategy: normalize_units\n",
      "Applying strategy: assign_only_product_as_production\n",
      "Applying strategy: clean_integer_codes\n",
      "Applying strategy: drop_unspecified_subcategories\n",
      "Applying strategy: normalize_biosphere_categories\n",
      "Applying strategy: normalize_biosphere_names\n",
      "Applying strategy: strip_biosphere_exc_locations\n",
      "Applying strategy: set_code_by_activity_hash\n",
      "Applying strategy: link_iterable_by_fields\n",
      "Applying strategy: link_technosphere_by_activity_hash\n",
      "Applied 10 strategies in 8.76 seconds\n",
      "822 datasets\n",
      "236600 exchanges\n",
      "129968 unlinked exchanges\n",
      "  Type biosphere: 1016 unique unlinked exchanges\n",
      "  Type production: 344 unique unlinked exchanges\n",
      "  Type technosphere: 728 unique unlinked exchanges\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(822, 236600, 129968)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "path = \"/Users/cmutel/Documents/LCA Documents/Agribalyse\"\n",
    "ag = SingleOutputEcospold1Importer(path, \"Agribalyse 1.2\")\n",
    "ag.apply_strategies()\n",
    "ag.statistics()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is quite a lot of linking problems. Let's export the unlinked exchanges to a spreadsheet so we can browse them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wrote matching file to:\n",
      "/Users/cmutel/Library/Application Support/Brightway3/Agribalyse.29703ab0819673042e6beb69e55073b0/output/db-matching-Agribalyse-12-unlinked.xlsx\n"
     ]
    }
   ],
   "source": [
    "ag.write_excel(True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Fix biosphere names\n",
    "\n",
    "One obvious problem is the names of biosphere flows changed from ecoinvent 2 to ecoinvent 3, **and** SimaPro uses another set of biosphere names and categories.\n",
    "\n",
    "Let's fix the SimaPro-specific problems first."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Applying strategy: normalize_simapro_biosphere_categories\n",
      "Applying strategy: normalize_simapro_biosphere_names\n"
     ]
    }
   ],
   "source": [
    "from bw2io.strategies.simapro import normalize_simapro_biosphere_categories, normalize_simapro_biosphere_names\n",
    "ag.apply_strategy(normalize_simapro_biosphere_categories)\n",
    "ag.apply_strategy(normalize_simapro_biosphere_names)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have modified the source data, but still need to try to link to the biosphere database.\n",
    "\n",
    "Read more about [currying functions](https://docs.python.org/3/library/functools.html#functools.partial) if this is new to you."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Applying strategy: link_iterable_by_fields\n",
      "822 datasets\n",
      "236600 exchanges\n",
      "17961 unlinked exchanges\n",
      "  Type biosphere: 316 unique unlinked exchanges\n",
      "  Type production: 344 unique unlinked exchanges\n",
      "  Type technosphere: 728 unique unlinked exchanges\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(822, 236600, 17961)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from bw2io.strategies import link_iterable_by_fields\n",
    "import functools\n",
    "ag.apply_strategy(functools.partial(link_iterable_by_fields, other=Database(\"biosphere3\"), kind=\"biosphere\"))\n",
    "ag.statistics()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That solved 70% of the biosphere flows, but there are still many unmatched flows. Again, we export the full list of unmatched exchanges."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Wrote matching file to:\n",
      "/Users/cmutel/Library/Application Support/Brightway3/Agribalyse.29703ab0819673042e6beb69e55073b0/output/db-matching-Agribalyse-12-unlinked.xlsx\n"
     ]
    }
   ],
   "source": [
    "ag.write_excel(True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The remaining unlinked biosphere flows *can't* be linked, because they don't exist in our biosphere database. This isn't the end of the world - we can add these new flows - but it does mean that they won't be assessed by our current LCIA methods.\n",
    "\n",
    "You can search the biosphere database to see what is in the current biosphere database using the search function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Nitrogen' (kilogram, None, ('water', 'ground-, long-term')),\n",
       " 'Nitrogen' (kilogram, None, ('soil',)),\n",
       " 'Nitrogen' (kilogram, None, ('soil', 'agricultural')),\n",
       " 'Nitrogen' (kilogram, None, ('water', 'ocean')),\n",
       " 'Nitrogen' (kilogram, None, ('air',)),\n",
       " 'Nitrogen' (kilogram, None, ('water',)),\n",
       " 'Nitrogen' (kilogram, None, ('water', 'ground-')),\n",
       " 'Nitrogen' (kilogram, None, ('soil', 'forestry')),\n",
       " 'Nitrogen' (kilogram, None, ('soil', 'industrial')),\n",
       " 'Nitrogen' (kilogram, None, ('water', 'surface water')),\n",
       " 'Nitrogen' (kilogram, None, ('natural resource', 'in air')),\n",
       " 'Nitrogen oxides' (kilogram, None, ('air', 'non-urban air or from high stacks')),\n",
       " 'Nitrogen fluoride' (kilogram, None, ('air', 'low population density, long-term')),\n",
       " 'Nitrogen oxides' (kilogram, None, ('air', 'lower stratosphere + upper troposphere')),\n",
       " 'Nitrogen fluoride' (kilogram, None, ('air', 'non-urban air or from high stacks')),\n",
       " 'Nitrogen fluoride' (kilogram, None, ('air', 'lower stratosphere + upper troposphere')),\n",
       " 'Nitrogen oxides' (kilogram, None, ('air', 'urban air close to ground')),\n",
       " 'Nitrogen fluoride' (kilogram, None, ('air', 'urban air close to ground')),\n",
       " 'Nitrogen oxides' (kilogram, None, ('air',)),\n",
       " 'Nitrogen oxides' (kilogram, None, ('air', 'low population density, long-term')),\n",
       " 'Nitrogen fluoride' (kilogram, None, ('air',)),\n",
       " 'Nitrogen, organic bound' (kilogram, None, ('water', 'surface water')),\n",
       " 'venting of nitrogen, liquid' (kilogram, None, ('social',)),\n",
       " 'Nitrogen, organic bound' (kilogram, None, ('water',)),\n",
       " 'Nitrogen, organic bound' (kilogram, None, ('water', 'ground-'))]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Database(\"biosphere3\").search(\"nitrogen\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We add these missing biosphere flows. We could add them to the default `biosphere3` database, but it is cleaner to create a new database with just the new flows added for Agribalyse."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Writing activities to SQLite3 database:\n",
      "0%                          100%\n",
      "[##############################] | ETA: 00:00:00\n",
      "Total time elapsed: 00:00:00\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Title: Writing activities to SQLite3 database:\n",
      "  Started: 11/30/2016 16:14:03\n",
      "  Finished: 11/30/2016 16:14:03\n",
      "  Total time elapsed: 00:00:00\n",
      "  CPU %: 98.10\n",
      "  Memory %: 2.15\n",
      "Applying strategy: link_iterable_by_fields\n"
     ]
    }
   ],
   "source": [
    "Database(\"Agribalyse new biosphere\").register()\n",
    "ag.add_unlinked_flows_to_biosphere_database(\"Agribalyse new biosphere\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We should now have no unlinked biosphere flows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "822 datasets\n",
      "236600 exchanges\n",
      "9062 unlinked exchanges\n",
      "  Type production: 344 unique unlinked exchanges\n",
      "  Type technosphere: 728 unique unlinked exchanges\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(822, 236600, 9062)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ag.statistics()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Fix production exchanges\n",
    "\n",
    "This is a weird one - production exchanges represent the flow produced by an activity, and should have the exact same name (because this is the standard in ecospold 1 - in ecospold 2 there is a difference between activity and product names). \n",
    "\n",
    "Let's look at the data for an unlinked production exchange and its activity. We are trying to figure out which field is different. We pick the first exchange in our spreadsheet."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "name\n",
      "\tActivity: name Alfalfa, conventional, for animal feeding, at farm gate\n",
      "\tProduct: name Alfalfa, conventional, for animal feeding, at farm gate\n",
      "unit\n",
      "\tActivity: unit kilogram\n",
      "\tProduct: unit kilogram\n",
      "location\n",
      "\tActivity: location FR\n",
      "\tProduct: location FR\n",
      "categories\n",
      "\tActivity: categories ['Agricultural', 'AGRIBALYSE\\\\Feeds\\\\Forage / green fodder']\n",
      "\tProduct: categories ('Agricultural', 'AGRIBALYSE\\\\tempLCI')\n"
     ]
    }
   ],
   "source": [
    "def get_unlinked(data):\n",
    "    for ds in ag.data:\n",
    "        for exc in ds['exchanges']:\n",
    "            if exc['type'] == 'production' and exc['name'] == 'Alfalfa, conventional, for animal feeding, at farm gate':\n",
    "                return ds, exc\n",
    "            \n",
    "ds, exc = get_unlinked(ag.data)\n",
    "            \n",
    "for field in ('name', 'unit', 'location', 'categories'):\n",
    "    print(field)\n",
    "    print(\"\\tActivity:\", field, ds.get(field))\n",
    "    print(\"\\tProduct:\", field, exc.get(field))    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this case, for whatever reason, the `categories` are different. The solution is to link without using the `categories` field. This strategy is smart - if excluding `categories` led to multiple possible links, it would raise an error instead of linking the (possibly) incorrect activity."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Applying strategy: link_technosphere_based_on_name_unit_location\n",
      "822 datasets\n",
      "236600 exchanges\n",
      "4553 unlinked exchanges\n",
      "  Type production: 16 unique unlinked exchanges\n",
      "  Type technosphere: 227 unique unlinked exchanges\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(822, 236600, 4553)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from bw2io.strategies import link_technosphere_based_on_name_unit_location\n",
    "ag.apply_strategy(link_technosphere_based_on_name_unit_location)\n",
    "ag.statistics()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Still a few problems. Let's look at one of them:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "name\n",
      "\tActivity: name Agricultural machinery with electronic motor, production\n",
      "\tProduct: name Disposal, plastics, mixture, 15.3% water, to municipal incineration\n",
      "unit\n",
      "\tActivity: unit kilogram\n",
      "\tProduct: unit kilogram\n",
      "location\n",
      "\tActivity: location GLO\n",
      "\tProduct: location CH\n",
      "categories\n",
      "\tActivity: categories ['Agricultural', 'AGRIBALYSE\\\\Others\\\\Machines']\n",
      "\tProduct: categories ()\n"
     ]
    }
   ],
   "source": [
    "def get_unlinked(data):\n",
    "    for ds in ag.data:\n",
    "        for exc in ds['exchanges']:\n",
    "            if exc['type'] == 'production' and not exc.get('input'):\n",
    "                return ds, exc\n",
    "            \n",
    "ds, exc = get_unlinked(ag.data)\n",
    "            \n",
    "for field in ('name', 'unit', 'location', 'categories'):\n",
    "    print(field)\n",
    "    print(\"\\tActivity:\", field, ds.get(field))\n",
    "    print(\"\\tProduct:\", field, exc.get(field))  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All the remaining outputs are disposal or recycling processes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Disposal, plastics, mixture, 15.3% water, to municipal incineration\n",
      "Disposal, used mineral oil, 10% water, to hazardous waste incineration\n",
      "Disposal, building, glass sheet, to final disposal\n",
      "Disposal, paper, 11.2% water, to municipal incineration\n",
      "Disposal, hard coal ash, 0% water, to residual material landfill\n",
      "Disposal, wood ash mixture, pure, 0% water, to municipal incineration\n",
      "Disposal, wood ash mixture, pure, 0% water, to landfarming\n",
      "Disposal, wood ash mixture, pure, 0% water, to sanitary landfill\n",
      "Disposal, steel, 0% water, to inert material landfill\n",
      "Disposal, electronics for control units\n",
      "Disposal, municipal solid waste, 22.9% water, to municipal incineration\n",
      "Recycling steel and iron\n",
      "Recycling aluminium\n",
      "Disposal, aluminium, 0% water, to sanitary landfill\n",
      "Recycling PVC\n",
      "Recycling mixed plastics\n"
     ]
    }
   ],
   "source": [
    "for exc in ag.unlinked:\n",
    "    if exc['type'] == \"production\":\n",
    "        print(exc['name'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The disposal processes are in ecoinvent, but the recycling processes aren't."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['disposal, plastics, mixture, 15.3% water, to sanitary landfill' (kilogram, CH, ['waste management', 'sanitary landfill']),\n",
       " 'disposal, plastics, mixture, 15.3% water, to municipal incineration' (kilogram, CH, ['waste management', 'municipal incineration'])]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Database(\"ecoinvent 2.2\").search(\"Disposal, plastics, mixture\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[]"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Database(\"ecoinvent 2.2\").search(\"recycling mixed plastics\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have to be a little careful here. SimaPro considers these exchanges *outputs*, but ecoinvent models disposal as in input (you consume the disposal service). The easiest way to handle this is to simply change these outputs into inputs, which will fix the sign.\n",
    "\n",
    "Note that we can't use `ag.unlinked`, as this only gives each unlinked exchange once, not every time it appears in the original data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fixing: Disposal, plastics, mixture, 15.3% water, to municipal incineration\n",
      "Fixing: Disposal, used mineral oil, 10% water, to hazardous waste incineration\n",
      "Fixing: Disposal, plastics, mixture, 15.3% water, to municipal incineration\n",
      "Fixing: Disposal, plastics, mixture, 15.3% water, to municipal incineration\n",
      "Fixing: Disposal, plastics, mixture, 15.3% water, to municipal incineration\n",
      "Fixing: Disposal, plastics, mixture, 15.3% water, to municipal incineration\n",
      "Fixing: Disposal, plastics, mixture, 15.3% water, to municipal incineration\n",
      "Fixing: Disposal, plastics, mixture, 15.3% water, to municipal incineration\n",
      "Fixing: Disposal, plastics, mixture, 15.3% water, to municipal incineration\n",
      "Fixing: Disposal, plastics, mixture, 15.3% water, to municipal incineration\n",
      "Fixing: Disposal, building, glass sheet, to final disposal\n",
      "Fixing: Disposal, building, glass sheet, to final disposal\n",
      "Fixing: Disposal, paper, 11.2% water, to municipal incineration\n",
      "Fixing: Disposal, plastics, mixture, 15.3% water, to municipal incineration\n",
      "Fixing: Disposal, plastics, mixture, 15.3% water, to municipal incineration\n",
      "Fixing: Disposal, plastics, mixture, 15.3% water, to municipal incineration\n",
      "Fixing: Disposal, used mineral oil, 10% water, to hazardous waste incineration\n",
      "Fixing: Disposal, used mineral oil, 10% water, to hazardous waste incineration\n",
      "Fixing: Disposal, building, glass sheet, to final disposal\n",
      "Fixing: Disposal, building, glass sheet, to final disposal\n",
      "Fixing: Disposal, paper, 11.2% water, to municipal incineration\n",
      "Fixing: Disposal, plastics, mixture, 15.3% water, to municipal incineration\n",
      "Fixing: Disposal, plastics, mixture, 15.3% water, to municipal incineration\n",
      "Fixing: Disposal, plastics, mixture, 15.3% water, to municipal incineration\n",
      "Fixing: Disposal, used mineral oil, 10% water, to hazardous waste incineration\n",
      "Fixing: Disposal, used mineral oil, 10% water, to hazardous waste incineration\n",
      "Fixing: Disposal, building, glass sheet, to final disposal\n",
      "Fixing: Disposal, building, glass sheet, to final disposal\n",
      "Fixing: Disposal, paper, 11.2% water, to municipal incineration\n",
      "Fixing: Disposal, plastics, mixture, 15.3% water, to municipal incineration\n",
      "Fixing: Disposal, plastics, mixture, 15.3% water, to municipal incineration\n",
      "Fixing: Disposal, plastics, mixture, 15.3% water, to municipal incineration\n",
      "Fixing: Disposal, used mineral oil, 10% water, to hazardous waste incineration\n",
      "Fixing: Disposal, used mineral oil, 10% water, to hazardous waste incineration\n",
      "Fixing: Disposal, plastics, mixture, 15.3% water, to municipal incineration\n",
      "Fixing: Disposal, plastics, mixture, 15.3% water, to municipal incineration\n",
      "Fixing: Disposal, building, glass sheet, to final disposal\n",
      "Fixing: Disposal, building, glass sheet, to final disposal\n",
      "Fixing: Disposal, paper, 11.2% water, to municipal incineration\n",
      "Fixing: Disposal, plastics, mixture, 15.3% water, to municipal incineration\n",
      "Fixing: Disposal, plastics, mixture, 15.3% water, to municipal incineration\n",
      "Fixing: Disposal, plastics, mixture, 15.3% water, to municipal incineration\n",
      "Fixing: Disposal, used mineral oil, 10% water, to hazardous waste incineration\n",
      "Fixing: Disposal, used mineral oil, 10% water, to hazardous waste incineration\n",
      "Fixing: Disposal, building, glass sheet, to final disposal\n",
      "Fixing: Disposal, building, glass sheet, to final disposal\n",
      "Fixing: Disposal, paper, 11.2% water, to municipal incineration\n",
      "Fixing: Disposal, plastics, mixture, 15.3% water, to municipal incineration\n",
      "Fixing: Disposal, plastics, mixture, 15.3% water, to municipal incineration\n",
      "Fixing: Disposal, plastics, mixture, 15.3% water, to municipal incineration\n",
      "Fixing: Disposal, used mineral oil, 10% water, to hazardous waste incineration\n",
      "Fixing: Disposal, used mineral oil, 10% water, to hazardous waste incineration\n",
      "Fixing: Disposal, building, glass sheet, to final disposal\n",
      "Fixing: Disposal, building, glass sheet, to final disposal\n",
      "Fixing: Disposal, paper, 11.2% water, to municipal incineration\n",
      "Fixing: Disposal, plastics, mixture, 15.3% water, to municipal incineration\n",
      "Fixing: Disposal, plastics, mixture, 15.3% water, to municipal incineration\n",
      "Fixing: Disposal, plastics, mixture, 15.3% water, to municipal incineration\n",
      "Fixing: Disposal, used mineral oil, 10% water, to hazardous waste incineration\n",
      "Fixing: Disposal, used mineral oil, 10% water, to hazardous waste incineration\n",
      "Fixing: Disposal, plastics, mixture, 15.3% water, to municipal incineration\n",
      "Fixing: Disposal, plastics, mixture, 15.3% water, to municipal incineration\n",
      "Fixing: Disposal, plastics, mixture, 15.3% water, to municipal incineration\n",
      "Fixing: Disposal, plastics, mixture, 15.3% water, to municipal incineration\n",
      "Fixing: Disposal, used mineral oil, 10% water, to hazardous waste incineration\n",
      "Fixing: Disposal, hard coal ash, 0% water, to residual material landfill\n",
      "Fixing: Disposal, wood ash mixture, pure, 0% water, to municipal incineration\n",
      "Fixing: Disposal, wood ash mixture, pure, 0% water, to landfarming\n",
      "Fixing: Disposal, wood ash mixture, pure, 0% water, to sanitary landfill\n",
      "Fixing: Disposal, steel, 0% water, to inert material landfill\n",
      "Fixing: Disposal, electronics for control units\n",
      "Fixing: Disposal, municipal solid waste, 22.9% water, to municipal incineration\n",
      "Fixing: Disposal, steel, 0% water, to inert material landfill\n",
      "Fixing: Recycling steel and iron\n",
      "Fixing: Recycling steel and iron\n",
      "Fixing: Recycling aluminium\n",
      "Fixing: Disposal, aluminium, 0% water, to sanitary landfill\n",
      "Fixing: Disposal, steel, 0% water, to inert material landfill\n",
      "Fixing: Disposal, electronics for control units\n",
      "Fixing: Disposal, municipal solid waste, 22.9% water, to municipal incineration\n",
      "Fixing: Disposal, steel, 0% water, to inert material landfill\n",
      "Fixing: Recycling steel and iron\n",
      "Fixing: Recycling steel and iron\n",
      "Fixing: Recycling aluminium\n",
      "Fixing: Disposal, aluminium, 0% water, to sanitary landfill\n",
      "Fixing: Disposal, steel, 0% water, to inert material landfill\n",
      "Fixing: Disposal, electronics for control units\n",
      "Fixing: Disposal, municipal solid waste, 22.9% water, to municipal incineration\n",
      "Fixing: Disposal, steel, 0% water, to inert material landfill\n",
      "Fixing: Disposal, municipal solid waste, 22.9% water, to municipal incineration\n",
      "Fixing: Disposal, steel, 0% water, to inert material landfill\n",
      "Fixing: Recycling steel and iron\n",
      "Fixing: Recycling steel and iron\n",
      "Fixing: Recycling aluminium\n",
      "Fixing: Disposal, aluminium, 0% water, to sanitary landfill\n",
      "Fixing: Recycling PVC\n",
      "Fixing: Recycling mixed plastics\n",
      "Fixing: Disposal, steel, 0% water, to inert material landfill\n",
      "Fixing: Disposal, electronics for control units\n",
      "Fixing: Disposal, municipal solid waste, 22.9% water, to municipal incineration\n",
      "Fixing: Disposal, steel, 0% water, to inert material landfill\n",
      "Fixing: Disposal, municipal solid waste, 22.9% water, to municipal incineration\n",
      "Fixing: Disposal, steel, 0% water, to inert material landfill\n",
      "Fixing: Recycling steel and iron\n",
      "Fixing: Recycling steel and iron\n",
      "Fixing: Recycling aluminium\n",
      "Fixing: Disposal, aluminium, 0% water, to sanitary landfill\n",
      "Fixing: Recycling PVC\n",
      "Fixing: Recycling mixed plastics\n",
      "Fixing: Disposal, steel, 0% water, to inert material landfill\n",
      "Fixing: Disposal, municipal solid waste, 22.9% water, to municipal incineration\n",
      "Fixing: Disposal, steel, 0% water, to inert material landfill\n",
      "Fixing: Disposal, municipal solid waste, 22.9% water, to municipal incineration\n",
      "Fixing: Disposal, steel, 0% water, to inert material landfill\n",
      "Fixing: Recycling steel and iron\n",
      "Fixing: Recycling steel and iron\n",
      "Fixing: Recycling PVC\n"
     ]
    }
   ],
   "source": [
    "for ds in ag.data:\n",
    "    for exc in ds['exchanges']:\n",
    "        if exc['type'] == 'production' and not exc.get('input'):\n",
    "            print(\"Fixing:\", exc['name'])\n",
    "            exc['type'] = 'technosphere'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will leave the recycling processes alone for now; first, we will fix all the ecoinvent links, including the disposal ones, and then we will get back to recycling.\n",
    "\n",
    "## 3. Fix technosphere inputs\n",
    "\n",
    "Looking at the spreadsheet, you notice that there is no `categories` field for any of the inputs. By default, `categories` is used when linking, so if ecoinvent 2.2 has the `categories` field (it does), then no suitable link will be found.\n",
    "\n",
    "This is a common problem with SimaPro, and we already know have a strategy to handle it already. We will try to fix both the internal links and the links to ecoinvent 2.2."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Applying strategy: link_technosphere_based_on_name_unit_location\n",
      "Applying strategy: link_technosphere_based_on_name_unit_location\n",
      "822 datasets\n",
      "236600 exchanges\n",
      "19 unlinked exchanges\n",
      "  Type technosphere: 4 unique unlinked exchanges\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(822, 236600, 19)"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ag.apply_strategy(link_technosphere_based_on_name_unit_location)\n",
    "ag.apply_strategy(functools.partial(link_technosphere_based_on_name_unit_location, external_db_name=\"ecoinvent 2.2\"))\n",
    "ag.statistics()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "So, that was relatively simple.\n",
    "\n",
    "## 4. Adding recycling processes\n",
    "\n",
    "The recycling processes don't exist, and don't have any impact, so the easiest way to handle these exchanges is to create new activities that produce the recycling flows. Luckily we have a method that does that for us. Note that the new recycling activities will be created in the Agribalyse database."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Applying strategy: link_iterable_by_fields\n",
      "826 datasets\n",
      "236600 exchanges\n",
      "0 unlinked exchanges\n",
      "  \n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(826, 236600, 0)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ag.add_unlinked_activities()\n",
    "ag.statistics()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Write the modified, fully linked database\n",
    "\n",
    "We are finished with the importing process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "ename": "NonuniqueCode",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mNonuniqueCode\u001b[0m                             Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-20-69eadd3895d9>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mag\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mwrite_database\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m/Users/cmutel/local35/bw3/lib/python3.5/site-packages/bw2io/importers/base_lci.py\u001b[0m in \u001b[0;36mwrite_database\u001b[0;34m(self, data, name, overwrite, backend, **kwargs)\u001b[0m\n\u001b[1;32m     90\u001b[0m             \u001b[0;32mraise\u001b[0m \u001b[0mWrongDatabase\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     91\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m{\u001b[0m\u001b[0mo\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'code'\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mo\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mdata\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m<\u001b[0m \u001b[0mlen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdata\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 92\u001b[0;31m             \u001b[0;32mraise\u001b[0m \u001b[0mNonuniqueCode\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     93\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     94\u001b[0m         \u001b[0mdata\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m{\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mds\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'database'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mds\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'code'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mds\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mds\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mdata\u001b[0m\u001b[0;34m}\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mNonuniqueCode\u001b[0m: "
     ]
    }
   ],
   "source": [
    "ag.write_database()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "OK, that is not good. The unique identifying codes for the activities come from the source data, which wouldn't be so foolish as to give non-unique identifiers to activities in the same export file, would it? Let's look at the codes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "265 826\n",
      "{'97', '270', '7', '26', '39', '77', '252', 'c0c1faef6c135ec10e6e8258eb047b5c', '241', '28', '1002', '227', '54', '83', '140', '114', '229', '111', '80', '137', '72', '42', '63', '124', '290', '237', '126', '10', '104', '278', '1', '110', '287', '232', '57', '73', '17', '228', '76', '277', '21', '92', '29', '58', '300', '286', '130', '45', '41', '116', '142', '2', '248', '292', '55', '314', '1000', '113', '38', '60', '81', '108', '312', '4', '154', '31', '145', '135', '236', '11', '256', '303', '318', '320', '50', '289', '261', '133', '155', '306', '293', '250', '79', '74', '106', '136', '102', '64', '6d606e201ebeddccf36d8a07e7c8bb76', '152', '294', '52', '132', '307', '321', '119', '147', '262', '283', '22', '225', '267', '141', '44', '85', '149', '16', '84', '309', '243', '263', '1003', '275', '129', '254', '9', '66', '40', '134', '308', '51', '271', '33', '78dbcef90e31a55f44192d3a14e6a0b0', '105', '264', '282', '89', '235', '281', '68', '246', '13', '62', '27', '32', '56', '103', '139', '3', '273', '284', '304', '48', '86', '274', '131', '71', '107', '310', '299', '43', '150', '35', '156', '151', '233', '96', '93', '239', '242', '249', '88', '1005', '34', '138', '143', '78', '280', '153', '128', '146', '260', '118', '1001', '115', '67', '36', '90', '65', '59', '127', '37', '5', '157', '296', '317', '53', '234', '288', '15', '98', '99', '144', '302', '20', '61', '23', '224', '109', '244', '269', '295', '315', '94', '82', '6', '123', '313', '268', '230', '69', '251', '297', '305', '46', '19', '101', '125', '265', '285', '12', '1004', '257', '14', '18', '316', '298', '100', '25', '95', '279', '117', '247', '291', '120', '259', '91', '148', '87', '276', '253', '255', '238', '47', '272', 'e80ed63a3b830fd7fea9b1664e984bb8', '311', '301', '122', '258', '70', '49', '112', '240', '24', '30', '231', '8', '121', '75', '245', '266', '226', '319'}\n"
     ]
    }
   ],
   "source": [
    "print(len({ds['code'] for ds in ag.data}), len(ag.data))\n",
    "print({ds['code'] for ds in ag.data})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That is not good. 826 activities, and only 265 unique codes. Let's look at the source data:\n",
    "\n",
    "    <dataset number=\"28\" timestamp=\"2015-02-22T17:27:17\" generator=\"SimaPro 8.0.3.14\">\n",
    "    <referenceFunction name=\"Bovine feed,MAT18, at farm gate\">\n",
    "\n",
    "    <dataset number=\"28\" timestamp=\"2014-12-21T14:10:26\" generator=\"SimaPro 8.0.3.14\">\n",
    "    <referenceFunction name=\"Greenhouse, glass walls and roof, plastic tubes\">\n",
    "\n",
    "    <dataset number=\"28\" timestamp=\"2013-09-18T16:53:22\" generator=\"CDT V1.2\">\n",
    "    <referenceFunction name=\"Harrowing, with rotary harrow (standard equipment)\">\n",
    "\n",
    "\n",
    "We need to add unique codes. We have a strategy for this, `set_code_by_activity_hash`, but it won't overwrite codes already present. We can fix that :)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "for ds in ag.data:\n",
    "    del ds['code']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Applying strategy: set_code_by_activity_hash\n"
     ]
    }
   ],
   "source": [
    "from bw2io.strategies import set_code_by_activity_hash\n",
    "ag.apply_strategy(set_code_by_activity_hash)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Only the internal links will need to be redone - the links to ecoinvent 2.2 and the biosphere database are fine.\n",
    "\n",
    "We can't use `link_technosphere_based_on_name_unit_location`, because we need to pass the parameter `relink`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Applying strategy: link_iterable_by_fields\n",
      "826 datasets\n",
      "236600 exchanges\n",
      "0 unlinked exchanges\n",
      "  \n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(826, 236600, 0)"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ag.apply_strategy(functools.partial(\n",
    "        link_iterable_by_fields,\n",
    "        other=ag.data,\n",
    "        fields=('name', 'location', 'unit'),\n",
    "        relink=True\n",
    "))\n",
    "ag.statistics()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Actually writing the final database\n",
    "\n",
    "We are now ready to try again."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Writing activities to SQLite3 database:\n",
      "0%                          100%\n",
      "[##############################] | ETA: 00:00:00\n",
      "Total time elapsed: 00:00:28\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Title: Writing activities to SQLite3 database:\n",
      "  Started: 11/30/2016 16:15:54\n",
      "  Finished: 11/30/2016 16:16:23\n",
      "  Total time elapsed: 00:00:28\n",
      "  CPU %: 96.60\n",
      "  Memory %: 2.24\n",
      "Created database: Agribalyse 1.2\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Brightway2 SQLiteBackend: Agribalyse 1.2"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ag.write_database()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Checking the imported datasets\n",
    "\n",
    "We need to do some basic validation to make sure we have meaningful results. Here I just do some basic testing, but you should validate against known scores if you are frequently using this database. The following code is rahter simple and is not a real validation check."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('IPCC 2013', 'climate change', 'GWP 20a')"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gwp = [x for x in methods if \"IPCC 2013\" in str(x)][0]\n",
    "gwp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "13.329517917350614"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "db = Database(\"Agribalyse 1.2\")\n",
    "\n",
    "lca = LCA({db.random(): 1}, gwp)\n",
    "lca.lci(factorize=True)\n",
    "lca.lcia()\n",
    "lca.score"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's calculate the LCIA scores of all activities in Agribalyse"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "0%                          100%\n",
      "[##############################] | ETA: 00:00:00\n",
      "Total time elapsed: 00:00:08\n"
     ]
    }
   ],
   "source": [
    "import pyprind\n",
    "\n",
    "scores = []\n",
    "\n",
    "for act in pyprind.prog_bar(db):\n",
    "    lca.redo_lcia({act: 1})\n",
    "    scores.append(lca.score)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "27 826\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "scores = np.array(scores)\n",
    "mask = scores == 0\n",
    "print(mask.sum(), len(db))\n",
    "\n",
    "scores = scores[~mask]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib notebook"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Warning: axes.color_cycle is deprecated and replaced with axes.prop_cycle; please use the latter.\n"
     ]
    }
   ],
   "source": [
    "import seaborn as sns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "/* Put everything inside the global mpl namespace */\n",
       "window.mpl = {};\n",
       "\n",
       "mpl.get_websocket_type = function() {\n",
       "    if (typeof(WebSocket) !== 'undefined') {\n",
       "        return WebSocket;\n",
       "    } else if (typeof(MozWebSocket) !== 'undefined') {\n",
       "        return MozWebSocket;\n",
       "    } else {\n",
       "        alert('Your browser does not have WebSocket support.' +\n",
       "              'Please try Chrome, Safari or Firefox ≥ 6. ' +\n",
       "              'Firefox 4 and 5 are also supported but you ' +\n",
       "              'have to enable WebSockets in about:config.');\n",
       "    };\n",
       "}\n",
       "\n",
       "mpl.figure = function(figure_id, websocket, ondownload, parent_element) {\n",
       "    this.id = figure_id;\n",
       "\n",
       "    this.ws = websocket;\n",
       "\n",
       "    this.supports_binary = (this.ws.binaryType != undefined);\n",
       "\n",
       "    if (!this.supports_binary) {\n",
       "        var warnings = document.getElementById(\"mpl-warnings\");\n",
       "        if (warnings) {\n",
       "            warnings.style.display = 'block';\n",
       "            warnings.textContent = (\n",
       "                \"This browser does not support binary websocket messages. \" +\n",
       "                    \"Performance may be slow.\");\n",
       "        }\n",
       "    }\n",
       "\n",
       "    this.imageObj = new Image();\n",
       "\n",
       "    this.context = undefined;\n",
       "    this.message = undefined;\n",
       "    this.canvas = undefined;\n",
       "    this.rubberband_canvas = undefined;\n",
       "    this.rubberband_context = undefined;\n",
       "    this.format_dropdown = undefined;\n",
       "\n",
       "    this.image_mode = 'full';\n",
       "\n",
       "    this.root = $('<div/>');\n",
       "    this._root_extra_style(this.root)\n",
       "    this.root.attr('style', 'display: inline-block');\n",
       "\n",
       "    $(parent_element).append(this.root);\n",
       "\n",
       "    this._init_header(this);\n",
       "    this._init_canvas(this);\n",
       "    this._init_toolbar(this);\n",
       "\n",
       "    var fig = this;\n",
       "\n",
       "    this.waiting = false;\n",
       "\n",
       "    this.ws.onopen =  function () {\n",
       "            fig.send_message(\"supports_binary\", {value: fig.supports_binary});\n",
       "            fig.send_message(\"send_image_mode\", {});\n",
       "            fig.send_message(\"refresh\", {});\n",
       "        }\n",
       "\n",
       "    this.imageObj.onload = function() {\n",
       "            if (fig.image_mode == 'full') {\n",
       "                // Full images could contain transparency (where diff images\n",
       "                // almost always do), so we need to clear the canvas so that\n",
       "                // there is no ghosting.\n",
       "                fig.context.clearRect(0, 0, fig.canvas.width, fig.canvas.height);\n",
       "            }\n",
       "            fig.context.drawImage(fig.imageObj, 0, 0);\n",
       "        };\n",
       "\n",
       "    this.imageObj.onunload = function() {\n",
       "        this.ws.close();\n",
       "    }\n",
       "\n",
       "    this.ws.onmessage = this._make_on_message_function(this);\n",
       "\n",
       "    this.ondownload = ondownload;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_header = function() {\n",
       "    var titlebar = $(\n",
       "        '<div class=\"ui-dialog-titlebar ui-widget-header ui-corner-all ' +\n",
       "        'ui-helper-clearfix\"/>');\n",
       "    var titletext = $(\n",
       "        '<div class=\"ui-dialog-title\" style=\"width: 100%; ' +\n",
       "        'text-align: center; padding: 3px;\"/>');\n",
       "    titlebar.append(titletext)\n",
       "    this.root.append(titlebar);\n",
       "    this.header = titletext[0];\n",
       "}\n",
       "\n",
       "\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function(canvas_div) {\n",
       "\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function(canvas_div) {\n",
       "\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_canvas = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var canvas_div = $('<div/>');\n",
       "\n",
       "    canvas_div.attr('style', 'position: relative; clear: both; outline: 0');\n",
       "\n",
       "    function canvas_keyboard_event(event) {\n",
       "        return fig.key_event(event, event['data']);\n",
       "    }\n",
       "\n",
       "    canvas_div.keydown('key_press', canvas_keyboard_event);\n",
       "    canvas_div.keyup('key_release', canvas_keyboard_event);\n",
       "    this.canvas_div = canvas_div\n",
       "    this._canvas_extra_style(canvas_div)\n",
       "    this.root.append(canvas_div);\n",
       "\n",
       "    var canvas = $('<canvas/>');\n",
       "    canvas.addClass('mpl-canvas');\n",
       "    canvas.attr('style', \"left: 0; top: 0; z-index: 0; outline: 0\")\n",
       "\n",
       "    this.canvas = canvas[0];\n",
       "    this.context = canvas[0].getContext(\"2d\");\n",
       "\n",
       "    var rubberband = $('<canvas/>');\n",
       "    rubberband.attr('style', \"position: absolute; left: 0; top: 0; z-index: 1;\")\n",
       "\n",
       "    var pass_mouse_events = true;\n",
       "\n",
       "    canvas_div.resizable({\n",
       "        start: function(event, ui) {\n",
       "            pass_mouse_events = false;\n",
       "        },\n",
       "        resize: function(event, ui) {\n",
       "            fig.request_resize(ui.size.width, ui.size.height);\n",
       "        },\n",
       "        stop: function(event, ui) {\n",
       "            pass_mouse_events = true;\n",
       "            fig.request_resize(ui.size.width, ui.size.height);\n",
       "        },\n",
       "    });\n",
       "\n",
       "    function mouse_event_fn(event) {\n",
       "        if (pass_mouse_events)\n",
       "            return fig.mouse_event(event, event['data']);\n",
       "    }\n",
       "\n",
       "    rubberband.mousedown('button_press', mouse_event_fn);\n",
       "    rubberband.mouseup('button_release', mouse_event_fn);\n",
       "    // Throttle sequential mouse events to 1 every 20ms.\n",
       "    rubberband.mousemove('motion_notify', mouse_event_fn);\n",
       "\n",
       "    rubberband.mouseenter('figure_enter', mouse_event_fn);\n",
       "    rubberband.mouseleave('figure_leave', mouse_event_fn);\n",
       "\n",
       "    canvas_div.on(\"wheel\", function (event) {\n",
       "        event = event.originalEvent;\n",
       "        event['data'] = 'scroll'\n",
       "        if (event.deltaY < 0) {\n",
       "            event.step = 1;\n",
       "        } else {\n",
       "            event.step = -1;\n",
       "        }\n",
       "        mouse_event_fn(event);\n",
       "    });\n",
       "\n",
       "    canvas_div.append(canvas);\n",
       "    canvas_div.append(rubberband);\n",
       "\n",
       "    this.rubberband = rubberband;\n",
       "    this.rubberband_canvas = rubberband[0];\n",
       "    this.rubberband_context = rubberband[0].getContext(\"2d\");\n",
       "    this.rubberband_context.strokeStyle = \"#000000\";\n",
       "\n",
       "    this._resize_canvas = function(width, height) {\n",
       "        // Keep the size of the canvas, canvas container, and rubber band\n",
       "        // canvas in synch.\n",
       "        canvas_div.css('width', width)\n",
       "        canvas_div.css('height', height)\n",
       "\n",
       "        canvas.attr('width', width);\n",
       "        canvas.attr('height', height);\n",
       "\n",
       "        rubberband.attr('width', width);\n",
       "        rubberband.attr('height', height);\n",
       "    }\n",
       "\n",
       "    // Set the figure to an initial 600x600px, this will subsequently be updated\n",
       "    // upon first draw.\n",
       "    this._resize_canvas(600, 600);\n",
       "\n",
       "    // Disable right mouse context menu.\n",
       "    $(this.rubberband_canvas).bind(\"contextmenu\",function(e){\n",
       "        return false;\n",
       "    });\n",
       "\n",
       "    function set_focus () {\n",
       "        canvas.focus();\n",
       "        canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    window.setTimeout(set_focus, 100);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var nav_element = $('<div/>')\n",
       "    nav_element.attr('style', 'width: 100%');\n",
       "    this.root.append(nav_element);\n",
       "\n",
       "    // Define a callback function for later on.\n",
       "    function toolbar_event(event) {\n",
       "        return fig.toolbar_button_onclick(event['data']);\n",
       "    }\n",
       "    function toolbar_mouse_event(event) {\n",
       "        return fig.toolbar_button_onmouseover(event['data']);\n",
       "    }\n",
       "\n",
       "    for(var toolbar_ind in mpl.toolbar_items) {\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) {\n",
       "            // put a spacer in here.\n",
       "            continue;\n",
       "        }\n",
       "        var button = $('<button/>');\n",
       "        button.addClass('ui-button ui-widget ui-state-default ui-corner-all ' +\n",
       "                        'ui-button-icon-only');\n",
       "        button.attr('role', 'button');\n",
       "        button.attr('aria-disabled', 'false');\n",
       "        button.click(method_name, toolbar_event);\n",
       "        button.mouseover(tooltip, toolbar_mouse_event);\n",
       "\n",
       "        var icon_img = $('<span/>');\n",
       "        icon_img.addClass('ui-button-icon-primary ui-icon');\n",
       "        icon_img.addClass(image);\n",
       "        icon_img.addClass('ui-corner-all');\n",
       "\n",
       "        var tooltip_span = $('<span/>');\n",
       "        tooltip_span.addClass('ui-button-text');\n",
       "        tooltip_span.html(tooltip);\n",
       "\n",
       "        button.append(icon_img);\n",
       "        button.append(tooltip_span);\n",
       "\n",
       "        nav_element.append(button);\n",
       "    }\n",
       "\n",
       "    var fmt_picker_span = $('<span/>');\n",
       "\n",
       "    var fmt_picker = $('<select/>');\n",
       "    fmt_picker.addClass('mpl-toolbar-option ui-widget ui-widget-content');\n",
       "    fmt_picker_span.append(fmt_picker);\n",
       "    nav_element.append(fmt_picker_span);\n",
       "    this.format_dropdown = fmt_picker[0];\n",
       "\n",
       "    for (var ind in mpl.extensions) {\n",
       "        var fmt = mpl.extensions[ind];\n",
       "        var option = $(\n",
       "            '<option/>', {selected: fmt === mpl.default_extension}).html(fmt);\n",
       "        fmt_picker.append(option)\n",
       "    }\n",
       "\n",
       "    // Add hover states to the ui-buttons\n",
       "    $( \".ui-button\" ).hover(\n",
       "        function() { $(this).addClass(\"ui-state-hover\");},\n",
       "        function() { $(this).removeClass(\"ui-state-hover\");}\n",
       "    );\n",
       "\n",
       "    var status_bar = $('<span class=\"mpl-message\"/>');\n",
       "    nav_element.append(status_bar);\n",
       "    this.message = status_bar[0];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.request_resize = function(x_pixels, y_pixels) {\n",
       "    // Request matplotlib to resize the figure. Matplotlib will then trigger a resize in the client,\n",
       "    // which will in turn request a refresh of the image.\n",
       "    this.send_message('resize', {'width': x_pixels, 'height': y_pixels});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.send_message = function(type, properties) {\n",
       "    properties['type'] = type;\n",
       "    properties['figure_id'] = this.id;\n",
       "    this.ws.send(JSON.stringify(properties));\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.send_draw_message = function() {\n",
       "    if (!this.waiting) {\n",
       "        this.waiting = true;\n",
       "        this.ws.send(JSON.stringify({type: \"draw\", figure_id: this.id}));\n",
       "    }\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype.handle_save = function(fig, msg) {\n",
       "    var format_dropdown = fig.format_dropdown;\n",
       "    var format = format_dropdown.options[format_dropdown.selectedIndex].value;\n",
       "    fig.ondownload(fig, format);\n",
       "}\n",
       "\n",
       "\n",
       "mpl.figure.prototype.handle_resize = function(fig, msg) {\n",
       "    var size = msg['size'];\n",
       "    if (size[0] != fig.canvas.width || size[1] != fig.canvas.height) {\n",
       "        fig._resize_canvas(size[0], size[1]);\n",
       "        fig.send_message(\"refresh\", {});\n",
       "    };\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_rubberband = function(fig, msg) {\n",
       "    var x0 = msg['x0'];\n",
       "    var y0 = fig.canvas.height - msg['y0'];\n",
       "    var x1 = msg['x1'];\n",
       "    var y1 = fig.canvas.height - msg['y1'];\n",
       "    x0 = Math.floor(x0) + 0.5;\n",
       "    y0 = Math.floor(y0) + 0.5;\n",
       "    x1 = Math.floor(x1) + 0.5;\n",
       "    y1 = Math.floor(y1) + 0.5;\n",
       "    var min_x = Math.min(x0, x1);\n",
       "    var min_y = Math.min(y0, y1);\n",
       "    var width = Math.abs(x1 - x0);\n",
       "    var height = Math.abs(y1 - y0);\n",
       "\n",
       "    fig.rubberband_context.clearRect(\n",
       "        0, 0, fig.canvas.width, fig.canvas.height);\n",
       "\n",
       "    fig.rubberband_context.strokeRect(min_x, min_y, width, height);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_figure_label = function(fig, msg) {\n",
       "    // Updates the figure title.\n",
       "    fig.header.textContent = msg['label'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_cursor = function(fig, msg) {\n",
       "    var cursor = msg['cursor'];\n",
       "    switch(cursor)\n",
       "    {\n",
       "    case 0:\n",
       "        cursor = 'pointer';\n",
       "        break;\n",
       "    case 1:\n",
       "        cursor = 'default';\n",
       "        break;\n",
       "    case 2:\n",
       "        cursor = 'crosshair';\n",
       "        break;\n",
       "    case 3:\n",
       "        cursor = 'move';\n",
       "        break;\n",
       "    }\n",
       "    fig.rubberband_canvas.style.cursor = cursor;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_message = function(fig, msg) {\n",
       "    fig.message.textContent = msg['message'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_draw = function(fig, msg) {\n",
       "    // Request the server to send over a new figure.\n",
       "    fig.send_draw_message();\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_image_mode = function(fig, msg) {\n",
       "    fig.image_mode = msg['mode'];\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function() {\n",
       "    // Called whenever the canvas gets updated.\n",
       "    this.send_message(\"ack\", {});\n",
       "}\n",
       "\n",
       "// A function to construct a web socket function for onmessage handling.\n",
       "// Called in the figure constructor.\n",
       "mpl.figure.prototype._make_on_message_function = function(fig) {\n",
       "    return function socket_on_message(evt) {\n",
       "        if (evt.data instanceof Blob) {\n",
       "            /* FIXME: We get \"Resource interpreted as Image but\n",
       "             * transferred with MIME type text/plain:\" errors on\n",
       "             * Chrome.  But how to set the MIME type?  It doesn't seem\n",
       "             * to be part of the websocket stream */\n",
       "            evt.data.type = \"image/png\";\n",
       "\n",
       "            /* Free the memory for the previous frames */\n",
       "            if (fig.imageObj.src) {\n",
       "                (window.URL || window.webkitURL).revokeObjectURL(\n",
       "                    fig.imageObj.src);\n",
       "            }\n",
       "\n",
       "            fig.imageObj.src = (window.URL || window.webkitURL).createObjectURL(\n",
       "                evt.data);\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "        else if (typeof evt.data === 'string' && evt.data.slice(0, 21) == \"data:image/png;base64\") {\n",
       "            fig.imageObj.src = evt.data;\n",
       "            fig.updated_canvas_event();\n",
       "            fig.waiting = false;\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        var msg = JSON.parse(evt.data);\n",
       "        var msg_type = msg['type'];\n",
       "\n",
       "        // Call the  \"handle_{type}\" callback, which takes\n",
       "        // the figure and JSON message as its only arguments.\n",
       "        try {\n",
       "            var callback = fig[\"handle_\" + msg_type];\n",
       "        } catch (e) {\n",
       "            console.log(\"No handler for the '\" + msg_type + \"' message type: \", msg);\n",
       "            return;\n",
       "        }\n",
       "\n",
       "        if (callback) {\n",
       "            try {\n",
       "                // console.log(\"Handling '\" + msg_type + \"' message: \", msg);\n",
       "                callback(fig, msg);\n",
       "            } catch (e) {\n",
       "                console.log(\"Exception inside the 'handler_\" + msg_type + \"' callback:\", e, e.stack, msg);\n",
       "            }\n",
       "        }\n",
       "    };\n",
       "}\n",
       "\n",
       "// from http://stackoverflow.com/questions/1114465/getting-mouse-location-in-canvas\n",
       "mpl.findpos = function(e) {\n",
       "    //this section is from http://www.quirksmode.org/js/events_properties.html\n",
       "    var targ;\n",
       "    if (!e)\n",
       "        e = window.event;\n",
       "    if (e.target)\n",
       "        targ = e.target;\n",
       "    else if (e.srcElement)\n",
       "        targ = e.srcElement;\n",
       "    if (targ.nodeType == 3) // defeat Safari bug\n",
       "        targ = targ.parentNode;\n",
       "\n",
       "    // jQuery normalizes the pageX and pageY\n",
       "    // pageX,Y are the mouse positions relative to the document\n",
       "    // offset() returns the position of the element relative to the document\n",
       "    var x = e.pageX - $(targ).offset().left;\n",
       "    var y = e.pageY - $(targ).offset().top;\n",
       "\n",
       "    return {\"x\": x, \"y\": y};\n",
       "};\n",
       "\n",
       "/*\n",
       " * return a copy of an object with only non-object keys\n",
       " * we need this to avoid circular references\n",
       " * http://stackoverflow.com/a/24161582/3208463\n",
       " */\n",
       "function simpleKeys (original) {\n",
       "  return Object.keys(original).reduce(function (obj, key) {\n",
       "    if (typeof original[key] !== 'object')\n",
       "        obj[key] = original[key]\n",
       "    return obj;\n",
       "  }, {});\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.mouse_event = function(event, name) {\n",
       "    var canvas_pos = mpl.findpos(event)\n",
       "\n",
       "    if (name === 'button_press')\n",
       "    {\n",
       "        this.canvas.focus();\n",
       "        this.canvas_div.focus();\n",
       "    }\n",
       "\n",
       "    var x = canvas_pos.x;\n",
       "    var y = canvas_pos.y;\n",
       "\n",
       "    this.send_message(name, {x: x, y: y, button: event.button,\n",
       "                             step: event.step,\n",
       "                             guiEvent: simpleKeys(event)});\n",
       "\n",
       "    /* This prevents the web browser from automatically changing to\n",
       "     * the text insertion cursor when the button is pressed.  We want\n",
       "     * to control all of the cursor setting manually through the\n",
       "     * 'cursor' event from matplotlib */\n",
       "    event.preventDefault();\n",
       "    return false;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function(event, name) {\n",
       "    // Handle any extra behaviour associated with a key event\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.key_event = function(event, name) {\n",
       "\n",
       "    // Prevent repeat events\n",
       "    if (name == 'key_press')\n",
       "    {\n",
       "        if (event.which === this._key)\n",
       "            return;\n",
       "        else\n",
       "            this._key = event.which;\n",
       "    }\n",
       "    if (name == 'key_release')\n",
       "        this._key = null;\n",
       "\n",
       "    var value = '';\n",
       "    if (event.ctrlKey && event.which != 17)\n",
       "        value += \"ctrl+\";\n",
       "    if (event.altKey && event.which != 18)\n",
       "        value += \"alt+\";\n",
       "    if (event.shiftKey && event.which != 16)\n",
       "        value += \"shift+\";\n",
       "\n",
       "    value += 'k';\n",
       "    value += event.which.toString();\n",
       "\n",
       "    this._key_event_extra(event, name);\n",
       "\n",
       "    this.send_message(name, {key: value,\n",
       "                             guiEvent: simpleKeys(event)});\n",
       "    return false;\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onclick = function(name) {\n",
       "    if (name == 'download') {\n",
       "        this.handle_save(this, null);\n",
       "    } else {\n",
       "        this.send_message(\"toolbar_button\", {name: name});\n",
       "    }\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.toolbar_button_onmouseover = function(tooltip) {\n",
       "    this.message.textContent = tooltip;\n",
       "};\n",
       "mpl.toolbar_items = [[\"Home\", \"Reset original view\", \"fa fa-home icon-home\", \"home\"], [\"Back\", \"Back to  previous view\", \"fa fa-arrow-left icon-arrow-left\", \"back\"], [\"Forward\", \"Forward to next view\", \"fa fa-arrow-right icon-arrow-right\", \"forward\"], [\"\", \"\", \"\", \"\"], [\"Pan\", \"Pan axes with left mouse, zoom with right\", \"fa fa-arrows icon-move\", \"pan\"], [\"Zoom\", \"Zoom to rectangle\", \"fa fa-square-o icon-check-empty\", \"zoom\"], [\"\", \"\", \"\", \"\"], [\"Download\", \"Download plot\", \"fa fa-floppy-o icon-save\", \"download\"]];\n",
       "\n",
       "mpl.extensions = [\"eps\", \"jpeg\", \"pdf\", \"png\", \"ps\", \"raw\", \"svg\", \"tif\"];\n",
       "\n",
       "mpl.default_extension = \"png\";var comm_websocket_adapter = function(comm) {\n",
       "    // Create a \"websocket\"-like object which calls the given IPython comm\n",
       "    // object with the appropriate methods. Currently this is a non binary\n",
       "    // socket, so there is still some room for performance tuning.\n",
       "    var ws = {};\n",
       "\n",
       "    ws.close = function() {\n",
       "        comm.close()\n",
       "    };\n",
       "    ws.send = function(m) {\n",
       "        //console.log('sending', m);\n",
       "        comm.send(m);\n",
       "    };\n",
       "    // Register the callback with on_msg.\n",
       "    comm.on_msg(function(msg) {\n",
       "        //console.log('receiving', msg['content']['data'], msg);\n",
       "        // Pass the mpl event to the overriden (by mpl) onmessage function.\n",
       "        ws.onmessage(msg['content']['data'])\n",
       "    });\n",
       "    return ws;\n",
       "}\n",
       "\n",
       "mpl.mpl_figure_comm = function(comm, msg) {\n",
       "    // This is the function which gets called when the mpl process\n",
       "    // starts-up an IPython Comm through the \"matplotlib\" channel.\n",
       "\n",
       "    var id = msg.content.data.id;\n",
       "    // Get hold of the div created by the display call when the Comm\n",
       "    // socket was opened in Python.\n",
       "    var element = $(\"#\" + id);\n",
       "    var ws_proxy = comm_websocket_adapter(comm)\n",
       "\n",
       "    function ondownload(figure, format) {\n",
       "        window.open(figure.imageObj.src);\n",
       "    }\n",
       "\n",
       "    var fig = new mpl.figure(id, ws_proxy,\n",
       "                           ondownload,\n",
       "                           element.get(0));\n",
       "\n",
       "    // Call onopen now - mpl needs it, as it is assuming we've passed it a real\n",
       "    // web socket which is closed, not our websocket->open comm proxy.\n",
       "    ws_proxy.onopen();\n",
       "\n",
       "    fig.parent_element = element.get(0);\n",
       "    fig.cell_info = mpl.find_output_cell(\"<div id='\" + id + \"'></div>\");\n",
       "    if (!fig.cell_info) {\n",
       "        console.error(\"Failed to find cell for figure\", id, fig);\n",
       "        return;\n",
       "    }\n",
       "\n",
       "    var output_index = fig.cell_info[2]\n",
       "    var cell = fig.cell_info[0];\n",
       "\n",
       "};\n",
       "\n",
       "mpl.figure.prototype.handle_close = function(fig, msg) {\n",
       "    fig.root.unbind('remove')\n",
       "\n",
       "    // Update the output cell to use the data from the current canvas.\n",
       "    fig.push_to_output();\n",
       "    var dataURL = fig.canvas.toDataURL();\n",
       "    // Re-enable the keyboard manager in IPython - without this line, in FF,\n",
       "    // the notebook keyboard shortcuts fail.\n",
       "    IPython.keyboard_manager.enable()\n",
       "    $(fig.parent_element).html('<img src=\"' + dataURL + '\">');\n",
       "    fig.close_ws(fig, msg);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.close_ws = function(fig, msg){\n",
       "    fig.send_message('closing', msg);\n",
       "    // fig.ws.close()\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.push_to_output = function(remove_interactive) {\n",
       "    // Turn the data on the canvas into data in the output cell.\n",
       "    var dataURL = this.canvas.toDataURL();\n",
       "    this.cell_info[1]['text/html'] = '<img src=\"' + dataURL + '\">';\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.updated_canvas_event = function() {\n",
       "    // Tell IPython that the notebook contents must change.\n",
       "    IPython.notebook.set_dirty(true);\n",
       "    this.send_message(\"ack\", {});\n",
       "    var fig = this;\n",
       "    // Wait a second, then push the new image to the DOM so\n",
       "    // that it is saved nicely (might be nice to debounce this).\n",
       "    setTimeout(function () { fig.push_to_output() }, 1000);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._init_toolbar = function() {\n",
       "    var fig = this;\n",
       "\n",
       "    var nav_element = $('<div/>')\n",
       "    nav_element.attr('style', 'width: 100%');\n",
       "    this.root.append(nav_element);\n",
       "\n",
       "    // Define a callback function for later on.\n",
       "    function toolbar_event(event) {\n",
       "        return fig.toolbar_button_onclick(event['data']);\n",
       "    }\n",
       "    function toolbar_mouse_event(event) {\n",
       "        return fig.toolbar_button_onmouseover(event['data']);\n",
       "    }\n",
       "\n",
       "    for(var toolbar_ind in mpl.toolbar_items){\n",
       "        var name = mpl.toolbar_items[toolbar_ind][0];\n",
       "        var tooltip = mpl.toolbar_items[toolbar_ind][1];\n",
       "        var image = mpl.toolbar_items[toolbar_ind][2];\n",
       "        var method_name = mpl.toolbar_items[toolbar_ind][3];\n",
       "\n",
       "        if (!name) { continue; };\n",
       "\n",
       "        var button = $('<button class=\"btn btn-default\" href=\"#\" title=\"' + name + '\"><i class=\"fa ' + image + ' fa-lg\"></i></button>');\n",
       "        button.click(method_name, toolbar_event);\n",
       "        button.mouseover(tooltip, toolbar_mouse_event);\n",
       "        nav_element.append(button);\n",
       "    }\n",
       "\n",
       "    // Add the status bar.\n",
       "    var status_bar = $('<span class=\"mpl-message\" style=\"text-align:right; float: right;\"/>');\n",
       "    nav_element.append(status_bar);\n",
       "    this.message = status_bar[0];\n",
       "\n",
       "    // Add the close button to the window.\n",
       "    var buttongrp = $('<div class=\"btn-group inline pull-right\"></div>');\n",
       "    var button = $('<button class=\"btn btn-mini btn-primary\" href=\"#\" title=\"Stop Interaction\"><i class=\"fa fa-power-off icon-remove icon-large\"></i></button>');\n",
       "    button.click(function (evt) { fig.handle_close(fig, {}); } );\n",
       "    button.mouseover('Stop Interaction', toolbar_mouse_event);\n",
       "    buttongrp.append(button);\n",
       "    var titlebar = this.root.find($('.ui-dialog-titlebar'));\n",
       "    titlebar.prepend(buttongrp);\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._root_extra_style = function(el){\n",
       "    var fig = this\n",
       "    el.on(\"remove\", function(){\n",
       "\tfig.close_ws(fig, {});\n",
       "    });\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._canvas_extra_style = function(el){\n",
       "    // this is important to make the div 'focusable\n",
       "    el.attr('tabindex', 0)\n",
       "    // reach out to IPython and tell the keyboard manager to turn it's self\n",
       "    // off when our div gets focus\n",
       "\n",
       "    // location in version 3\n",
       "    if (IPython.notebook.keyboard_manager) {\n",
       "        IPython.notebook.keyboard_manager.register_events(el);\n",
       "    }\n",
       "    else {\n",
       "        // location in version 2\n",
       "        IPython.keyboard_manager.register_events(el);\n",
       "    }\n",
       "\n",
       "}\n",
       "\n",
       "mpl.figure.prototype._key_event_extra = function(event, name) {\n",
       "    var manager = IPython.notebook.keyboard_manager;\n",
       "    if (!manager)\n",
       "        manager = IPython.keyboard_manager;\n",
       "\n",
       "    // Check for shift+enter\n",
       "    if (event.shiftKey && event.which == 13) {\n",
       "        this.canvas_div.blur();\n",
       "        // select the cell after this one\n",
       "        var index = IPython.notebook.find_cell_index(this.cell_info[0]);\n",
       "        IPython.notebook.select(index + 1);\n",
       "    }\n",
       "}\n",
       "\n",
       "mpl.figure.prototype.handle_save = function(fig, msg) {\n",
       "    fig.ondownload(fig, null);\n",
       "}\n",
       "\n",
       "\n",
       "mpl.find_output_cell = function(html_output) {\n",
       "    // Return the cell and output element which can be found *uniquely* in the notebook.\n",
       "    // Note - this is a bit hacky, but it is done because the \"notebook_saving.Notebook\"\n",
       "    // IPython event is triggered only after the cells have been serialised, which for\n",
       "    // our purposes (turning an active figure into a static one), is too late.\n",
       "    var cells = IPython.notebook.get_cells();\n",
       "    var ncells = cells.length;\n",
       "    for (var i=0; i<ncells; i++) {\n",
       "        var cell = cells[i];\n",
       "        if (cell.cell_type === 'code'){\n",
       "            for (var j=0; j<cell.output_area.outputs.length; j++) {\n",
       "                var data = cell.output_area.outputs[j];\n",
       "                if (data.data) {\n",
       "                    // IPython >= 3 moved mimebundle to data attribute of output\n",
       "                    data = data.data;\n",
       "                }\n",
       "                if (data['text/html'] == html_output) {\n",
       "                    return [cell, data, j];\n",
       "                }\n",
       "            }\n",
       "        }\n",
       "    }\n",
       "}\n",
       "\n",
       "// Register the function which deals with the matplotlib target/channel.\n",
       "// The kernel may be null if the page has been refreshed.\n",
       "if (IPython.notebook.kernel != null) {\n",
       "    IPython.notebook.kernel.comm_manager.register_target('matplotlib', mpl.mpl_figure_comm);\n",
       "}\n"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<img src=\"data:image/png;base64,[encoded data, 41668 characters]\">"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Warning: using a non-integer number instead of an integer will result in an error in the future\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x118ce3cc0>"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sns.distplot(scores)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion\n",
    "\n",
    "We have imported the Agribalyse database. In the process of importing, we found and resolved several problems:\n",
    "\n",
    "1. First, we had to fix the names and categories of biosphere flow names, to make them compatible with the names and categories used in ecoinvent version 3.\n",
    "2. Next, we created a new database for the new biosphere flows that we couldn't match.\n",
    "3. We linked production flows to the activities that produced them, using a strategy that didn't use the field `categories`, as this field is not given consistently in SimaPro exports.\n",
    "4. We switched some outputs to inputs, to be consistent with how ecoinvent models disposal and recycling processes.\n",
    "5. We linked inputs to activities in ecoinvent 2.2, again ignoring the field `categories`, because SimaPro.\n",
    "6. We created new processes to provide recycling services.\n",
    "7. We deleted the unique identifying codes used by SimaPro, as they were not actually unique, and created our own codes.\n",
    "\n",
    "This was a bit of a pain, but compared to other database exports, was actually not all that difficult. This is the sad truth of LCA data compatibility - it currently isn't all that great."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
