<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:wfw="http://wellformedweb.org/CommentAPI/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:sy="http://purl.org/rss/1.0/modules/syndication/" xmlns:slash="http://purl.org/rss/1.0/modules/slash/" xmlns:georss="http://www.georss.org/georss" xmlns:geo="http://www.w3.org/2003/01/geo/wgs84_pos#" xmlns:media="http://search.yahoo.com/mrss/">
  <channel>
    <title>OUseful.Info, the blog...</title>
    <atom:link href="http://blog.ouseful.info/feed/" rel="self" type="application/rss+xml" />
    <link>http://blog.ouseful.info</link>
    <description>Trying to find useful things to do with emerging technologies in open education</description>
    <lastBuildDate>Sun, 24 Aug 2014 18:09:27 +0000</lastBuildDate>
    <language>en</language>
    <sy:updatePeriod>hourly</sy:updatePeriod>
    <sy:updateFrequency>1</sy:updateFrequency>
    <generator>http://wordpress.com/</generator>
    <cloud domain='blog.ouseful.info' port='80' path='/?rsscloud=notify' registerProcedure='' protocol='http-post' />
    <image>
      <url>http://s2.wp.com/i/buttonw-com.png</url>
      <title>OUseful.Info, the blog...</title>
      <link>http://blog.ouseful.info</link>
    </image>
    <atom:link rel="search" type="application/opensearchdescription+xml" href="http://blog.ouseful.info/osd.xml" title="OUseful.Info, the blog..." />
    <atom:link rel='hub' href='http://blog.ouseful.info/?pushpress=hub' />
    <item>
      <title>Running &#8220;Native&#8221; Data Wrangling Applications in the Browser &#8211; IPython Notebooks (and R?) in Chrome</title>
      <link>http://blog.ouseful.info/2014/08/22/running-native-data-wrangling-applications-in-the-browser-ipython-notebooks-and-r-in-chrome/</link>
      <comments>http://blog.ouseful.info/2014/08/22/running-native-data-wrangling-applications-in-the-browser-ipython-notebooks-and-r-in-chrome/#comments</comments>
      <pubDate>Fri, 22 Aug 2014 09:42:21 +0000</pubDate>
      <dc:creator>
        <![CDATA[Tony Hirst]]>
      </dc:creator>
      <category>
        <![CDATA[Anything you want]]>
      </category>
      <category>
        <![CDATA[Rstats]]>
      </category>
      <guid isPermaLink="false">http://blog.ouseful.info/?p=12065</guid>
      <description>
        <![CDATA[Using browser based data analysis toolkits such as pandas in IPython notebooks, or R in RStudio, means you need to have access to python or R and the corresponding application server either on your own computer, or running on a remote server that you have access to. When running occasional training sessions or workshops, this [&#8230;]<img alt="" border="0" src="http://pixel.wp.com/b.gif?host=blog.ouseful.info&#038;blog=325417&#038;post=12065&#038;subd=ouseful&#038;ref=&#038;feed=1" width="1" height="1" />]]></description>
      <content:encoded>
        <![CDATA[<p>Using browser based data analysis toolkits such as pandas in IPython notebooks, or R in RStudio, means you need to have access to python or R and the corresponding application server either on your own computer, or running on a remote server that you have access to.</p>
<p>When running occasional training sessions or workshops, this can cause several headaches: either a remote service needs to be set up that is capable of supporting the expected number of participants, security may need putting in place, accounts configured (or account management tools supported), network connections need guaranteeing so that participants can access the server, and so on; or participants need to install software on their own computers: ideally this would be done in advance of a training session, otherwise training time is spent installing, configuring and debugging software installs; some computers may have security policies that prevent users installing software, or require and IT person with admin privileges to install the software, and so on.</p>
<p>That&#8217;s why the <a href="https://github.com/jupyter/colaboratory"><em>coLaboratory</em></a> <a href="https://chrome.google.com/webstore/detail/colaboratory-notebook/pianggobfjcgeihlmfhfgkfalopndooo/">Chrome extension</a> looks like an interesting innovation &#8211; it runs an IPython notebook fork, with <em>pandas</em> and <em>matplotlib</em> as a Chrome Native Client application. I posted a quick walkthrough of the extension over on the School of Data blog: <a href="http://schoolofdata.org/2014/08/20/working-with-data-in-the-browser-using-python/">Working With Data in the Browser Using python – coLaboratory</a>.</p>
<p>Via a Twitter exchange with @nativeclient, it seems that there&#8217;s also the possibility that R could run as a dependency free Chrome extension. Native Client seems to like things written in C/C++, which underpins R, although I think R also has some fortran dependencies. (One of the coLaboratory talks mentioned the to do list item of getting scipy (I think?) running in the coLaboratory extension, the major challenge there (or whatever the package was) being the fortran src; so there maybe be synergies in working the fortran components there?)) </p>
<p>Within a couple of hours of the twitter exchange starting, Brad Nelson/@flagxor posted a <a href="https://codereview.chromium.org/495183003/#ps20001">first attempt at an R port to the Native Client</a>. I don&#8217;t pretend to understand what&#8217;s involved in moving from this to an extension with some sort of useable UI, even if only a command line, but it represents an interesting possibility: of being able to run R in the browser (or at least, in Chrome). Package availability would be limited of course to packages compiled to run using PNaCl.</p>
<p>For training events, there is still the requirement that users install a Chrome browser on their computer and then install the extension into that. However, I think it is possible to run Chrome as a portable app &#8211; that is, from a flash drive such as a USB memory stick: <a href="http://portableapps.com/apps/internet/google_chrome_portable">Google Chrome Portable (Windows)</a>.</p>
<p>I&#8217;m not sure how <em>fast</em> it would be able to run, but it suggests there may be a way of carrying a portable, dependency free <em>pandas</em> environment around that you can run on a Windows computer from a USB key?! And maybe R too&#8230;?</p><br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/ouseful.wordpress.com/12065/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/ouseful.wordpress.com/12065/" /></a> <img alt="" border="0" src="http://pixel.wp.com/b.gif?host=blog.ouseful.info&#038;blog=325417&#038;post=12065&#038;subd=ouseful&#038;ref=&#038;feed=1" width="1" height="1" />]]>
      </content:encoded>
      <wfw:commentRss>http://blog.ouseful.info/2014/08/22/running-native-data-wrangling-applications-in-the-browser-ipython-notebooks-and-r-in-chrome/feed/</wfw:commentRss>
      <slash:comments>0</slash:comments>
      <media:content url="http://1.gravatar.com/avatar/abbd9f90565ce9ae4d065d93a81d8c03?s=96&#38;d=http%3A%2F%2F1.gravatar.com%2Favatar%2Fad516503a11cd5ca435acc9bb6523536%3Fs%3D96" medium="image">
        <media:title type="html">Tony Hirst</media:title>
      </media:content>
    </item>
    <item>
      <title>More OpenData Published &#8211; So What?</title>
      <link>http://blog.ouseful.info/2014/08/15/more-opendata-published-so-what/</link>
      <comments>http://blog.ouseful.info/2014/08/15/more-opendata-published-so-what/#comments</comments>
      <pubDate>Fri, 15 Aug 2014 09:56:24 +0000</pubDate>
      <dc:creator>
        <![CDATA[Tony Hirst]]>
      </dc:creator>
      <category>
        <![CDATA[Open Data]]>
      </category>
      <category>
        <![CDATA[Policy]]>
      </category>
      <category>
        <![CDATA[360 Giving]]>
      </category>
      <guid isPermaLink="false">http://blog.ouseful.info/?p=12059</guid>
      <description>
        <![CDATA[Whenever a new open data dataset is released, the #opendata wires hum a little more. More open data is a Good Thing, right? Why? Haven&#8217;t we got enough already? In a blog post a few weeks ago, Alan Levine, aka @cogdog, set about Stalking the Mythical OER Reuse: Seeking Non-Blurry Videos. OERs are open educational [&#8230;]<img alt="" border="0" src="http://pixel.wp.com/b.gif?host=blog.ouseful.info&#038;blog=325417&#038;post=12059&#038;subd=ouseful&#038;ref=&#038;feed=1" width="1" height="1" />]]></description>
      <content:encoded>
        <![CDATA[<p>Whenever a new open data dataset is released, the #opendata wires hum a little more. More open data is a Good Thing, right? Why? Haven&#8217;t we got enough already?</p>
<p>In a blog post a few weeks ago, Alan Levine, aka @cogdog, set about <a href="http://cogdogblog.com/2014/07/18/stalking-the-mythical/">Stalking the Mythical OER Reuse: Seeking Non-Blurry Videos</a>. OERs are <em>open educational resources</em>, openly licensed materials produced by educators and released to the world so others could make use of them. Funding was put into developing and releasing them and then, &#8230; what?</p>
<blockquote><p>OERs. People build them. People house them in repositories. People do journal articles, conference presentations, research on them. I doubt never their existence.</p>
<p>But the ultimate thing they are supposed to support, maybe their raison d’être – the re use by other educators, what do we have to show for that except whispered stories, innuendo, and blurry photos in the forest?<br />
&#8211;<em>Alan Levine</em></p></blockquote>
<p>Alan went in search of the OER reuse in his own inimitable way&#8230;</p>
<span class='embed-youtube' style='text-align:center; display: block;'><iframe class='youtube-player' type='text/html' width='700' height='424' src='http://www.youtube.com/embed/bb7QUqBUoyI?version=3&#038;rel=1&#038;fs=1&#038;showsearch=0&#038;showinfo=1&#038;iv_load_policy=1&#038;wmode=transparent' frameborder='0'></iframe></span>
<p>&#8230; but came back without much success. He then used the rest of the post to put out all for stories about how OERs have actually been used in the world&#8230; Not just mythical stories, not coulds and mights: real examples.</p>
<p>So what about opendata &#8211; is there much use, or reuse, going on there?</p>
<p>It seems as is more datasets get opened every day, but is there more use every day, first day use of newly released datasets, incremental reuse of the datasets that are already out, linkage between the new datasets and the previously released ones.</p>
<p>Yesterday, I spotted via @owenboswarva the release of a dataset that aggregated and normalised data relating to charitable grant awards: <a href="http://guest.thirdsector.co.uk/2014/08/14/a-big-day-for-charity-data/">A big day for charity data</a>. Interesting&#8230; The supporting website &#8211; <a href="http://lin-360giving.aptivate.org/">360 Giving</a> &#8211; (self-admittedly in it&#8217;s early days) allows you to search by funder, recipient or key word. You have to search using the right keywords, though, and the right capitalisation of keywords&#8230;</p>
<p><a href="https://ouseful.files.wordpress.com/2014/08/360giving-unioxford.png"><img src="http://ouseful.files.wordpress.com/2014/08/360giving-unioxford.png?w=700&#038;h=457" alt="360giving-uniOxford" width="700" height="457" class="alignnone size-full wp-image-12060" /></a></p>
<p>And you may have to add in white space.. so *University of Oxford * as well as *University of Oxford*.</p>
<p>I don&#8217;t want to knock the site, but I am really interested to know how this data might be used. Really. Genuinely. I am properly interested. How would someone working in the charitable sector use that website to help them do something? What thing? How would it support them? My imagination may be able to go off on crazy flights of fancy in certain areas, but my lack of sector knowledge or a current headful of summer cold leaves me struggling to work out what this website would tangibly help someone to do. (I tried to ask a similar question around charities data before, giving the example of <a href="http://blog.ouseful.info/2013/05/01/a-quick-peek-at-some-charities-data/">Charities Commission data grabbed from OpenCharities</a>, but drew a blank then.) Like @cogdog in his search for real OER use case stories, I&#8217;d love to hear examples of real questions &#8211; no matter how trivial &#8211; that the 360 Giving site could help answer.</p>
<p>As well as the website, 360 Giving folk provide a data download as a CSV file containing getting on for a quarter of a million records. The date stamp on the file I grabbed is 5th June 2014. Skimming through the data quickly &#8211; my own opening conversation with it can be found here: <a href="http://nbviewer.ipython.org/gist/psychemedia/30a9353b6c0bcf2dc866">360 Giving Grant Navigator &#8211; Initial Data Conversation</a> &#8211; I noticed through comparison with the data on the website some gaps&#8230; </p>
<ul>
<li><a href="http://lin-360giving.aptivate.org/grants/360G-HLF%3AHG-03-00085">this item</a> doesn&#8217;t seem to appear in the CSV download, perhaps because it doesn&#8217;t appear to have a funder?</li>
<li><a href="http://lin-360giving.aptivate.org/grants/360G-northernrock:981002">this item</a> on the website has an address for the recipient organisation, but the CSV document doesn&#8217;t have any address fields. In fact, on close inspection, the record relates to a grant by the <em>Northern Rock Foundation</em>, and I see no records from that body in the CSV file?</li>
<li>Although there is a project title field in the CSV document, no project titles are supplied. Looking through a sample of grants on the website, are any titles provided?</li>
<li>The website lists the following funders:
<p>Arts Council England<br />
Arts Council Wales<br />
Big Lottery<br />
Creative Scotland<br />
DSDNI<br />
Heritage Lottery Fund<br />
Indigo Trust<br />
Nesta<br />
Nominet Trust<br />
Northern Rock Foundation<br />
Paul Hamlyn Foundation<br />
Sport England<br />
Sport Northern Ireland<br />
Sport Wales<br />
TSB<br />
Wellcome Trust</p>
<p>The CSV file has data from these funders:</p>
<p>Arts Council England<br />
Arts Council Wales<br />
Big Lottery<br />
Creative Scotland<br />
DSDNI<br />
Nesta<br />
Nominet Trust<br />
Sport England<br />
Sport Northern Ireland<br />
Sport Wales<br />
TSB<br />
Wellcome Trust</p>
<p>That is, the CSV contains a subset of the data on the website; data from Heritage Lottery Fund, Indigo Trust, Northern Rock Foundation, Paul Hamlyn Foundation doesn&#8217;t seem to have made it into the data download? I also note that data from the Research Councils&#8217; Gateway to Research (aside from the TSB data) doesn&#8217;t seem to have made it into either dataset. For anyone researching grants to universities, this could be useful information. (Could?! Why?!;-)</li>
<li>No company numbers or Charity Numbers are given. Using opendata from <a href="http://download.companieshouse.gov.uk/en_output.html">Companies House</a> a quick join on recipient names and company names from the Companies House register (without any attempts at normalising out things like LTD and LIMITED  &#8211; that is, purely looking for an exact match) gives me just over 15,000 matched company names (which means I now have their address, company number, etc. too). And presumably if I try to match on names from the OpenCharities data, I&#8217;ll be able to match some charity numbers. Now both these annotations will be far from complete, but they&#8217;d be more than we have at the moment. A question to then ask is &#8211; is this better or worse? Does the dataset only have value if it is in some way complete? One of the clarion calls for open data initiatives has been to &#8216;just get the data out there&#8217; so that it can be started to be worked on, and improved on. So presumably having some company numbers of charity numbers matched is a plus?
<p>Now I know there is a risk to this. Funders may want to not release details about the addresses of the charities of they are funding because that data may be used to plot maps to say &#8220;this is where the money&#8217;s going&#8221; when it isn&#8217;t. The charity may have a Kensington address and the received funding for an initiative in Oswaldtwistle, but the map might see all the money sinking into Kensington; which would be wrong. But that&#8217;s where you have to start educating the data users. Or releasing data fields like &#8220;address of charity&#8221; and &#8220;postcode area of point of use&#8221;, or whatever, even if the latter is empty. As it is, if you give me a charity or company name, I can look up it&#8217;s address. And its company or charity number if it has one.</li>
</ul>
<p>As I mentioned, I don&#8217;t want to knock the work 360 Giving have done, but I&#8217;m keen to understand what it is they have done, what they haven&#8217;t done, and what the opendata they have aggregated and re-presented could &#8211; practically, tractably, tangibly &#8211; be used for. Really used for.</p>
<p>Time to pack my bags and head out into the wood, maybe&#8230;</p><br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/ouseful.wordpress.com/12059/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/ouseful.wordpress.com/12059/" /></a> <img alt="" border="0" src="http://pixel.wp.com/b.gif?host=blog.ouseful.info&#038;blog=325417&#038;post=12059&#038;subd=ouseful&#038;ref=&#038;feed=1" width="1" height="1" />]]>
      </content:encoded>
      <wfw:commentRss>http://blog.ouseful.info/2014/08/15/more-opendata-published-so-what/feed/</wfw:commentRss>
      <slash:comments>8</slash:comments>
      <media:content url="http://1.gravatar.com/avatar/abbd9f90565ce9ae4d065d93a81d8c03?s=96&#38;d=http%3A%2F%2F1.gravatar.com%2Favatar%2Fad516503a11cd5ca435acc9bb6523536%3Fs%3D96" medium="image">
        <media:title type="html">Tony Hirst</media:title>
      </media:content>
      <media:content url="http://ouseful.files.wordpress.com/2014/08/360giving-unioxford.png" medium="image">
        <media:title type="html">360giving-uniOxford</media:title>
      </media:content>
    </item>
    <item>
      <title>Opening Up Access to Data: Why APIs May Not Be Enough&#8230;</title>
      <link>http://blog.ouseful.info/2014/08/11/opening-up-access-to-data-why-apis-may-not-be-enough/</link>
      <comments>http://blog.ouseful.info/2014/08/11/opening-up-access-to-data-why-apis-may-not-be-enough/#comments</comments>
      <pubDate>Mon, 11 Aug 2014 14:04:21 +0000</pubDate>
      <dc:creator>
        <![CDATA[Tony Hirst]]>
      </dc:creator>
      <category>
        <![CDATA[Data]]>
      </category>
      <category>
        <![CDATA[Rstats]]>
      </category>
      <category>
        <![CDATA[ONS]]>
      </category>
      <guid isPermaLink="false">http://blog.ouseful.info/?p=12051</guid>
      <description>
        <![CDATA[Last week, a post on the ONS (Office of National Statistics) Digital Publishing blog caught my eye: Introducing the New Improved ONS API which apparently &#8220;mak[es] things much easier to work with&#8221;. Ooh&#8230; exciting&#8230;. maybe I can use this to start hacking together some notebooks?:-) It was followed a few days later by this one [&#8230;]<img alt="" border="0" src="http://pixel.wp.com/b.gif?host=blog.ouseful.info&#038;blog=325417&#038;post=12051&#038;subd=ouseful&#038;ref=&#038;feed=1" width="1" height="1" />]]></description>
      <content:encoded>
        <![CDATA[<p>Last week, a post on the ONS (Office of National Statistics) Digital Publishing blog caught my eye: <a href="http://digitalpublishing.ons.gov.uk/2014/08/04/introducing-the-new-improved-ons-api/">Introducing the New Improved ONS API</a> which apparently <em>&#8220;mak[es] things much easier to work with&#8221;</em>.</p>
<p>Ooh&#8230; exciting&#8230;. maybe I can use this to start hacking together some notebooks?:-)</p>
<p>It was followed a few days later by this one &#8211; <a href="http://digitalpublishing.ons.gov.uk/2014/08/07/ons-api-just-the-numbers/">ONS-API, Just the Numbers</a> which described <em>&#8220;a simple bit of code for requesting some data and then turning that into ‘just the raw numbers’&#8221;</em> &#8211; a blog post that describes how to get a simple statistic, as a number, from the API. The API that <em>&#8220;mak[es] things much easier to work with&#8221;</em>.</p>
<p>After a few hours spent hacking away over the weekend, looking round various bits of the API, I still wasn&#8217;t really in a position to discover where to find the numbers, let alone get numbers out of the API in a reliable way. (You can see my fumblings <a href="http://nbviewer.ipython.org/gist/psychemedia/be0c279f2e51f6266b45">here</a>.) Note that I&#8217;m happy to be told I&#8217;m going about this completely the wrong way and didn&#8217;t find the baby steps guide I need to help me use it properly.</p>
<p>So FWIW, here are some reflections, from a personal standpoint, about the whole API thing from the perspective of someone who couldn&#8217;t get it together enough to get the thing working &#8230;</p>
<hr />
<p>Most data users aren&#8217;t programmers. And I&#8217;m not sure how many programmers are data junkies, let alone statisticians and data analysts.</p>
<p>For data users who do dabble with programming &#8211; in R, for example, or python (for example, using the <em>pandas</em> library) &#8211; the offer of an API is often seen as providing a way of interrogating a data source and getting the bits of data you want. The alternative to this is often having to download a huge great dataset yourself and then querying it or partitioning it yourself to get just the data elements you want to make use of (for example, <a href="http://schoolofdata.org/2013/12/05/working-with-large-text-files-finding-uk-companies-by-postcode-or-business-area/">Working With Large Text Files – Finding UK Companies by Postcode or Business Area</a>).</p>
<p>That&#8217;s fine, insofar as it goes, but it starts to give the person who wants to do some data analysis a data management problem too. And for data users who aren&#8217;t happy working with gigabyte data files, it can sometimes be a blocker. (Big file downloads also take time, and incur bandwidth costs.)  </p>
<p>For me, a stereotypical data user might be someone who typically wants to be able to quickly and easily get just the data they want from the API into a data representation that is native to the environment they are working in, and that they are familiar with working with.</p>
<p>This might be a spreadsheet user or it might be a code (R, pandas etc) user.</p>
<p>In the same way that spreadsheet users want files in XLS or CSV format that they can easily open, (formats that can be also be directly opened into appropriate data structures in R or <em>pandas</em>), I increasingly look not for APIs, but for API wrappers, that bring API calls and the results from them directly into the environment I&#8217;m working in in a form appropriate to that environment.</p>
<p>So for example, in R, I make use of the <a href="http://cran.r-project.org/web/packages/FAOSTAT/index.html">FAOstat</a> package, which also offers an interface to the World Bank Indicators datasets. In pandas, a <a href="http://pandas.pydata.org/pandas-docs/version/0.14.1/remote_data.html">remote data access</a> handler for the World Bank Indicators portal allows me to make simple requests for that data.</p>
<p>At a level up (or should that be &#8220;down&#8221;?) from the API wrapper are libraries that parse typical response formats. For example, <em>Statistics Norway</em> seem to publish data using the json-stat format, the format used in the new ONS API update. <a href="http://nbviewer.ipython.org/gist/predicador37/9736709e086f685939eb">This IPython notebook</a> shows how to use the <a href="https://pypi.python.org/pypi/pyjstat/">pyjstat</a> python package to parse the json-stat data directly into a pandas dataframe (I couldn&#8217;t get it to work with the ONS data feed &#8211; not sure if the problem was me, the package, or the data feed; which is another problem &#8211; working out where the problem is&#8230;). For parsing data returned from SPARQL Linked Data endpoints, packages such as <a href="http://rdflib.github.io/sparqlwrapper/">SPARQLwrapper</a> get the data into Python dicts, if not pandas dataframes directly. (A SPARQL i/o wrapper for pandas could be quite handy?)</p>
<p>At the user level, IPython Notebooks (my current &#8216;can be used to solve all known problems&#8217; piece of magic tech!;-) provide a great way of demonstrating not just how to get started with an API, but also encourage the development within the notebook or reusable components, as well as demonstrations of how to use the data. The latter demonstrations have the benefit of requiring that the API demo does actually get the data into a form that is useable within the environment. It also helps folk see what it means to be able to get data into the environment (it means you can do things like the things done in the demo&#8230;; and if you can do that, then you can probably also do other related things&#8230;)</p>
<p>So am I happy when I see APIs announced? Yes and no&#8230; I&#8217;m more interested in having API wrappers available within my data wrangling environment. If that&#8217;s a fully blown wrapper, great. If that sort of wrapper isn&#8217;t available, but I can use a standard data feed parsing library to parse results pulled from easily generated RESTful URLs, I can just about work out how to create the URLs, so that&#8217;s not too bad either.</p>
<p>When publishing APIs, it&#8217;s worth considering who can address them and use them. Just because you publish a data API doesn&#8217;t mean a data analyst can necessarily use the data, because they may not be (are likely not to be) a programmer. And if ten, or a hundred, or a thousand potential data users all have to implement the same sort of glue code to get the data from the API into the same sort of analysis environment, that&#8217;s not necessarily efficient either. (Data users may feel they can hack some code to get the data from the API into the environment for their particular use case, but may not be willing to release it as a general, tested and robust API wrapper, certainly not a stable production level one.)</p>
<p>This isn&#8217;t meant to be a slight against the ONS API, more a reflection on some of the things I was thinking as I hacked my weekend away&#8230;</p>
<p>PS I don&#8217;t know how easy it is to run Python code in R, but the R magic in IPython notebooks supports the running of R code within a notebook running a Python kernel, with the handing over of data from R dataframes to python dataframes. Which is to say, if there&#8217;s an R package available, for someone who can run R via an IPython context, it&#8217;s available via python too.</p>
<p>PPS I notice that from some of the ONS API calls we can get links to URLs of downloadable datasets (though when I tried some of them, I got errors trying to unzip the results). This provides an intermediate way of providing API access to a dataset &#8211; search based API calls that allow discovery of a dataset, then the download and automatic unpacking of that dataset into a native data representation, such as one or more data frames.</p><br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/ouseful.wordpress.com/12051/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/ouseful.wordpress.com/12051/" /></a> <img alt="" border="0" src="http://pixel.wp.com/b.gif?host=blog.ouseful.info&#038;blog=325417&#038;post=12051&#038;subd=ouseful&#038;ref=&#038;feed=1" width="1" height="1" />]]>
      </content:encoded>
      <wfw:commentRss>http://blog.ouseful.info/2014/08/11/opening-up-access-to-data-why-apis-may-not-be-enough/feed/</wfw:commentRss>
      <slash:comments>0</slash:comments>
      <media:content url="http://1.gravatar.com/avatar/abbd9f90565ce9ae4d065d93a81d8c03?s=96&#38;d=http%3A%2F%2F1.gravatar.com%2Favatar%2Fad516503a11cd5ca435acc9bb6523536%3Fs%3D96" medium="image">
        <media:title type="html">Tony Hirst</media:title>
      </media:content>
    </item>
    <item>
      <title>Using Open Data to Hold Companies to Account?</title>
      <link>http://blog.ouseful.info/2014/08/03/using-open-data-to-hold-companies-to-account/</link>
      <comments>http://blog.ouseful.info/2014/08/03/using-open-data-to-hold-companies-to-account/#comments</comments>
      <pubDate>Sun, 03 Aug 2014 00:10:03 +0000</pubDate>
      <dc:creator>
        <![CDATA[Tony Hirst]]>
      </dc:creator>
      <category>
        <![CDATA[Open Data]]>
      </category>
      <category>
        <![CDATA[opengov]]>
      </category>
      <category>
        <![CDATA[Policy]]>
      </category>
      <category>
        <![CDATA[Stirring]]>
      </category>
      <category>
        <![CDATA[beertalkin']]>
      </category>
      <category>
        <![CDATA[opendata]]>
      </category>
      <category>
        <![CDATA[transparency]]>
      </category>
      <guid isPermaLink="false">http://blog.ouseful.info/?p=12022</guid>
      <description>
        <![CDATA[Some rambling but possibly associated thoughts&#8230; I suggest you put Alice&#8217;s Restaurant on&#8230; For some time now, I&#8217;ve had an uncomfortable feeling about the asymmetries that exist in the open data world as well as total confusion about the notion of transparency. Part of the nub of the problem (for me) lies with the asymmetric [&#8230;]<img alt="" border="0" src="http://pixel.wp.com/b.gif?host=blog.ouseful.info&#038;blog=325417&#038;post=12022&#038;subd=ouseful&#038;ref=&#038;feed=1" width="1" height="1" />]]></description>
      <content:encoded>
        <![CDATA[<p>Some rambling but possibly associated thoughts&#8230; I suggest you put <a href="https://www.youtube.com/watch?v=m57gzA2JCcM">Alice&#8217;s Restaurant</a> on&#8230; </p>
<p>For some time now, I&#8217;ve had an uncomfortable feeling about the asymmetries that exist in the open data world as well as <a href="http://blog.ouseful.info/2014/06/24/confused-about-transparency/">total confusion about the notion of <em>transparency</em></a>.</p>
<p>Part of the nub of the problem (for me) lies with the asymmetric disclosure requirements of public and private services. Public bodies have disclosure requirements (eg <a href="https://www.gov.uk/government/publications/local-government-transparency-code-2014">Local Government Transparency Code</a>), private companies don&#8217;t. Public bodies disclose metrics and spend data, data that can be used in public contract tendering processes by private bodies against public ones tendering for the same service. The private body uses this information &#8211; and prices in a discount associated with <em>not</em> having to carry the cost of public reporting &#8211; into the bid. The next time the contract is tendered, the public body won&#8217;t have access to the (previously publicly disclosed) information that the private body originally had when making its bid. Possibly. I don&#8217;t know how tendering works. But from the outside, that&#8217;s how it appears to me. (Maybe there needs to be more <em>transparency</em> about the process?)</p>
<p>Open data is possibly a Big Thing. Who knows? Maybe it isn&#8217;t. Certainly the big consulting firms are calling it as something worth squillionty billionty of pounds. I&#8217;m not sure how they cost it. Maybe I need to dig through the references and footnotes in their reports (Cap Gemini&#8217;s <a href="http://www.uk.capgemini.com/resources/the-open-data-economy-unlocking-economic-value-by-opening-government-and-public-data">Open Data Economy: Unlocking Economic Value by Opening Government and Public Data</a>, Deloitte&#8217;s <a href="http://www.deloitte.com/view/en_GB/uk/market-insights/deloitte-analytics/bfb570a79416b310VgnVCM1000003256f70aRCRD.htm">Open growth: Stimulating demand for open data in the UK</a> or McKinsey&#8217;s <a href="http://www.mckinsey.com/insights/business_technology/open_data_unlocking_innovation_and_performance_with_liquid_information">Open data: Unlocking innovation and performance with liquid information</a>). I don&#8217;t know how much those companies have received in fees for producing those reports, or how much they have received in consultancy fees associated with public open data initiatives &#8211; somehow, that spend data doesn&#8217;t seem to have been curated in a convenient way, or as a #opendatadata bundle? &#8211; but I have to assume they&#8217;re not doing it to fleece the public bodies and tee up benefits for their other private corporate clients.</p>
<p>Reminds me &#8211; I need to read Owen Boswarva&#8217;s <a href="http://mapgubbins.tumblr.com/post/91803140720/who-supports-the-privatisation-of-land-registry-mainly">Who supports the privatisation of Land Registry?</a> and <a href="http://mapgubbins.tumblr.com/post/92541240530/odug-benefits-case-for-open-data-release-of-an">ODUG benefits case for open data release of an authoritative GP dataset</a> again&#8230; And remind myself of <a href="https://www.gov.uk/government/groups/open-data-user-group">who sits on the Open Data User Group (ODUG)</a>, and other UK gov departmental transparency boards&#8230;</p>
<p>And read the FTC&#8217;s report <a>Data Brokers: A Call For Transparency and Accountability</a>&#8230;</p>
<p>Just by the by, one thing I&#8217;ve noticed about a lot of opendata releases is that, along with many other sorts of data, they are most useful when aggregated over time or space, and/or combined with other data sets. Looking at the month on month reports of local spending data from my local council is all very well, but <a href="http://blog.ouseful.info/2013/11/06/local-council-spending-data-time-series-charts/">it gets more interesting when viewed over several months or years</a>. Looking at the month on month reports of local spending data from my local council is all very well, but <a href="http://blog.ouseful.info/2012/05/21/inter-council-payments-in-the-new-look-google-fusion-tables/">it gets more interesting when looking at spend across councils</a>, as for example in the case of <a href="http://blog.ouseful.info/2012/05/26/visualising-spending-flows-to-serco-using-openlylocal-aggregated-spending-data/">looking at spend to particular companies</a>.</p>
<p>Aggregating public data is one of the business models that helps create some of the GDP figure that contributes to the claimed, anticipated squillionty billionty pounds of financial benefit that will arise from open data  &#8211; companies like <a href="http://opencorporates.com">opencorporates</a> aggregating company data, or <a href="http://www.spendnetwork.com/">Spend Network</a> aggregating UK public spending data who hope to start making money selling products off the back of public open data they have curated. Yes &#8211; I know a lot of work goes in to cleaning and normalising that data, and that exploiting the data collection <em>as a whole</em> is what their business models are about &#8211; and why they don&#8217;t offer downloads of their complete datasets, though maybe licenses require they do make links to, or downloads of, the original (&#8220;partial&#8221;) datasets available?</p>
<p>But you know where I think the real value of those companies lies? In being bought out. By Experian, or Acxiom (if there&#8217;s even a hint of personally identifiable data through reverse engineering in the mix), or whoever&#8230; A weak, cheap, cop out business model. Just like this: <a href="http://www.irishtimes.com/business/farmers-up-in-arms-over-potential-misuse-of-data-1.1863181">Farmers up in arms over potential misuse of data</a>. (In case you missed it, <a href="http://www.climate.com">Climate Corporation</a> was one of the <a href="http://www.opendata500.com/us/Climate-Corporation/">OpenData500</a> that aggregated shed loads of open data &#8211; according to Andrew Stott&#8217;s <a href="http://blogs.worldbank.org/opendata/open-data-economic-growth-latest-evidence">Open Data for Economic Growth </a> report for the World Bank,  Climate Corp <em>&#8220;uses 60 years of detailed crop yield data, weather observations from one million locations in the United States and 14 terabytes of soil quality data &#8211; all free from the US Government &#8211; to provide applications that help farmers improve their profits by making better informed operating and financing decisions&#8221;</em>. It was also recently acquired by Monsanto &#8211; Monsanto &#8211; for just under a billion US $. That&#8217;s part of the squillionty billionties I guess. Good ol&#8217; open data. Monsanto.)</p>
<p>Sort of related to this &#8211; that is, companies buying others to asset strip them for their data &#8211; you know all that data of yours locked up in Facebook and Google? Remember MySpace? Remember NNTP? According to the Sophos blog, <a href="http://blog.ouseful.info/2010/02/13/just-because-you-dont-give-your-personal-data-to-google-doesnt-mean-they-cant-acquire-it/">Just Because You Don’t Give Your Personal Data to Google Doesn’t Mean They Can’t Acquire It</a>. Or that someone else might buy it.</p>
<p>And as another aside &#8211; Google &#8211; remember Google? They don&#8217;t really &#8220;read&#8221; your email, at least, people don&#8217;t, they just let algorithms process it so the algorithms can privately just use that data to send you ads, but no-one will ever know what the content of the email was to trigger you getting that ad (&#8216;cos the <a href="http://blog.ouseful.info/2013/09/21/centralising-user-tracking-on-the-web-let-google-track-everyone-for-you/">cookie tracking, cookie matching</a> services can&#8217;t unpick ad bids, ad displays, click thrus, surely, can they?!), well &#8211; maybe there are side effects: <a>Google tips off cops after spotting child abuse images in email</a> (for some reason, after initially being able to read that article, my browser can&#8217;t load it atm. Server fatigue?). Of course, if Google reads your ads for blind business purposes and ad serving is part of that blind process you accept it. But how does the law enforcement <em>&#8216;because we can even though you didn&#8217;t warrant us to?&#8217;</em> angle work? Does the Post Office look inside the envelope? Is surveillance actually part of Google&#8217;s business model?</p>
<p>If you want to up the paranoia stakes, <a href="http://b2fxxx.blogspot.co.uk/2014/07/response-from-mp-on-dripa.html">this</a> (from Ray Corrigan, in particular: <em>&#8220;Without going through the process of matching each government assurance with contradictory evidence, something I suspect would be of little interest, I would like to draw your attention to one important misunderstanding. It seems increasingly to be the belief amongst MPs that blanket data collection and retention is acceptable in law and that the only concern should be the subsequent access to that data. Assertions to this effect are simply wrong.&#8221;</em>) + <a href="http://www.technologyreview.com/view/529396/software-that-can-see-will-change-privacy-forever/">that</a>. Because one day, one day, they may just find your name on an envelope of some sort under a tonne of garbage. <em>Or an algorithm might&#8230;</em> Kid.</p>
<p>But that&#8217;s not what this post is about  &#8211; what this post is about is&#8230; Way back when, so very long ago, not so very long ago, there was a license called GPL. GPL. And GPL was a tainting license. <a>findlaw</a> describes the consequences of reusing GPL licensed code as follows: Kid, <em>&#8216;if a user of GPL code decides to distribute or publish a work that &#8220;in whole or in part contains or is derived from the [open source] or any part thereof,&#8221; it must make the source code available and license the work as a whole at no charge to third parties under the terms of the GPL (thereby allowing further modification and redistribution).</em></p>
<p><em>&#8216;In other words, this can be a trap for the unwary: a company can unwittingly lose valuable rights to its proprietary code.&#8217;</em></p>
<p>Now, friends, GPL scared people so much that another license called LGPL was created, and LGPL allowed you to use LGPL licensed code without fear of tainting your own code with the requirement to open up your own code as GPL would require of it. &#8216;Cos licenses can be used against you.</p>
<p>And when it comes to open data licenses, they seem to be like LGPL. You can take open public data and aggregate it, and combine it, and mix it and mash it and do what you like with it and that&#8217;s fine&#8230; And then someone can come along buy that good work you&#8217;ve done and do what they want with it. Even Monsanto. Even Experian. And that&#8217;s good and right, right? Wrong. The ODUG. Remember the ODUG? The ODUG is the Open Data User Group that lobbies government for what datasets to open up next. And who&#8217;s on the ODUG? Who&#8217;s there, sitting there, on the ODUG bench, right there, right next to you?</p>
<p>Kid&#8230; you wanna be the all-open, all-liberal open data advocate? You wanna see open data used for innovation and exploitation and transparency and all the Good Things (big G, big T) that open data might be used for? Or you wanna sit down on the ODUG bench? With Deloitte, and Experian, and so on&#8230;</p>
<p>And if you think that using a tainting open data license so anyone who uses that data has to share it likewise, aggregated, congregated, conjugated, disaggregated, mixed, matched, joined, summarised or just otherwise and anyways processed, is a Good Thing&#8230;? Then kid&#8230; they&#8217;ll all move away from you on the bench there&#8230;</p>
<p>Because when they come to buy you, they won&#8217;t your data to be tainted in any way that means they&#8217;ll have to give up the commercial advantage they&#8217;ll have from buying up your work on that open data&#8230;</p>
<p>But this post? That&#8217;s not what this post is about. This post about holding companies to account. Open data used to hold companies to account. There&#8217;s a story to be told that&#8217;s not been told about Dr Foster, and open NHS data and fear-mongering and the privatisation of the NHS and that&#8217;s one thing&#8230;</p>
<p>But another thing is how government might use data to help us protect ourselves. Because government can&#8217;t protect us. Government can&#8217;t make companies pay taxes and behave responsibly and not rip off consumers. Government needs our help to do that. But can government help us do that too? Protect and Survive.</p>
<p>There&#8217;s a thing that DECC &#8211; the Department of Energy and Climate Change &#8211; do, and that&#8217;s publish <a href="https://www.gov.uk/government/collections/domestic-energy-prices">statistics about domestic energy price statistics and industrial energy price statistics and road fuel and other petroleum product price statistics</a>, and they&#8217;re all meaningless. Because they bear little resemblance to spot prices paid when consumers pay their domestic energy bills and road fuel and other petroleum product bills.</p>
<p>To find out what those prices are you have to buy the data from someone like Experian, from something like Experian&#8217;s <a href="http://www.catalist.com/Products%20and%20Services/Fuel%20price%20data.aspx">Catalist</a> fuel price data – daily site retail fuel prices &#8211; data product. You may be able to caluclate the DECC statistics from that data (or you may not) but you certainly can&#8217;t go the other way, from the DECC statistics to anything like the Experian data.</p>
<p>But can you go into your local library and ask to look at a copy of the Experian data? A copy of the data that may or may not be used to generate the DECC road fuel and other petroleum product price statistics (how do they generate those statistics anyway? What raw data do they use to generate those statistics?)</p>
<p>Can you imagine ant-eye-ant-eye-consumer data sets being published by your local council or your county council or your national government that can be used to help you hold companies to account and help you tell them that you know they&#8217;re ripping you off and your council off and your government off and that together, you&#8217;re not going to stand for it?</p>
<p>Can you imagine your local council publishing the forecourt fuel prices for one petrol stations, just one petrol station, in your local council area every day? And how about if they do it for two petrol stations, two petrol stations, each day? And if they do it for three forecourts, three, can you imagine if they do it for three petrol stations&#8230;? And can you, can you imagine prices for 50 petrol stations a day being published by your local council, your council helping you inform yourself about how you&#8217;re being manipulated, can you imagine&#8230;? (It may not be so hard &#8211; <a href="http://ratings.food.gov.uk/">food hygiene ratings</a> are published for food retail environments across the England, Northern Ireland and Wales&#8230;</p>
<p>So let&#8217;s hear it for open data, and how open data can be used to hold corporates to account, and how public bodies can use open data to help you make better decisions (which is a good neoliberal position to take and one which the other folk on the bench tell you that that&#8217;s what you want and that and markets work, though they also fall short of telling you that the models say that markets work with full information but you don&#8217;t have the information, and even if you did, you wouldn&#8217;t understand it, because you don&#8217;t really know how to  make a good decision, but at the end of the day you don&#8217;t want a decision, you just want a good service fairly delivered, but they don&#8217;t tell that it&#8217;s all right to just want that&#8230;)</p>
<p>And let&#8217;s hear it for public bodies making data available whether it&#8217;s open or not, making it available by paying for it if they have to and making it available via library services so that we can start using it to start holding companies to account and start helping our public services, and ourselves, protect ourselves from the attacks being mounted on us by companies, and their national government supporters, who take on debt, and who allow them to take on debt, to make dividend payouts but not capital investment and subsidise the temporary driving down of prices (which is NOT a capital investment) through debt subsidised loss leading designed to crush competition in a last man standing contest that will allow monopolistic last man standing price hikes at the end of it&#8230;</p>
<p>And just remember, if there&#8217;s anything you want, you know where you can get it&#8230; At Alice&#8217;s&#8230; or the library&#8230; only they&#8217;re shutting them down, aren&#8217;t they&#8230;? So that leaves what..? Google?</p><br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/ouseful.wordpress.com/12022/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/ouseful.wordpress.com/12022/" /></a> <img alt="" border="0" src="http://pixel.wp.com/b.gif?host=blog.ouseful.info&#038;blog=325417&#038;post=12022&#038;subd=ouseful&#038;ref=&#038;feed=1" width="1" height="1" />]]>
      </content:encoded>
      <wfw:commentRss>http://blog.ouseful.info/2014/08/03/using-open-data-to-hold-companies-to-account/feed/</wfw:commentRss>
      <slash:comments>0</slash:comments>
      <media:content url="http://1.gravatar.com/avatar/abbd9f90565ce9ae4d065d93a81d8c03?s=96&#38;d=http%3A%2F%2F1.gravatar.com%2Favatar%2Fad516503a11cd5ca435acc9bb6523536%3Fs%3D96" medium="image">
        <media:title type="html">Tony Hirst</media:title>
      </media:content>
    </item>
    <item>
      <title>A Conversation With Data &#8211; Isle of Wight Car Parking Meter Transaction Logs</title>
      <link>http://blog.ouseful.info/2014/07/25/a-conversation-with-data-isle-of-wight-car-parking-meter-transaction-logs/</link>
      <comments>http://blog.ouseful.info/2014/07/25/a-conversation-with-data-isle-of-wight-car-parking-meter-transaction-logs/#comments</comments>
      <pubDate>Fri, 25 Jul 2014 18:04:11 +0000</pubDate>
      <dc:creator>
        <![CDATA[Tony Hirst]]>
      </dc:creator>
      <category>
        <![CDATA[Anything you want]]>
      </category>
      <category>
        <![CDATA[Data]]>
      </category>
      <category>
        <![CDATA[onthewight]]>
      </category>
      <category>
        <![CDATA[parkingmeters]]>
      </category>
      <guid isPermaLink="false">http://blog.ouseful.info/?p=12010</guid>
      <description>
        <![CDATA[Killer post title, eh? Some time ago I put in an FOI request to the Isle of Wight Council for the transaction logs from a couple of ticket machines in the car park at Yarmouth. Since then, the Council made some unpopular decisions about car parking charges, got a recall and then in passing made [&#8230;]<img alt="" border="0" src="http://pixel.wp.com/b.gif?host=blog.ouseful.info&#038;blog=325417&#038;post=12010&#038;subd=ouseful&#038;ref=&#038;feed=1" width="1" height="1" />]]></description>
      <content:encoded>
        <![CDATA[<p>Killer post title, eh?</p>
<p>Some time ago I put in an <a href="https://www.whatdotheyknow.com/request/pay_and_display_ticket_machine_l">FOI request</a> to the Isle of Wight Council for the transaction logs from a couple of ticket machines in the car park at Yarmouth. Since then, the Council made some unpopular decisions about car parking charges, got a recall and then in passing made the local BBC news (along with other councils) in respect of the extent of parking charge overpayments&#8230; </p>
<p>Here&#8217;s how hyperlocal news outlet OnTheWight reported the unfolding story&#8230;</p>
<ul>
<li><a href="http://onthewight.com/2014/01/03/car-parking-isle-wight-council-propse-11-new-ways-expensive/">11 new ways the council propose to make car parking more expensive</a></li>
<li><a href="http://onthewight.com/2014/02/17/look-parking-leisure-centre-charges-say-island-conservatives/">Look again at parking and leisure centre charges, say Island Conservatives</a></li>
<li><a href="http://onthewight.com/2014/03/20/council-reveals-new-parking-charges/">Increased car parking charges revealed</a></li>
<li><a href="http://onthewight.com/2014/05/29/legal-challenge-parking-increases-looms/">Council could face legal action over car parking increases</a></li>
<li><a href="http://onthewight.com/2014/06/04/council-gives-view-legal-use-car-parking-income/">Council gives their view on the legal uses of car parking income</a></li>
<li><a href="http://onthewight.com/2014/06/24/council-claim-dont-know-many-people-wrote-parking-changes/">Council claim they don’t yet know how many people wrote to them about parking changes</a></li>
<li><a href="http://onthewight.com/2014/07/08/parking-charges-voted-rip-free-parking-isle-wight/">Executive vote: Free parking in 24 car parks goes, including Appley and Puckpool and parking charges up</a></li>
<li><a href="http://onthewight.com/2014/07/15/councillors-call-decision-parking-changes/">Councillors ‘call-in’ decision on parking changes</a></li>
<li><a href="http://onthewight.com/2014/07/17/decision-changes-parking-charges-scrutinised/">Date set for scrutiny of changes to parking charges</a></li>
<li><a href="http://onthewight.com/2014/07/23/scrutineers-consider-decision-changes-parking-charges-tonight/">Follow live coverage of parking changes being scrutinised (Updated)</a> (includes a copy of the call-in notice)</li>
<li><a href="http://onthewight.com/2014/07/24/isle-wight-parkers-paid-186706-35-parking-2011-13/">Isle of Wight car parkers overpaid £186,706.35 between 2011-13</a></li>
</ul>
<p>I really missed a trick not getting involved in this process &#8211; because there is, or could be, a significant data element to it. And I had a sample of data that I could have doodled with, and then gone for the whole data set.</p>
<p>Anyway, I finally made a start on looking at the data I did have with a view to seeing what stories or insight we might be able to pull from it &#8211; the first sketch of my conversation with the data is here: <a href="http://nbviewer.ipython.org/gist/psychemedia/9a4526b65990d96816cd">A Conversation With Data &#8211; Car Parking Meter Data</a>.</p>
<p>It&#8217;s not just the parking meter data that can be brought to bear in this case &#8211; there&#8217;s another set of relevant data too, and I also had a sample of that: <a href="https://www.whatdotheyknow.com/request/traffic_penalty_charge_notices#incoming-400444">traffic penalty charge notices</a> (i.e. traffic warden ticket issuances&#8230;)</p>
<p>With a bit of luck, I&#8217;ll have a go at a quick conversation with <em>that</em> data over the next week or so&#8230; Then maybe put in a full set of FOI requests for data from all the Council operated ticket machines, and all the penalty notices issued, for a couple of financial years.</p>
<p>Several things I think might be interesting to look at Island-wide:</p>
<ul>
<li>in much the same was as Tube platforms suffer from loading problems, where folk surge around one entrance or another, do car parks &#8220;fill up&#8221; in some sort of order, eg within a car park (one meter lags the other in terms of tickets issued) or one car park lags another overall;</li>
<li>do different car parks have a different balance of ticket types issued (are some used for long stay, others for short stay?) and does this change according to what day of the week it is?</li>
<li>how does the issuance of traffic penalty charge notices compare with the sorts of parking meter tickets issued?</li>
<li>from the timestamps of when traffic penalty charge notices tickets are issued, can we work out the rounds of different traffic warden patrols?</li>
</ul>
<p>The last one might be a little bit cheeky &#8211; just like <a href="http://www.oxfordmail.co.uk/news/10104624.Police_try_to_prevent_speed_trap__flashers_/?ref=rl">you aren&#8217;t supposed to share information about the mobile speed traps</a>, perhaps you also aren&#8217;t supposed to share information that there&#8217;s a traffic warden doing the rounds&#8230;?!</p><br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/ouseful.wordpress.com/12010/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/ouseful.wordpress.com/12010/" /></a> <img alt="" border="0" src="http://pixel.wp.com/b.gif?host=blog.ouseful.info&#038;blog=325417&#038;post=12010&#038;subd=ouseful&#038;ref=&#038;feed=1" width="1" height="1" />]]>
      </content:encoded>
      <wfw:commentRss>http://blog.ouseful.info/2014/07/25/a-conversation-with-data-isle-of-wight-car-parking-meter-transaction-logs/feed/</wfw:commentRss>
      <slash:comments>0</slash:comments>
      <media:content url="http://1.gravatar.com/avatar/abbd9f90565ce9ae4d065d93a81d8c03?s=96&#38;d=http%3A%2F%2F1.gravatar.com%2Favatar%2Fad516503a11cd5ca435acc9bb6523536%3Fs%3D96" medium="image">
        <media:title type="html">Tony Hirst</media:title>
      </media:content>
    </item>
    <item>
      <title>MOOCs as Partworks</title>
      <link>http://blog.ouseful.info/2014/07/22/moocs-as-partworks/</link>
      <comments>http://blog.ouseful.info/2014/07/22/moocs-as-partworks/#comments</comments>
      <pubDate>Tue, 22 Jul 2014 15:26:14 +0000</pubDate>
      <dc:creator>
        <![CDATA[Tony Hirst]]>
      </dc:creator>
      <category>
        <![CDATA[Anything you want]]>
      </category>
      <category>
        <![CDATA[MOOCs]]>
      </category>
      <guid isPermaLink="false">http://blog.ouseful.info/?p=11989</guid>
      <description>
        <![CDATA[A couple of recent posts crossed my feeds recently mooching around the idea that doing is MOOC is Like Reading a Newspaper; or not: MOOC Completion Rates DO Matter. First up, Downes suggests that: The traditional course is designed like a book &#8211; it is intended to run in a sequence, the latter bits build [&#8230;]<img alt="" border="0" src="http://pixel.wp.com/b.gif?host=blog.ouseful.info&#038;blog=325417&#038;post=11989&#038;subd=ouseful&#038;ref=&#038;feed=1" width="1" height="1" />]]></description>
      <content:encoded>
        <![CDATA[<p>A couple of recent posts crossed my feeds recently mooching around the idea that doing is MOOC is <a href="http://halfanhour.blogspot.co.uk/2014/03/like-reading-newspaper.html">Like Reading a Newspaper</a>; or not:  <a href="http://nogoodreason.typepad.co.uk/no_good_reason/2014/07/mooc-completion-rates-do-matter.html">MOOC Completion Rates DO Matter</a>.</p>
<p>First up, Downes suggests that:</p>
<blockquote><p>The traditional course is designed like a book &#8211; it is intended to run in a sequence, the latter bits build on the first bits, and if you start a book and abandon it p[art way through there is a real sense in which you can say the book has failed, because the whole point of a book is to read it from beginning to end.</p>
<p>But our MOOCs are not designed like that. Though they have a beginning and an end and a range of topics in between, they&#8217;re not designed to be consumed in a linear fashion the way a book it. Rather, they&#8217;re much more like a magazine or a newspaper (or an atlas or a city map or a phone book). The idea is that there&#8217;s probably more content than you want, and that you&#8217;re supposed to pick and choose from the items, selecting those that are useful and relevant to your present purpose.</p>
<p>And so here&#8217;s the response to completion rates: nobody ever complained that newspapers have low completion rates. And yet no doubt they do,. Probably far below the &#8216;abysmal&#8217; MOOC completion rates (especially if you include real estate listings and classified ads). People don&#8217;t read a newspaper to complete it, they read a newspaper to find out what&#8217;s important.</p></blockquote>
<p>Martin (Weller) responds:</p>
<blockquote><p> Stephen Downes has a nice analogy, (which he blogged at my request, thankyou Stephen) in that it&#8217;s like a newspaper, no-one drops out of a newspaper, they just take what they want. This has become repeated rather like a statement of fact now. I think Stephen&#8217;s analogy is very powerful, but it is really a statement of intent. If you design MOOCs in a certain way, then the MOOC experience could be like reading a newspaper. The problem is 95% of MOOCs aren&#8217;t designed that way. And even for the ones that are, completion rates are still an issue.</p>
<p>Here&#8217;s why they&#8217;re an issue. MOOCs are nearly always designed on a week by week basis (which would be like designing a newspaper where you had to read a certain section by a certain time). I&#8217;ve blogged about this before, but from Katy Jordan&#8217;s data we reckon 45% of those who sign up, never turn up or do anything. It&#8217;s hard to argue that they&#8217;ve had a meaningful learning experience in any way. If we register those who have done anything at all, eg just opened a page, then by the end of week 2 we&#8217;re down to about 35% of initial registrations. And by week 3 or 4 it&#8217;s plateauing near 10%. The data suggests that people are definitely not treating it like a newspaper. In Japan some research was done on what sections of newspapers people read.</p></blockquote>
<p>He goes on:</p>
<blockquote><p>&#8230; Most MOOCs are about 6-7 weeks long, so 90% of your registered learners are never even looking at 50% of your content. That must raise the question of why are you including it in the first place? If a subject requires a longer take at it, beyond 3 weeks say, then MOOCs really may not be a very good approach to it. There is a hard, economic perspective here, it costs money to make and run MOOCs, and people will have to ask if the small completion rates are the most effective way to get people to learn that subject. You might be better off creating more stand alone OERs, or putting money into better supported outreach programmes where you can really help people stay with the course. Or maybe you will actually design your MOOC to be like a newspaper.</p></blockquote>
<p>So&#8230;</p>
<p>I buy three newspapers a week &#8211; the Isle of Wight County Press (to get a feel for what&#8217;s happened and is about to happen locally, as well as seeing who&#8217;s currently recruiting), the Guardian on a Saturday (see what news stories made it as far as Saturday comment, do the Japanese number puzzles, check out the book reviews, maybe read the odd long form interview and check a recipe or two), and the Observer on a Sunday (read colleagues&#8217; columns, longer form articles by journalists I know or have met, check out any F1 sports news that made it into that paper, book reviews, columns, and Killer again&#8230;).</p>
<p>So I skim bits, have old faithfuls I read religiously, and occasionally follow through on a long form article that was maybe advertised on the cover and I might have missed otherwise.</p>
<p>Newspapers are organised in a particular way, and that lets me quickly access the bits I know I want to access, and throw the rest straight onto the animal bedding pile, often unread and unopened.</p>
<p>So MOOCs are not really like that, at least, not for me.</p>
<p>For me MOOCs are freebie papers I&#8217;ve picked up and then thrown, unread, onto the animal bedding pile. For me.</p>
<p>What I can see, though, as MOOCs as partworks. Partworks are those titles you see week on week in the local newsagent with a new bit on the cover that, if collected over weeks and months and assembled in the right way, result in a flimsy plastic model you&#8217;ve assembled yourself with an effective cost price running into hundreds of pounds. </p>
<p>[Retro: seems I floated the MOOC as partwork idea before - <a href="http://blog.ouseful.info/2013/04/04/online-courses-or-long-form-journalism-communicating-how-the-world-works/">Online Courses or Long Form Journalism? Communicating How the World Works…</a> - and no-one really bit then either...]</p>
<p>In the UK, there are several notable publishers of partwork titles, including for example <a href="http://www.hachettepartworks.com/">Hachette</a>, <a href="http://shop.deagostini.co.uk/about-us">De Agostini</a>,<a href="http://eaglemoss.com/default.aspx">Eaglemoss</a>. Check out their homepages &#8211; then check out the homepages of a few MOOC providers. (Note to self &#8211; see if any folk working in marketing of MOOC platform providers came from a partwork publishing background.)</p>
<p>Here&#8217;s a riff reworking the <a href="http://en.wikipedia.org/wiki/Partwork">Wikipedia partwork page</a>:</p>
<p><em>A <s>partwork</s><tt>MOOC</tt> is <s>a written publication</s><tt>an online course</tt> released as a series of planned magazine-like <s>issues</s><tt>lessons</tt> over a period of time. <s>Issues</s><tt>Lessons</tt> are typically released on a weekly, fortnightly or monthly basis, and often a completed set is designed to form a <s>reference work on</s><tt>complete course in</tt> a particular topic.</em></p>
<p><em><s>Partwork series</s><tt>MOOCs</tt> run for a determined length and have a finite life. Generally, <s>partworks</s><tt>MOOCs</tt> cover specific areas of interest, such as <s>sports, hobbies, or children&#8217;s interest and stories such as PC Ace and the successful The Ancestral Trail series by Marshall Cavendish Ltd</s><tt>random university module subjects, particularly ones that tie in to the telly or hyped areas of pseudo-academic interest</tt>. They are generally <s>sold at newsagents and are mostly supported by massive television advertising campaigns for the launch</s><tt>hosted on MOOC platforms because exploiting user data and optimising user journeys through learning content is something universities don't really understand and avoid trying to do</tt>. In the United Kingdom, <s>partworks</s><tt>MOOCs</tt> are <s>usually launched by heavy television advertising each January</s><tt>mentioned occasionally in the press, often following a PR campaign by the UK MOOC platfrom, FutureLearn</tt>.</em></p>
<p><em><s>Partworks</s><tt>MOOCs</tt> often include cover-mounted items with each issue that build into a complete set over time. For example, a <s>partwork about art</s><tt>MOOC</tt> might include a small number of <s>paints or pencils that build into a complete art-set</s><tt>so-called "badges" that can be put into an online "backpack" to show off to your friends, family, and LinkedIn trawlers</tt><s>; a partwork about dinosaurs might include a few replica bones that build a complete model skeleton at the end of the series; a partwork about films may include a DVD with each issue. In Europe, partworks with collectable models are extremely popular; there are a number of different publications that come with character figurines or diecast model vehicles, for example: The James Bond Car Collection</s>.</em></p>
<p><em>In addition, completed <s>partworks</s><tt>MOOCs</tt> have sometimes been used <tt>as the basis for receiving a non-academic credit bearing course completion certificate, or</tt> to create <s>case-bound reference works and encyclopedias</s><tt>a basis for a piece of semi-formal assessment and recognition</tt>. An example is <s>the multi-volume Illustrated Science and Invention Encyclopedia which was created with material first published in the How It Works partwork</s><tt>NEED TO FIND A GOOD EXAMPLE</tt>.</em></p>
<p><em>In the UK, <s>partworks</s><tt>MOOCs</tt> are the <s>fourth-best selling magazine sector, after TV listing guides, women&#8217;s weeklies and women&#8217;s monthlies</s><tt>NEED SOME NUMBERS HERE*...</tt>. A common inducement is <s>a heavy discount for the first one or two issues</s><tt>??HOW DO MOOCs SELL GET SOLD?</tt>. The same <s>series</s><tt>MOOC</tt> can be sold worldwide in different languages and even in different variations.</em></p>
<p>* Possibly useful starting point? <a href="http://news.bbc.co.uk/1/hi/magazine/6287773.stm">BBC News Magazine: Let&#8217;s get this partwork started</a></p>
<p>The Wikipedia page goes on to talk about serialisation (ah, the good old days when I still had hoped for feeds and syndication&#8230; eg <a href="http://ouseful.open.ac.uk/blogarchive/008866.html">OpenLearn Daily Learning Chunks via RSS</a> and then <a href="http://blog.ouseful.info/2009/01/06/serialised-openlearn-daily-rss-feeds-via-wordpress/">Serialised OpenLearn Daily RSS Feeds via WordPress</a>) and the Pecia System (new to me), which looks like it could provide an interesting starting point on a model of peer-co-created learning, or somesuch. There&#8217;s probably a section on it in this year&#8217;s Innovating Pedagogy report. Or maybe there isn&#8217;t?!;-)</p>
<p>Sort of related but also not, this article from <em>icrossing</em> on <a href="http://connect.icrossing.co.uk/subscribe-is-the-new-shop-are-subscription-business-models-taking-over_12072">‘Subscribe is the new shop.’ – Are subscription business models taking over?</a> and John Naughton&#8217;s column last week on the (as then, just leaked) Kindle subscription model  &#8211; <a href="http://www.theguardian.com/technology/2014/jul/20/kindle-unlimited-amazon-leaked-netflix-for-books-alarm-publishers">Kindle Unlimited: it&#8217;s the end of losing yourself in a good book</a>, I&#8217;m reminded of <a href="http://blog.ouseful.info/2010/11/25/subscription-models-for-lifelong-students/">Subscription Models for Lifelong Students</a> and <a href="http://blog.ouseful.info/2011/05/12/graduate-with-who-whom-exactly/">Graduate With Who (Whom?!;-), Exactly…?</a>, which several people argued against and which I never really tried to defend, though I can&#8217;t remember what the arguments were, and I never really tried to build a case with numbers in it to see whether or not it might make sense. (Because sometimes you think the numbers <em>should</em> work out in your favour, but then they don&#8217;t&#8230; as in this example: <a href="http://perfdynamics.blogspot.co.uk/2014/07/restaurant-performance-sunk-by-selfies.html">Restaurant Performance Sunk by Selfies</a> [via RBloggers].)</p>
<p>Erm, oh yes &#8211; back to the MOOCs.. and the partworks models. Martin mentioned the economics &#8211; just thinking about the partwork model (pun intended, or maybe not) here, how are parts costed? Maybe an expensive loss leader part in week 1, then cheap parts for months, then the expensive parts at the end when only two people still want them? How will print on demand affect partworks (newsagent has a partwork printer round the back to print of the bits that are needed for whatever magazines are sold that week?) And how do the partwork costing models then translate to MOOC production and presentation models?</p>
<p>Big expensively produced materials in front loaded weeks, then maybe move to smaller presentation methods, get the forums working a little better with smaller, more engaged groups? How about the cMOOC ideas &#8211; up front in early weeks, or pushed back to later weeks, where different motivations, skills, interest and engagement models play out.</p>
<p>MOOCs are newspapers? Nah&#8230; MOOCs as partwork &#8211; that works better as a model for me. (You can always buy a partwork mid-way through because you are interested in that week&#8217;s content, or the content covered by the magazine generally, not because you are interested in the plastic model or badge.</p>
<p>Thinks: hmm, partworks come in at least two forms, don&#8217;t they &#8211; one to get pieces to build a big model of a boat or a steam train or whatever. The other where you get a different superhero figurine each week and the aim it attract the completionist. Which isn&#8217;t to say that part 37 might not be stupidly popular because it has a figure that is just generally of interest, ex- of being part of a set?</p><br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/ouseful.wordpress.com/11989/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/ouseful.wordpress.com/11989/" /></a> <img alt="" border="0" src="http://pixel.wp.com/b.gif?host=blog.ouseful.info&#038;blog=325417&#038;post=11989&#038;subd=ouseful&#038;ref=&#038;feed=1" width="1" height="1" />]]>
      </content:encoded>
      <wfw:commentRss>http://blog.ouseful.info/2014/07/22/moocs-as-partworks/feed/</wfw:commentRss>
      <slash:comments>2</slash:comments>
      <media:content url="http://1.gravatar.com/avatar/abbd9f90565ce9ae4d065d93a81d8c03?s=96&#38;d=http%3A%2F%2F1.gravatar.com%2Favatar%2Fad516503a11cd5ca435acc9bb6523536%3Fs%3D96" medium="image">
        <media:title type="html">Tony Hirst</media:title>
      </media:content>
    </item>
    <item>
      <title>F1 Timing Screen as a Spreadsheet?</title>
      <link>http://blog.ouseful.info/2014/07/17/f1-timing-screen-as-a-spreadsheet/</link>
      <comments>http://blog.ouseful.info/2014/07/17/f1-timing-screen-as-a-spreadsheet/#comments</comments>
      <pubDate>Thu, 17 Jul 2014 16:38:33 +0000</pubDate>
      <dc:creator>
        <![CDATA[Tony Hirst]]>
      </dc:creator>
      <category>
        <![CDATA[Anything you want]]>
      </category>
      <category>
        <![CDATA[f1data]]>
      </category>
      <category>
        <![CDATA[f1datajunkie]]>
      </category>
      <category>
        <![CDATA[tata]]>
      </category>
      <guid isPermaLink="false">http://blog.ouseful.info/?p=11987</guid>
      <description>
        <![CDATA[One of the comment themes I&#8217;ve noticed around the first Challenge in the Tata F1 Connectivity Innovation Prize, a challenge to rethink what&#8217;s possible around the timing screen given only the data in the real time timing feed, is that the non-programmers don&#8217;t get to play. I don&#8217;t think that&#8217;s true &#8211; the challenge seems [&#8230;]<img alt="" border="0" src="http://pixel.wp.com/b.gif?host=blog.ouseful.info&#038;blog=325417&#038;post=11987&#038;subd=ouseful&#038;ref=&#038;feed=1" width="1" height="1" />]]></description>
      <content:encoded>
        <![CDATA[<p>One of the comment themes I&#8217;ve noticed around the first Challenge in the <a href="https://prize.tatacommunications.com/home">Tata F1 Connectivity Innovation Prize</a>, a challenge to rethink what&#8217;s possible around the timing screen given only the data in the real time timing feed, is that the non-programmers don&#8217;t get to play. I don&#8217;t think that&#8217;s true &#8211; the challenge seems to be open to ideas as well as practical demonstrations, but it got me thinking about what technical ways in might be to non-programmers who wouldn&#8217;t know where to start when it came to working with the timing stream messages.</p>
<p>The answer is surely the timing screen itself&#8230; One of the issues I still haven&#8217;t fully resolved is a proven way of getting useful information events from the timing feed &#8211; it updates the timing screen on a cell by cell basis, so we have to finesse the way we associate new laptimes or sector times with a particular driver, bearing in mind cells update one at a time, in a potentially arbitrary order, and with potentially different timestamps.</p>
<p><a href="http://f1-innovation-prize.s3.amazonaws.com/challenge_packs/The%20F1%20Connectivity%20Innovation%20Prize%20%E2%80%93%20Challenge%201%20Brief.pdf"><img src="http://ouseful.files.wordpress.com/2014/07/f1-innovation-prize_s3_amazonaws_com_challenge_packs_the_f1_connectivity_innovation_prize_e28093_challenge_1_brief_pdf.png?w=700&#038;h=573" alt="f1-innovation-prize_s3_amazonaws_com_challenge_packs_The_F1_Connectivity_Innovation_Prize_–_Challenge_1_Brief_pdf" width="700" height="573" class="alignnone size-full wp-image-11896" /></a></p>
<p>So how about if we work with a &#8220;live information model&#8221; by creating a copy of an example timing screen in a spreadsheet. If we know how, we might be able to parse the real data stream to directly update the appropriate cells, but that&#8217;s largely by the by. At least we have something we can work work to start playing with the timing screen in terms of a literal reimagining of it. So what can we do if we put the data from an example timing screen into a spreadsheet?</p>
<p>If we create a new worksheet, we can reference the cells in the &#8220;original&#8221; timing sheet and pull values over. The timing feed updates cells on a cell by cell basis, but spreadsheets are really good at rippling through changes from one or more cells which are themselves reference by one or more others.</p>
<p>The first thing we might do is just transform the shape of the timing screen. For example, we can take the cells in a column relating to sector 1 times and put them into a row. </p>
<p>The second thing we might do is start to think about some sums. For example, we might find the difference between each of those sector times and (for practice and qualifying sessions at least) the best sector time recorded in that session.</p>
<p>The third thing we might do is to use a calculated value as the basis for a custom cell format that colours the cell according to the delta from the best session time.</p>
<p>Simple, but a start.</p>
<p>I&#8217;ve not really tried to push this idea very far &#8211; I&#8217;m not much of a spreadsheet jockey &#8211; but I&#8217;d be interested to know how folk who are might be able to push this idea&#8230;</p>
<p>If you need example data, there&#8217;s some on the F1 site &#8211; <a href="http://www.formula1.com/results/season/2014/919/">f1.com &#8211; results for Spanish Grand Prix, 2014</a> and more on <a href="http://ergast.com/mrd/">Ergast Developer API</a>.</p>
<p>PS FWIW, my entry to the competition is here: <a href="https://github.com/psychemedia/tata-f1">#f1datajunkie challenge 1 entry</a>. It&#8217;s perhaps a little off-brief, but I&#8217;ve been meaning to do this sort of summary for some time, and this was a good starting point. If I get a chance, I&#8217;ll have a go a getting the parsers to work properly properly!</p><br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/ouseful.wordpress.com/11987/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/ouseful.wordpress.com/11987/" /></a> <img alt="" border="0" src="http://pixel.wp.com/b.gif?host=blog.ouseful.info&#038;blog=325417&#038;post=11987&#038;subd=ouseful&#038;ref=&#038;feed=1" width="1" height="1" />]]>
      </content:encoded>
      <wfw:commentRss>http://blog.ouseful.info/2014/07/17/f1-timing-screen-as-a-spreadsheet/feed/</wfw:commentRss>
      <slash:comments>0</slash:comments>
      <media:content url="http://1.gravatar.com/avatar/abbd9f90565ce9ae4d065d93a81d8c03?s=96&#38;d=http%3A%2F%2F1.gravatar.com%2Favatar%2Fad516503a11cd5ca435acc9bb6523536%3Fs%3D96" medium="image">
        <media:title type="html">Tony Hirst</media:title>
      </media:content>
      <media:content url="http://ouseful.files.wordpress.com/2014/07/f1-innovation-prize_s3_amazonaws_com_challenge_packs_the_f1_connectivity_innovation_prize_e28093_challenge_1_brief_pdf.png" medium="image">
        <media:title type="html">f1-innovation-prize_s3_amazonaws_com_challenge_packs_The_F1_Connectivity_Innovation_Prize_–_Challenge_1_Brief_pdf</media:title>
      </media:content>
    </item>
    <item>
      <title>DRIP&#8230; Letter to My MP</title>
      <link>http://blog.ouseful.info/2014/07/14/drip-letter-to-my-mp/</link>
      <comments>http://blog.ouseful.info/2014/07/14/drip-letter-to-my-mp/#comments</comments>
      <pubDate>Mon, 14 Jul 2014 15:07:29 +0000</pubDate>
      <dc:creator>
        <![CDATA[Tony Hirst]]>
      </dc:creator>
      <category>
        <![CDATA[Uncategorized]]>
      </category>
      <guid isPermaLink="false">http://blog.ouseful.info/2014/07/14/drip-letter-to-my-mp/</guid>
      <description>
        <![CDATA[Dear Andrew Turner, I am writing to you today to express concern about the way in which the Data Retention and Investigatory Powers Bill [1] is being rushed through Parliament without appropriate time allowed for scrutiny of the Bill or members to elicit expert and informed opinion about what the consequences of the Bill might [&#8230;]<img alt="" border="0" src="http://pixel.wp.com/b.gif?host=blog.ouseful.info&#038;blog=325417&#038;post=11975&#038;subd=ouseful&#038;ref=&#038;feed=1" width="1" height="1" />]]></description>
      <content:encoded>
        <![CDATA[<blockquote><p>Dear Andrew Turner,</p>
<p>I am writing to you today to express concern about the way in which the Data Retention and Investigatory Powers Bill [1] is being rushed through Parliament without appropriate time allowed for scrutiny of the Bill or members to elicit expert and informed opinion about what the consequences of the Bill might be.</p>
<p>I do not know if you have read the Bill [1], the supporting documentation (explanatory notes and impact  assessment) [2], or the ECJ ruling [3] that prompted the emergency legislation, or how you intend to vote in the matter of the Bill. If you have not read the supporting documents, but you do intend to vote in support of the Bill, I would like to ask by what rationale you came to that decision? Because it would not be in my name as a constituent and you would not have my support in the matter.</p>
<p>If you do not have time to read through the official documents, may I ask what other sources of information you turned to in forming your opinion. For your information,  [4] considers some of the issues raised in the impact assessment document.</p>
<p>Informed opinion appears to differ in several important respects in the extent to which it believes the Bill may introduce new or extended powers or extend the scope of regulation, rather than just securing previous legislation, from the public statement that introduced the bill, and the  Home Secretary&#8217;s comments in the HoC Home Affairs Committee today.  This suggests that more time for considering this Bill is required &#8211; if the House is not clear about what the legislation says, it should not pass it. If it is clear, how does it counter the claims raised by the informed dissenters?</p>
<p>As I have written to you previously, I would also like to express concern once again about the cavalier and ill-informed way in which the Government introduce and attempt to pass legislation particularly in respect of &#8220;digital&#8221; matters.</p>
<p>Yours sincerely,</p>
<p>Tony Hirst</p>
<p>[1]</p>
<p><a href="https://www.gov.uk/government/publications/the-data-retention-and-investigatory-powers-bill" rel="nofollow">https://www.gov.uk/government/publications/the-data-retention-and-investigatory-powers-bill</a></p>
<p>[2]</p>
<p><a href="http://www.parliament.uk/business/publications/business-papers/commons/deposited-papers/?fd=2014-07-09&#038;td=2014-07-31&#038;search_term=Home+Office#toggle-1053" rel="nofollow">http://www.parliament.uk/business/publications/business-papers/commons/deposited-papers/?fd=2014-07-09&#038;td=2014-07-31&#038;search_term=Home+Office#toggle-1053</a></p>
<p>[3]</p>
<p><a href="http://curia.europa.eu/juris/document/document_print.jsf?doclang=EN&#038;text=&#038;pageIndex=0&#038;part=1&#038;mode=req&#038;docid=150642&#038;occ=first&#038;dir=&#038;cid=731497" rel="nofollow">http://curia.europa.eu/juris/document/document_print.jsf?doclang=EN&#038;text=&#038;pageIndex=0&#038;part=1&#038;mode=req&#038;docid=150642&#038;occ=first&#038;dir=&#038;cid=731497</a></p>
<p>[4]</p>
<p><a href="http://cyberleagle.blogspot.co.uk/2014/07/dissecting-emergency-data-retention-and.html" rel="nofollow">http://cyberleagle.blogspot.co.uk/2014/07/dissecting-emergency-data-retention-and.html</a></p></blockquote><br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/ouseful.wordpress.com/11975/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/ouseful.wordpress.com/11975/" /></a> <img alt="" border="0" src="http://pixel.wp.com/b.gif?host=blog.ouseful.info&#038;blog=325417&#038;post=11975&#038;subd=ouseful&#038;ref=&#038;feed=1" width="1" height="1" />]]>
      </content:encoded>
      <wfw:commentRss>http://blog.ouseful.info/2014/07/14/drip-letter-to-my-mp/feed/</wfw:commentRss>
      <slash:comments>0</slash:comments>
      <media:content url="http://1.gravatar.com/avatar/abbd9f90565ce9ae4d065d93a81d8c03?s=96&#38;d=http%3A%2F%2F1.gravatar.com%2Favatar%2Fad516503a11cd5ca435acc9bb6523536%3Fs%3D96" medium="image">
        <media:title type="html">Tony Hirst</media:title>
      </media:content>
    </item>
    <item>
      <title>Tracking Anonymous Wikipedia Edits From Specific IP Ranges</title>
      <link>http://blog.ouseful.info/2014/07/11/tracking-anonymous-wikipedia-edits-from-specific-ip-ranges/</link>
      <comments>http://blog.ouseful.info/2014/07/11/tracking-anonymous-wikipedia-edits-from-specific-ip-ranges/#comments</comments>
      <pubDate>Fri, 11 Jul 2014 07:45:35 +0000</pubDate>
      <dc:creator>
        <![CDATA[Tony Hirst]]>
      </dc:creator>
      <category>
        <![CDATA[Anything you want]]>
      </category>
      <guid isPermaLink="false">http://blog.ouseful.info/?p=11941</guid>
      <description>
        <![CDATA[Via @davewiner&#8217;s blog, I spotted a link to @congressedits, &#8220;a bot that tweets anonymous Wikipedia edits that are made from IP addresses in the US Congress&#8221;. (For more info, see why @congressedits?, /via @ostephens.) I didn&#8217;t follow the link to the home page for that account (doh!), but in response to a question about whether [&#8230;]<img alt="" border="0" src="http://pixel.wp.com/b.gif?host=blog.ouseful.info&#038;blog=325417&#038;post=11941&#038;subd=ouseful&#038;ref=&#038;feed=1" width="1" height="1" />]]></description>
      <content:encoded>
        <![CDATA[<p>Via @davewiner&#8217;s blog, I spotted a link to @congressedits, &#8220;a bot that tweets anonymous Wikipedia edits that are made from IP addresses in the US Congress&#8221;. (For more info, see <a href="http://inkdroid.org/journal/2014/07/10/why-congressedits/">why @congressedits?</a>, /via @ostephens.) I didn&#8217;t follow the link to the home page for that account (doh!), but in response to a question about whether white label code was available, @superglaze pointed me to <a href="https://github.com/edsu/anon">https://github.com/edsu/anon</a>, a script that &#8220;will watch Wikipedia for edits from a set of named IP ranges and will tweet when it notices one&#8221;.</p>
<p>It turns out the script was inspired by @parliamentedits, a bot built by @tomscott that &#8220;tracks edits to Wikipedia made from Parliamentary IP addresses&#8221; built using IFTT and possibly a list of IP ranges operated by the House of Commons gleaned from <a href="https://www.whatdotheyknow.com/request/parliaments_access_to_the_intern?unfold=1">this FOI request</a>?</p>
<p>Nice&#8230;</p>
<p>My immediate thought was set up something to track edits made to Wikipedia from OU IP addresses, then idly wondered if set of feeds for tracking edits from HEIs in general might also be useful (something to add to the <a href="http://observatory.data.ac.uk/">UK University Web Observatory</a> for example?)</p>
<p>To the extent that Wikipedia represents an authoritative source of information, for some definition of authoritative(?!), it could be interesting to track the &#8220;impact&#8221; of our <a href="http://blog.ouseful.info/2014/07/10/foolish-is-as-foolish-does-fragments/">foolish universities</a> in terms of contributing to the sum of of human knowledge as represented by Wikipedia.</p>
<p>It&#8217;d also be interesting to track the sorts of edits made from anonymous and named editors from HEI IP ranges. I wonder what classes they may fall into?</p>
<ol>
<li>edits from the marketing and comms folk?</li>
<li>ego and peer ego edits, eg from academics keeping the web pages of other academics in their field up to date?</li>
<li>research topic edits &#8211; academics maintaining pages that relate to their research areas or areas of scholarly interest?</li>
<li>teaching topic edits &#8211; academics maintaining pages that relate to their teaching activities?</li>
<li>library edits &#8211; edits made from the library?</li>
<li>student edits &#8211; edits made by students as part of a course?</li>
<li>&#8220;personal&#8221; edits &#8211; edits made by folk who class themselves and Wikimedians in general and just happen to make edits while they are on an HEI network?</li>
</ol>
<p>My second thought was to wonder to what extent might <em>news and media</em> organisations be maintaining &#8211; or tweaking &#8211; Wikipedia pages? The BBC, for example, who have made widespread use of Wikipedia in their <a href="http://www.w3.org/2001/sw/sweo/public/UseCases/BBC/">Linked Data driven music and wildlife pages</a>.</p>
<p><em>Hmmm&#8230; news.. reminds me: wasn&#8217;t a civil servant who made abusive edits to a Wikipedia page sacked recently? Ah, yes: <a href="http://www.telegraph.co.uk/sport/football/teams/liverpool/10904540/Civil-servant-fired-after-Telegraph-investigation-into-Hillsborough-Wikipedia-slurs.html">Civil servant fired after Telegraph investigation into Hillsborough Wikipedia slurs</a>, as my OU colleague <a href="http://theconversation.com/wikipedia-editors-never-walk-alone-hillsborough-changes-can-be-traced-25952">Andrew Smith suggested might happen</a>.</em></p>
<p>Or how about other cultural organisations &#8211; museums and galleries for example?</p>
<p>Or charities?</p>
<p>Or particular brands? Hmm&#8230;</p>
<p>So I wonder: could we try to identify areas of expertise on, or attempted/potential influence over, particular topics by doing reverse IP lookups from pages focussed on those topics? This sort of mapping activity pivots the idea of <a href="http://blog.ouseful.info/2012/07/03/visualising-related-entries-in-wikipedia-using-gephi/">visualising related entries in Wikipedia</a> to map IP ranges, and perhaps from that locations and individuals associated with maintaining a set of resources around a particular topic area (cf. <a href="http://blog.ouseful.info/2011/01/08/visualising-delicious-tag-communities-using-gephi/">Visualising Delicious Tag Communities</a>).</p>
<p>I think I started looking at how we might start to map IP ranges for organisations once&#8230;.? Erm&#8230; maybe not, actually: it was <a href="http://blog.ouseful.info/2012/09/07/fragment-pondering-mapping-the-pearson-network/">looking up domains a company owned from its nameservers</a>.</p>
<p>Hmm.. thinks&#8230; webstats show IP ranges of incoming requests &#8211; can we create maps from those? In fact, are there maps/indexes that give IP ranges for eg companies or universities?</p>
<p>I&#8217;m rambling again&#8230;</p>
<p>PS Related: <a href="http://blog.ouseful.info/2013/02/27/repository-googalytics-visits-from-heis/">Repository Googalytics – Visits from HEIs</a> which briefly reviews the idea of tracking visits to HEI repositories from other HEIs&#8230;</p><br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/ouseful.wordpress.com/11941/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/ouseful.wordpress.com/11941/" /></a> <img alt="" border="0" src="http://pixel.wp.com/b.gif?host=blog.ouseful.info&#038;blog=325417&#038;post=11941&#038;subd=ouseful&#038;ref=&#038;feed=1" width="1" height="1" />]]>
      </content:encoded>
      <wfw:commentRss>http://blog.ouseful.info/2014/07/11/tracking-anonymous-wikipedia-edits-from-specific-ip-ranges/feed/</wfw:commentRss>
      <slash:comments>1</slash:comments>
      <media:content url="http://1.gravatar.com/avatar/abbd9f90565ce9ae4d065d93a81d8c03?s=96&#38;d=http%3A%2F%2F1.gravatar.com%2Favatar%2Fad516503a11cd5ca435acc9bb6523536%3Fs%3D96" medium="image">
        <media:title type="html">Tony Hirst</media:title>
      </media:content>
    </item>
    <item>
      <title>Foolish is as foolish does&#8230; fragments&#8230;</title>
      <link>http://blog.ouseful.info/2014/07/10/foolish-is-as-foolish-does-fragments/</link>
      <comments>http://blog.ouseful.info/2014/07/10/foolish-is-as-foolish-does-fragments/#comments</comments>
      <pubDate>Thu, 10 Jul 2014 18:11:43 +0000</pubDate>
      <dc:creator>
        <![CDATA[Tony Hirst]]>
      </dc:creator>
      <category>
        <![CDATA[Anything you want]]>
      </category>
      <category>
        <![CDATA[corporate fool]]>
      </category>
      <guid isPermaLink="false">http://blog.ouseful.info/?p=11933</guid>
      <description>
        <![CDATA[It&#8217;s been that CDSA paperwork time of year again, and if nothing else it has forced me to start pulling together some fragments of ideas backed up by other peoples&#8217; (or people&#8217;s &#8211; I never can remember) words&#8230; So here are some fragments that are, I think, aligned to some of the things that I&#8217;ve [&#8230;]<img alt="" border="0" src="http://pixel.wp.com/b.gif?host=blog.ouseful.info&#038;blog=325417&#038;post=11933&#038;subd=ouseful&#038;ref=&#038;feed=1" width="1" height="1" />]]></description>
      <content:encoded>
        <![CDATA[<p>It&#8217;s been that CDSA paperwork time of year again, and if nothing else it has forced me to start pulling together some fragments of ideas backed up by other peoples&#8217; (or people&#8217;s &#8211; I never can remember) words&#8230;</p>
<p>So here are some fragments that are, I think, aligned to some of the things that I&#8217;ve <a href="http://blog.ouseful.info/2008/12/09/corporate-foolery-and-the-abilene-paradox/">thought for a long time</a> and have been thinking again more recently, things that resonated with me as I read them just now&#8230;</p>
<p>The way in to this this time round for me was part of a talk by Prof Richard Keeble at the University of Lincoln School of Journalism Research Symposium last week, when he mentioned the role of the fool as a safety valve or regulator in the classical court&#8230;</p>
<p>First up, from &#8220;<a href="https://circle.ubc.ca/bitstream/handle/2429/5042/ubc_1994-0177.pdf?...1">Class Clown and Court Jester</a>&#8220;, David Chevreau, MA thesis, UBC, April 1994</p>
<p>p10-11<br />
<em>The fool can fill a number of specific roles in the group. He represents the rejected values, lost causes, fiascoes, and incompetencies of the larger gathering. His lowly yet valued position in the office of scapegoat and butt of humour gives him license to depart from the group&#8217;s accepted social norms with a unique impunity.</em><br />
<em>Footnote 10 &#8211; The fool is a rebel, outcast, prophet and whipping boy, and his office is a well defined social phenomenon.</em></p>
<p>As a scapegoat, you can speak the truth and people can choose to listen or not. The truth can be ignored if spoken by the fool, because the fool said it.. <!-- As for whipping boy, the blackball that killed my chances of OU promotion years ago I've recently reassessed as a one of a series of consequences, in part, from the need for a fall guy at the time, as well as a complete lack of understanding about alternative ways of getting things done. Which has made me feel a lot better... I'm still a lot poorer as a consequence, though... --></p>
<p>p13<br />
<em>Unperturbed by misplaced authority, the Class Clown seizes an opportunity either the naivete of the natural innocent or the insight of the wizened fool.</em></p>
<p>I&#8217;m <a href="http://blog.ouseful.info/2014/07/02/confused-about-confused-about-posts/">often confused&#8230;</a></p>
<p>p16<br />
<em>The Fool exists on the fringe of social convention, and thus has a license which frees him from responsibility and consequence.</em></p>
<p>Scruffy hippy&#8230;</p>
<p>p18<br />
<em>It is difficult to accuse a Fool of meaning anything, for his foolish words may be nothing more than the babbling of the idiot or a disguise which reveals hidden truth to some but appears to be senseless chatter to others.</em></p>
<p>WTF is he on about?</p>
<p>p22<br />
<em>Transgressing the bounds of propriety in his failure to cope with convention the Fool does not suffer the usual loss of dignity associated with social failure.</em></p>
<p>FFS&#8230; swearing again&#8230;</p>
<p>p22<br />
<em>Standing at the fringe, the Fool may be a disinterested truth-teller whose apparent madness masks his breadth of perspective. The Fool is a detached observer who lives at the boundary not just between order and chaos but also between what is and what appears to be, and is often confused with the silly and deluded.</em></p>
<p>So, I deserve a Chair, right?!:-)</p>
<p>p23<br />
<em>The Fool perceives that the world has more to do with seeming than with seeing, for too much of our world is actually unknown and irrational. To take the imposition of order too seriously is the height of folly, and so the Fool, standing aside, sees through the illusion.</em></p>
<p>Ooh, big data&#8230; and squillionty billionty pounds will be made from open data&#8230; innit.</p>
<p>So that was that one&#8230;</p>
<p>Here&#8217;s another take&#8230; As I read (/red/) this, I also though about my own role within the university&#8230; As above,so below&#8230;? &#8220;Institutional Heterogeneity and Change: The University as Fool&#8221;, Donncha Kavanagh, Organization Volume 16(4): 575–595 ISSN 1350–5084 , 2009. </p>
<p>First, the scene is set&#8230;</p>
<p>p577 &#8220;Detailed study of the history of the University suggests that it is an institution that acts and has a role akin to the Fool in the royal court of medieval times.&#8221;</p>
<p>The paper then explores this idea in narrative form&#8230;</p>
<p>Fool as normative narrator:<br />
p586 &#8220;The Fool is a story-teller, but its stories are always embedded in a framework of norms and values that connect the moment into longer conversations over time and space.&#8221;</p>
<p>There is a context to what we do, and a tradition that informs it, in both form and in content&#8230;</p>
<p>p587 &#8220;Akin to the medieval fool, who is not there to merely tell stories, the University is expected to provide a normative narrative or a critical interpretation of the world. &#8230; the University’s long tradition of academic freedom mirrors the Fool’s position as the Sovereign’s independent critic. &#8230; The university does not just (re-)tell stories, parables, and proverbs. Its power also comes about from its material ability to <em>sort things out</em> (Bowker and Star, 1999 [Bowker, G. and Star, S. L. (1999) Sorting Things Out: Classification and its Consequences. London: MIT Press.]); it is a <em>sorter par excellence</em>.&#8221;</p>
<p>The university helps make sense of the world&#8230; it can do this by putting things into perspective, or ordering them/organising them, in a particular way (that is, is can &#8220;sort&#8221; them, as you might sort a sock drawer, albeit one  that doesn&#8217;t necessarily contain any actual pairs of socks&#8230;)</p>
<p>p588 &#8220;Through these twin processes of normative narrating and sorting the university constructs and maintains what I term the <em>semiotic nexus</em>. The semiotic nexus gives meaning to an institution — be it the University, its sovereign or one of the other institutions in the realm—through telling a multi-part, compelling, value-laden tale about the institution and its place in the world. The university is not the only institution engaged in this process of ‘making meaning’—narrating is a form of theorizing that everyone engages in—but it plays a central role in determining what counts as knowledge, as well as defining what is valuable, peripheral, obscene, sacred, profane, reputable, opinion, fact, etc. The University, like the Fool, personifies truth and reason, in that it is <em>required</em> to tell the truth, to abolish myth, and to distinguish fact from mere opinion. In other words, the University’s normative story-telling ability allied to its sorting practices and technologies are basic to how the University realizes its imagined community of academics, how it at once becomes an institution itself, and also how it maintains and sustains the semiotic nexus underpinning other institutions. In other words, these practices play a significant role in the process of institutionalization.&#8221;</p>
<p>I&#8217;ve noticed that people find it very hard to play&#8230; I can play all day&#8230; Erm&#8230; I can <em>only</em> play?!</p>
<p>p589 Play in the fool<br />
&#8220;The Fool is a ludic spirit within the institutional complex, and play—a free activity standing outside of and opposed to the seriousness of ordinary life (Huizinga, 1955)—is its <em>modus operandi</em>. As with the child, the Fool is allowed, expected and given time and space to play. Through playing with language the Fool sparks a new (yet old) understanding of the here and now. This incandescent quality at once makes events alive—giving them immediate meaning—while simultaneously framing them within a longer temporal structure or <em>longue durée</em> that articulates the empirical with a transcendent truth. Each ‘play’ then endures as a new mental creation, to be repeated and retained in memory, echoing older refrains of truth and tradition. Following Huizinga, play is primordial and because of its close links with the sacred, it works to keep old norms and beliefs alive. The Fool as playmaker <em>extraordinaire</em> is central to this continual process of institutional re-creation through which an institution breathes, lives and renews itself.<br />
Yet, because it takes work to create order within play, play always (sub- liminally) reminds us that the world is fundamentally chaotic and that any meaning within this chaos is always provisional and artificial. The Fool’s work of play then is to institutionalize order and at once to open up order to de-institutionalization. Through its role as playmaker, the Fool puts an institution ‘into play’, which means that work must be done to either recreate or de-stabilize the institution. In this way, the Fool’s ability and license to play is paradoxically central to both institutionalization and de-institutionalization.&#8221;</p>
<p>FWIW, seeing that mention of Huizinga, I&#8217;m reminded of how <em>play</em> is a serious business&#8230; see for example, <a href="http://digitalworlds.wordpress.com/2008/03/11/getting-philosophical-about-games/">Getting Philosophical About Games</a>. The <em>Magic Circle</em> applies similarly to the little closed off workds we lock ourselves into when doing a research project. Don&#8217;t let anyone ever tell you that research is anything more than play, (though it&#8217;s often less&#8230;). Note also that that <em>Digital Worlds</em> blog post was itself is an &#8216;output&#8217; from an uncourse I played with creating a few years ago. The material ended up being used by an actual OU course that followed on. I don&#8217;t think anyone in the OU really got it. Then MOOC hype shite came along and nothing really changed.</p>
<p>Playing the fool is a responsbile job, If you aren&#8217;t responsible, you can step beyond the bounds of playful foolishness and start &#8220;stirring&#8221;, or trying to use the cloak of foolishness to cause trouble directly&#8230;</p>
<p>&#8220;Another transgression occurs when the Fool cannot see beyond the play-making; i.e. the Fool becomes a Trickster, a Lucifer figure working solely to undermine and destroy order. This happens when the Fool forgets that part of the Fool’s role is sustaining order in the institutional complex.&#8221;</p>
<p>Beware Anansi taking over, in other words&#8230;</p>
<p>The &#8220;Emperor&#8217;s New Clothes&#8221; is one of my favourite stories. The boy is portrayed as foolish in his innocence, but he speaks a truth as a naif, or innocent. We see how corollaries to that story can be played out by the wise fool, rather than truth telling innocent&#8230;</p>
<p>p590 fool as educator<br />
&#8220;Pursuing the metaphor of the Fool presents an interesting perspective on the University as an educational institution. While the Fool is an educator of sorts, she does not really ‘own’ knowledge that she ‘passes on’ as per our conventional understanding of pedagogy. Unlike the teacher who is usually cast as the learner’s caring coach, the Fool is an irritant, a provocateur, whose <em>modus operandi</em> is to provoke new wisdom in others. The Fool’s approach is, quite literally, to play the fool, acting as a lucid and ludic lens through which others perceive and recognize profound truths, truths that indeed may be lost in the conventions of learning and scholarship. The fool (like the child) is not expected to ‘know’ anything and is therefore free to act the fool, because she cannot, by definition, ‘know any better’. Paradoxically, this epistemic vacuum is also a potential source of great wisdom, which is why the idea of the ‘wise fool’ has such a long tradition. Moreover, the oxymoron ‘wise fool’ is also reversible: he that believes himself to be wise is necessarily foolish. For the Fool also reminds us that knowledge of the mystery of life is always beyond even the wise; at best we can only know that there is much of which we are and can only be ignorant.<br />
&#8230;<br />
[The university] must be the institutional manifestation of an <em>oxymoron</em>, remembering that this word comes from the Greek, <em>oxumo ̄rone</em>, meaning ‘pointedly foolish’.&#8221;</p>
<p><em>&#8211; Fin</em></p><br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/ouseful.wordpress.com/11933/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/ouseful.wordpress.com/11933/" /></a> <img alt="" border="0" src="http://pixel.wp.com/b.gif?host=blog.ouseful.info&#038;blog=325417&#038;post=11933&#038;subd=ouseful&#038;ref=&#038;feed=1" width="1" height="1" />]]>
      </content:encoded>
      <wfw:commentRss>http://blog.ouseful.info/2014/07/10/foolish-is-as-foolish-does-fragments/feed/</wfw:commentRss>
      <slash:comments>1</slash:comments>
      <media:content url="http://1.gravatar.com/avatar/abbd9f90565ce9ae4d065d93a81d8c03?s=96&#38;d=http%3A%2F%2F1.gravatar.com%2Favatar%2Fad516503a11cd5ca435acc9bb6523536%3Fs%3D96" medium="image">
        <media:title type="html">Tony Hirst</media:title>
      </media:content>
    </item>
  </channel>
</rss>
