Coursera Downloader
===================

|Build Status| |Build status| |Coverage Status| |Latest version on PyPI|
|Downloads from PyPI| |Code Climate|

-  `Introduction <#introduction>`__
-  `Features <#features>`__
-  `Disclaimer <#disclaimer>`__
-  `Installation instructions <#installation-instructions>`__

   -  `Recommended installation method for all Operating
      Systems <#recommended-installation-method-for-all-operating-systems>`__
   -  `Alternative ways of installing missing
      dependencies <#alternative-ways-of-installing-missing-dependencies>`__

      -  `Alternative installation method for Unix
         systems <#alternative-installation-method-for-unix-systems>`__
      -  `Installing dependencies on your
         own <#installing-dependencies-on-your-own>`__

   -  `Windows <#windows>`__
   -  `Create an account with
      Coursera <#create-an-account-with-coursera>`__
   -  `Running the script <#running-the-script>`__
   -  `Resuming downloads <#resuming-downloads>`__

-  `Troubleshooting <#troubleshooting>`__

   -  `Found 0 sections and 0 lectures on this
      page <#found-0-sections-and-0-lectures-on-this-page>`__
   -  `Windows: Failed to create
      process <#windows-failed-to-create-process>`__
   -  `SSLError: Errno 1 \_ssl.c:504: error:14094410:SSL
      routines:SSL3\_READ\_BYTES:sslv3 alert handshake
      failure <#sslerror-errno-1-_sslc504-error14094410ssl-routinesssl3_read_bytessslv3-alert-handshake-failure>`__

-  `Reporting issues <#reporting-issues>`__
-  `Filing an issue/Reporting a bug <#filing-an-issuereporting-a-bug>`__
-  `Feedback <#feedback>`__
-  `Contact <#contact>`__

Introduction
============

`Coursera <https://www.coursera.org>`__ is arguably the leader in
*massive open online courses* (MOOC) with a selection of more than 300
classes from 62 different institutions `as of February
2013 <http://techcrunch.com/2013/02/20/coursera-adds-29-schools-90-courses-and-4-new-languages-to-its-online-learning-platform/>`__.
Generous contributions by educators and institutions are making
excellent education available to many who could not afford it otherwise.
There are even non-profits with "feet on the ground" in remote areas of
the world who are helping spread the wealth (see the feedback below from
`Tunapanda <http://www.tunapanda.org>`__).

This script makes it easier to batch download lecture resources (e.g.,
videos, ppt, etc) for Coursera classes. Given one or more class names
and account credentials, it obtains week and class names from the
*lectures* page, and then downloads the related materials into
appropriately named files and directories.

Why is this helpful? A utility like
`wget <https://sourceforge.net/projects/gnuwin32/files/wget/1.11.4-1/wget-1.11.4-1-setup.exe>`__
can work, but has the following limitations:

1. Video names have numbers in them, but this does not correspond to the
   actual order. Manually renaming them is a pain that is best left for
   computers.
2. Using names from the syllabus page provides more informative names.
3. Using ``wget`` in a for loop picks up extra videos which are not
   posted/linked, and these are sometimes duplicates.

Browser extensions like *DownloadThemAll* is another possibility, but
``coursera-dl`` provides more features such as appropriately named
files.

This work was originally inspired in part by
`youtube-dl <http://rg3.github.io/youtube-dl>`__ by which I've
downloaded many other good videos such as those from Khan Academy.

Features
========

-  Support for all kinds of courses (i.e., "Old Platform"/time-based as
   well as "New Platform"/on-demand courses).
-  Intentionally detailed names, so that it will display and sort
   properly on most interfaces (e.g.,
   `VLC <https://f-droid.org/repository/browse/?fdid=org.videolan.vlc>`__
   or MX Video on Android devices).
-  Regex-based section (week) and lecture name filters to download only
   certain resources.
-  File format extension filter to grab resource types you want.
-  Login credentials accepted on command-line or from ``.netrc`` file.
-  Core functionality tested on Linux, Mac and Windows.

Disclaimer
==========

``coursera-dl`` is meant to be used only for your material that Coursera
gives you access to download.

We do not encourage any use that violates their `Terms Of
Use <https://www.coursera.org/about/terms>`__. A relevant excerpt:

    "[...] Coursera grants you a personal, non-exclusive,
    non-transferable license to access and use the Sites. You may
    download material from the Sites only for your own personal,
    non-commercial use. You may not otherwise copy, reproduce,
    retransmit, distribute, publish, commercially exploit or otherwise
    transfer any material, nor may you modify or create derivatives
    works of the material."

Installation instructions
=========================

``coursera-dl`` requires Python 2 or Python 3 and a free Coursera
account enrolled in the class of interest. (As of February of 2016, we
test automatically the execution of the program with Python versions
2.6, 2.7, Pypy, 3.2, 3.3, 3.4, and 3.5).

**Note:** We *strongly* recommend that you use a Python 3 interpreter
(3.4 or later).

On any operating system, ensure that the Python executable location is
added to your ``PATH`` environment variable and, once you have the
dependencies installed (see next section), for a *basic* usage, you will
need to invoke the script from the main directory of the project and
prepend it with the word ``python``. You can also use more advanced
features of the program by looking at the "Running the script" section
of this document.

*Note:* You must already have (manually) agreed to the Honor of Code of
the particular courses that you want to use with ``coursera-dl``.

Recommended installation method for all Operating Systems
---------------------------------------------------------

From a command line (preferrably, from a virtual environment), simply
issue the command:

::

    pip install coursera-dl

This will download `the latest released
version <https://pypi.python.org/pypi/coursera-dl>`__ of the program
from the `Python Package Index (PyPI) <https://pypi.python.org/>`__
along with *all* the necessary dependencies. At this point, you should
be ready to start using it.

If this does not work, because your Python 2 version is too old (e.g.
2.7.5 on Ubuntu 14.4), try:

::

    apt-get install python3 python3-pip
    pip3 install coursera-dl

instead.

**Note 1:** We strongly recommend that you *don't* install the package
globally on your machine (i.e., with root/administrator privileges), as
the installed modules may conflict with other Python applications that
you have installed in your system (or they can interfere with
``coursera-dl``). Prefer to use the option ``--user`` to
``pip install``, if you need can.

**Note 2:** As already mentioned, we *strongly* recommend that you use a
new Python 3 interpreter (e.g., 3.4 or later), since Python 3 has better
support for SSL/TLS (for secure connections) than earlier versions. If
you must use Python 2, be sure that you have at least Python 2.7.9
(later versions are OK). Otherwise, you can still use ``coursera-dl``,
but you will have to install the extra package ``ndg-httpsclient``,
which may involve compilation (at least on Linux systems).

Alternative ways of installing missing dependencies
---------------------------------------------------

We strongly recommend that you consider installing Python packages with
`pip <https://pip.pypa.io/en/latest/>`__, as in it is the current
`preferred
method <http://ww45.python-distribute.org/pip_distribute.png>`__, unless
directed otherwise by one of the project members (for instance, when
testing or debugging a new feature or using the source code directly
from our git repository). If you are using ``pip``, you can directly
install all the dependencies from the requirements file using
``pip install -r requirements.txt``.

Alternative installation method for Unix systems
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

We strongly recommend that you install ``coursera-dl`` and all its
dependencies in a way that does *not* interfere with the rest of your
Python installation. This is accomplished by the creation of a *virtual
environment*, or "virtualenv".

For the initial setup, in a Unix-like operating system, please use the
following steps (create/adapt first the directory
``/directory/where/I/want/my/courses``):

::

    cd /directory/where/I/want/my/courses
    virtualenv my-coursera
    cd my-coursera
    source bin/activate
    git clone https://github.com/coursera-dl/coursera-dl
    cd coursera-dl
    pip install -r requirements.txt
    ./coursera-dl ...

To further download new videos from your classes, simply perform:

::

    cd /directory/where/I/want/my/courses/my-coursera
    source bin/activate
    cd coursera-dl
    ./coursera-dl ...

We are working on streamlining this whole process so that it is as
simple as possible, but to support older versions of Python and to cope
with Coursera disabling SSLv3, we have to take a few extra steps. In any
case, it is *highly* recommended that you always install the latest
version of the Python interpreter that you can.

ArchLinux
~~~~~~~~~

AUR package:
`coursera-dl <https://aur.archlinux.org/packages/coursera-dl/>`__

Installing dependencies on your own
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

**Warning:** This method is not recommended unless you have experience
working with multiple Python environments.

You can use the ``pip`` program to install the dependencies on your own.
They are all listed in the ``requirements.txt`` file (and the extra
dependencies needed for development are listed in the
``requirements-dev.txt`` file).

To use this method, you would proceed as:

::

    pip install -r requirements.txt
    pip install -r requirements-dev.txt

The second line above should only be needed if you intend to help with
development (and help is *always* welcome) or if a maintainer of the
project asks you to install extra packages for debugging purposes.

Once again, before filing bug reports, if you installed the dependencies
on your own, please check that the versions of your modules are at least
those listed in the ``requirements.txt`` file (and,
``requirements-dev.txt`` file, if applicable).

Windows
-------

``python -m pip install coursera-dl``

Be sure that the Python install path is added to the PATH system
environment variables. This can be found in Control Panel > System >
Advanced System Settings > Environment Variables.

::

    Example:
    C:\Python35\Scripts\;C:\Python35\;

Or if you have restricted installation permissions and you've installed
Python under AppData, add this to your PATH.

::

    Example:
    C:\Users\<user>\AppData\Local\Programs\Python\Python35-32\Scripts;C:\Users\<user>\AppData\Local\Programs\Python\Python35-32;

Coursera-dl can now be run from commandline or powershell.

Create an account with Coursera
-------------------------------

If you don't already have one, create a
`Coursera <https://www.coursera.org>`__ account and enroll in a class.
See https://www.coursera.org/courses for the list of classes.

Running the script
------------------

Run the script to download the materials by providing your Coursera
account credentials (e.g. email address and password or a ``~/.netrc``
file), the class names, as well as any additional parameters:

::

    General:                     coursera-dl -u <user> -p <pass> modelthinking-004
    Multiple classes:            coursera-dl -u <user> -p <pass> saas historyofrock1-001 algo-2012-002
    Filter by section name:      coursera-dl -u <user> -p <pass> -sf "Chapter_Four" crypto-004
    Filter by lecture name:      coursera-dl -u <user> -p <pass> -lf "3.1_" ml-2012-002
    Download only ppt files:     coursera-dl -u <user> -p <pass> -f "ppt" qcomp-2012-001
    Use a ~/.netrc file:         coursera-dl -n -- matrix-001
    Get the preview classes:     coursera-dl -n -b ni-001
    Specify download path:       coursera-dl -n --path=C:\Coursera\Classes\ comnetworks-002
    Display help:                coursera-dl --help

    Maintain a list of classes in a dir:
      Initialize:              mkdir -p CURRENT/{class1,class2,..classN}
      Update:                  coursera-dl -n --path CURRENT `\ls CURRENT`

**Note:** If your ``ls`` command is aliased to display a colorized
output, you may experience problems. Be sure to escape the ``ls``
command (use ``\ls``) to assure that no special characters get sent to
the script.

Note that we *do* support the New Platform ("on-demand") classes.

On \*nix platforms, the use of a ``~/.netrc`` file is a good alternative
to specifying both your username (i.e., your email address) and password
every time on the command line. To use it, simply add a line like the
one below to a file named ``.netrc`` in your home directory (or the
`equivalent <http://stackoverflow.com/a/6031266/962311>`__, if you are
using Windows) with contents like:

::

    machine coursera-dl login <user> password <pass>

Create the file if it doesn't exist yet. From then on, you can switch
from using ``-u`` and ``-p`` to simply call ``coursera-dl`` with the
option ``-n`` instead. This is especially convenient, as typing
usernames (email addresses) and passwords directly on the command line
can get tiresome (even more if you happened to choose a "strong"
password).

Resuming downloads
------------------

In default mode when you interrupt the download process by pressing
CTRL+C, partially downloaded files will be deleted from your disk and
you have to start the download process from the begining. If your
download was interrupted by something other than KeyboardInterrupt
(CTRL+C) like sudden system crash, partially downloaded files will
remain on your disk and the next time you start the process again, these
files will be discraded from download list!, therefore it's your job to
delete them manually before next start. For this reason we added an
option called ``--resume`` which continues your downloads from where
they stopped:

::

    coursera-dl -u <user> -p <pass> --resume sdn1-001

This option can also be used with external downloaders:

::

    coursera-dl --wget -u <user> -p <pass> --resume sdn1-001

*Note 1*: Some external downloaders use their own built-in resume
feature which may not be compatible with others, so use them at your own
risk.

*Note 2*: Remember that in resume mode, interrupted files **WON'T** be
deleted from your disk.

**NOTE**: If your password contains punctuation, quotes or other "funny
characters" (e.g., ``<``, ``>``, ``#``, ``&``, ``|`` and so on), then
you may have to escape them from your shell. With bash or other
Bourne-shell clones (and probably with many other shells) one of the
better ways to do so is to enclose your password in single quotes, so
that you don't run into problems. See `issue
#213 <https://github.com/coursera-dl/coursera-dl/issues/213>`__ for more
information.

Troubleshooting
===============

If you have problems when downloading class materials, please try to see
if one of the following actions solve your problem:

-  Make sure the class name you are using corresponds to the resource
   name used in the URL for that class:
   ``https://class.coursera.org/<CLASS_NAME>/class/index``

-  Have you tried to clean the cached cookies/credentials with the
   ``--clear-cache`` option?

-  Note that many courses (most, perhaps?) may remove the materials
   after a little while after the course is completed, while other
   courses may retain the materials up to a next session/offering of the
   same course (to avoid problems with academic dishonesty, apparently).
   In short, it is not guaranteed that you will be able to download
   after the course is finished and this is, unfortunately, nothing that
   we can help you with.

-  Make sure you have installed and/or updated all of your dependencies
   according to the ``requirements.txt`` file as described above.

-  One can export a Netscape-style cookies file with a browser extension
   (`1 <https://chrome.google.com/webstore/detail/cookietxt-export/lopabhfecdfhgogdbojmaicoicjekelh>`__,
   `2 <https://addons.mozilla.org/en-US/firefox/addon/export-cookies/>`__)
   and use it with the ``-c`` option. This comes in handy when the
   authentication via password is not working (the authentication
   process changes now and then).

-  If results show 0 sections, you most likely have provided invalid
   credentials (username and/or password in the command line or in your
   ``.netrc`` file).

-  For courses that have not started yet, but have had a previous
   iteration sometimes a preview is available, containing all the
   classes from the last course. These files can be downloaded by
   passing the ``--preview`` parameter.

-  If you get an error like ``Could not find class: <CLASS_NAME>``,
   then:

   -  Verify that the name of the course is correct. Current class names
      in coursera are composed by a short course name e.g. ``class`` and
      the current version of the course (a number). For example, for a
      class named ``class``, you would have to use ``class-001``,
      ``class-002`` etc.
   -  Second, verify that you are enrolled in the course. You won't be
      able to access the course materials if you are not officially
      enrolled and agreed to the honor course *via the website*.

-  If:

   -  You get an error when using ``-n`` to specify that you want to use
      a ``.netrc`` file and,
   -  You want the script to use your default netrc file and,
   -  You get a message saying ``coursera-dl: error: too few arguments``

   Then you should specify ``--`` as an argument after ``-n``, that is,
   ``-n --`` or change the order in which you pass the arguments to the
   script, so that the argument after ``-n`` begins with an hyphen
   (``-``). Otherwise, Python's ``argparse`` module will think that what
   you are passing is the name of the netrc file that you want to use.
   See issue #162.

-  If your password has spaces, don't forget to write it using quotes.

-  Have you installed the right project ? **Warning**: If you installed
   the script using PyPi (pip) please verify that you installed the
   correct project. We had to use a different name in pip because our
   original name was already taken. Remember to install it using:
   ``pip install coursera-dl``

Found 0 sections and 0 lectures on this page
--------------------------------------------

First of all, make sure you are enrolled to the course you want to
download.

Many old courses have already closed enrollment so often it's not an
option. In this case, try downloading with ``--preview`` option. Some
courses allow to download lecture materials without enrolling, but it's
not common and is not guaranteed to work for every course.

Finally, you can download the videos if you have, at least, the index
file that lists all the course materials. Maybe your friend who is
enrolled could save that course page for you. In that case use the
``--process_local_page`` option.

Alternatively you may want to try this Chrome extension:
https://chrome.google.com/webstore/detail/coursera-materials-downlo/ijkboagofaehocnjacacdhdcbbcpilih

If none of the above works for you, there is nothing we can do.

Windows: Failed to create process
---------------------------------

In ``C:\Users\<user>\AppData\Local\Programs\Python\Python35-32\Scripts``
or wherever Python installed (above is default for Windows) edit below
file in idle: (right click on script name and select 'edit with idle in
menu)

::

    coursera-dl-script

from

::

    #!c:\users\<user>\appdata\local\programs\python\python35-32\python.exe

to

::

    #"!c:\users\<user>\appdata\local\programs\python\python35-32\python.exe"

(add quotes). This is a known pip bug.

Source: `issue
#500 <https://github.com/coursera-dl/coursera-dl/issues/500>`__
`StackOverflow <http://stackoverflow.com/questions/31808180/installing-pyinstaller-via-pip-leads-to-failed-to-create-process>`__

SSLError: [Errno 1] \_ssl.c:504: error:14094410:SSL routines:SSL3\_READ\_BYTES:sslv3 alert handshake failure
------------------------------------------------------------------------------------------------------------

This is a known error, please do not report about this error message!
The problem is in **YOUR** environment. To fix it, do the following:

.. code:: bash

    sudo apt-get install build-essential python-dev libssl-dev libffi-dev
    pip install --user urllib3 pyasn1 ndg-httpsclient pyOpenSSL

If the error remains, try installing coursera-dl from github following
this instruction:
https://github.com/coursera-dl/coursera-dl#alternative-installation-method-for-unix-systems

If you still have the problem, please read the following issues for more
ideas on how to fix it:
`#330 <https://github.com/coursera-dl/coursera-dl/issues/330>`__
`#377 <https://github.com/coursera-dl/coursera-dl/issues/377>`__
`#329 <https://github.com/coursera-dl/coursera-dl/issues/329>`__

This is also worth reading:
https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning

Reporting issues
================

Before reporting any issue please follow the steps below:

1. Verify that you are running the latest version of the script, and the
   recommended versions of its dependencies, see them in the file
   ``requirements.txt``. Use the following command if in doubt:

   ::

       pip install --upgrade coursera-dl

2. If the problem persists, feel free to `open an
   issue <https://github.com/coursera-dl/coursera-dl/issues>`__ in our
   bugtracker, please fill the issue template with *as much information
   as possible*.

Filing an issue/Reporting a bug
===============================

When reporting bugs against ``coursera-dl``, please don't forget to
include enough information so that you can help us help you:

-  Is the problem happening with the latest version of the script?
-  What operating system are you using?
-  Do you have all the recommended versions of the modules? See them in
   the file ``requirements.txt``.
-  What is the course that you are trying to access?
-  What is the precise command line that you are using (feel free to
   hide your username and password with asterisks, but leave all other
   information untouched).
-  What are the precise messages that you get? Please, use the
   ``--debug`` option before posting the messages as a bug report.
   Please, copy and paste them. Don't reword/paraphrase the messages.

Feedback
========

I enjoy getting feedback. Here are a few of the comments I've received:

-  "Thanks for the good job! Knowledge will flood the World a little
   more thanks to your script!" Guillaume V. 11/8/2012

-  "Just wanted to send you props for your Python script to download
   Coursera courses. I've been using it in Kenya for my non-profit to
   get online courses to places where internet is really expensive and
   unreliable. Mostly kids here can't afford high school, and
   downloading one of these classes by the usual means would cost more
   than the average family earns in one week. Thanks!" Jay L.,
   `Tunapanda <http://www.tunapanda.org>`__ 3/20/2013

-  "I am a big fan of Coursera and attend lots of different courses.
   Time constraints don't allow me to attend all the courses I want at
   the same time. I came across your script, and I am very happily using
   it! Great stuff and thanks for making this available on Github - well
   done!" William G. 2/18/2013

-  "This script is awesome! I was painstakingly downloading each and
   every video and ppt by hand -- looked into wget but ran into wildcard
   issues with HTML, and then.. I came across your script. Can't tell
   you how many hours you've just saved me :) If you're ever in Paris /
   Stockholm, it is absolutely mandatory that I buy you a beer :)"
   Razvan T. 11/26/2012

-  "Thanks a lot! :)" Viktor V. 24/04/2013

Contact
=======

Please, post bugs and issues on
`github <https://github.com/coursera-dl/coursera-dl/issues>`__. Send
other comments to Rogério Theodoro de Brito (the current maintainer):
rbrito@ime.usp.br (twitter:
[@rtdbrito]\ `21 <https://twitter.com/rtdbrito>`__) or to John Lehmann
(the original author): first last at geemail dotcom (twitter:
[@jplehmann]\ `12 <https://twitter.com/jplehmann>`__).

|Bitdeli Badge|

.. |Build Status| image:: https://travis-ci.org/coursera-dl/coursera-dl.svg?branch=master
   :target: https://travis-ci.org/coursera-dl/coursera-dl
.. |Build status| image:: https://ci.appveyor.com/api/projects/status/3hru0ycv5fbny5k8/branch/master?svg=true
   :target: https://ci.appveyor.com/project/balta2ar/coursera-dl/branch/master
.. |Coverage Status| image:: https://coveralls.io/repos/coursera-dl/coursera-dl/badge.svg
   :target: https://coveralls.io/r/coursera-dl/coursera-dl
.. |Latest version on PyPI| image:: https://img.shields.io/pypi/v/coursera-dl.svg
   :target: https://pypi.python.org/pypi/coursera-dl
.. |Downloads from PyPI| image:: https://img.shields.io/pypi/dm/coursera-dl.svg
   :target: https://pypi.python.org/pypi/coursera-dl
.. |Code Climate| image:: https://codeclimate.com/github/coursera-dl/coursera-dl/badges/gpa.svg
   :target: https://codeclimate.com/github/coursera-dl/coursera-dl
.. |Bitdeli Badge| image:: https://d2weczhvl823v0.cloudfront.net/coursera-dl/coursera-dl/trend.png
   :target: https://bitdeli.com/free
