# Why should I test IVRE? #

The `tests` directory is mainly intended for people who want to
contribute to IVRE and want to make sure their changes do not break
IVRE.

# The tests #

## Dependencies ##

To run IVRE tests you will need
[coverage.py](http://nedbatchelder.com/code/coverage/). Under debian,
run `apt-get install python-coverage`.

## Test case ##

The first thing is to find samples. You need both (recent) Nmap XML
scan result files (or Nmap JSON files, as generated by `ivre scancli
--json`) and PCAP files (dump while you browse, and browse a lot, or
sniff a busy open Wi-Fi network, if that's legal in your country).

A good test case should have a lot of various data (sniff from
different places, scan different hosts with different Nmap options).

It is mandatory to have at least, for the Nmap test:

  - Two scanned (and up) hosts with different IP addresses
  - One host scanned with the script `http-robots.txt` reporting
    `/cgi-bin` in its output.
  - One host scanned with an anonymous FTP server.
  - One scan result with traceroute and at least one hop with a
    hostname.
  - One host scanned with a hostname ending with ".com".

For the passive test:

  - Two records with different IP addresses.
  - One SSL certificate.

## First run ##

From the `tests` directory, create the `samples` subdirectory and
place your samples there (the PCAP files must have the extension
`.pcap`, the Nmap XML result files must have the extension `.xml`, and
the Nmap JSON results must have the extension `.json`).

Then, run `python ./tests.py` (optionally replace `python` by the
alternative interpreter you want to use, e.g., `python2.6`; note that
[coverage.py](#Dependencies) must be installed for this
interpreter). The first run will create a `samples/results` file with
the expected values for some results. The next runs will use those
values to check if something has been broken.

For this reason, it is important to:

  - Run the tests for the first time with a "known-working" version.
  - Remove the file `samples/results` whenever a sample file is added,
    modified or removed.

## Improving the test case ##

If you want to make sure to have enough samples, you can:

  - Check the `samples/results` file for `*_count` entries with low
    values (particularly 0, of course) and find or create new samples
    that will improve those values.
  - Check the report generated by coverage.py under the `htmlcov`
    directory, and check whether your current test case covers at
    least the code you want to change.

## Failures ##

Tests failure are not always an issue. Apart from a new bug, of
course, here are some reasons that can explain test failures:

  - You have added new samples and have not removed the
    `samples/results` file.
  - Your samples do not match the minimum requirements detailed above.
  - A new feature has been added to IVRE and the new results are
    actually better than the stored ones.


---

This file is part of IVRE. Copyright 2011 - 2015
[Pierre LALET](mailto:pierre.lalet@cea.fr)
