============
Installation
============

It supports installation with devstack.

This document describes parts related to networking-spp. For the entire
devstack please refer to the devstack document_.

.. _document: https://docs.openstack.org/devstack/latest/

Control node
============

In control node, there are not many parameters related to networking-spp
that should be set in local.conf.

Parameters related to networking-spp are briefly described below with examples.
See :doc:`devstack` for details on parameters added by networking-spp.

Note that it is a fragment extracted only for networking-spp::

  [[local|localrc]]

  Q_AGENT=linuxbridge    # this must be specified for VM operation network. linuxbridge is an example.

  enable_plugin networking-spp https://opendev.org/x/networking-spp.git master  # this line must be specified.

  SPP_MODE=controller    # this line must be specified for control node.

  enable_service etcd3   # enable etcd3 since networking-spp uses etcd.

  [[post-config|/$Q_PLUGIN_CONF_FILE]]
  [ml2]
  type_drivers=vxlan,flat,vlan       # add 'flat' and 'vlan' if necessary, in addition to VM operation network (vxlan is an example).
  mechanism_drivers=linuxbridge,spp  # 'spp' must be added in addition to VM operation network (linuxbridge is an example).

  [ml2_type_flat]
  flat_networks=phys1                # specify physical networks for SPP flat type network. the value is an example.

  [ml2_type_vlan]
  network_vlan_ranges=phys2:400:499  # specify parameters for SPP vlan type network. the value is an example.

Compute node
============

In compute node, there are some tasks to do before executing devstack.

If you want to use KVM instances for the compute nodes which SPP runs for the
purpose of constructing CI/development testing environment, you should note
the following section.

* :ref:`using_kvm_instance_for_compute_node`

Preliminary design
------------------

Since SPP occupies memory and core, it must be designed beforehand for
its allocation amount. The amount of resources allocated to SPP and
the number of VMs that can be ran are limited by the host's memory and the
number of cores. In other words, it is necessaty to prepare enough memory
and cores to operate.

Allocation of hugepage
++++++++++++++++++++++

SPP uses hugepage and VMs that use SPP networks also need to use hugepage.
Normally, the memory on the host should be allocated as hugepage except
for the amount used by the system services.
The allocated hugepages are shared by SPP and VMs.

Distribution of core
++++++++++++++++++++

SPP needs to occupy some cores. It is necessary to separate the cores
for SPP so as not to be used from system services or VMs. Also, it is
better to separate the cores used by VMs from the cores used by system
services. Therefore, the cores on the host are classified into the
following three.

* For SPP
* For VMs
* other (for system services)

The number of cores occupied by SPP can be calculated by the following
formula.

"Number of physical NICs assigned to SPP" * 2 + "Total number of vhostusers" * 2

Preliminary Setting
-------------------

Set the following kernel boot parameters.

* hugepage related parameters
* isolcpus

(Note: The followng example is executed on ubuntu. Other distributios
may be different.)

Edit /etc/default/grub and add parameters to GRUB_CMDLINE_LINUX. For example::

  $ sudo vi /etc/default/grub
  ...
  # isolcpus specifies the cores excluding the cores for system services.
  GRUB_CMDLINE_LINUX="hugepagesz=2M hugepages=5120 isolcpus=2-19"
  # default_hugepagesz should be specified when using 1GB hugepage.
  #GRUB_CMDLINE_LINUX="default_hugepagesz=1G hugepagesz=1G hugepages=16 isolcpus=2-19"

Execute update-grub::

  $ sudo update-grub

Reboot the host::

  $ sudo reboot

The amount of allocated hugepages can be confirmed in /proc/meminfo. For example::

  $ cat /proc/meminfo
  ...
  HugePages_Total:      16
  HugePages_Free:       16
  HugePages_Rsvd:        0
  HugePages_Surp:        0
  Hugepagesize:    1048576 kB

Run devstack
------------

Note that it is necessary to execute devstack of compute node with control
node in operation.

Parameters related to networking-spp are briefly described below with examples.
See :doc:`devstack` for details on parameters added by networking-spp.

Note that it is a fragment extracted only for networking-spp::

  [[local|localrc]]

  Q_AGENT=linuxbridge       # this must be specified for VM operation network. linuxbridge is an example.

  enable_plugin networking-spp https://opendev.org/x/networking-spp.git master  # this line must be specified.

  SPP_PRIMARY_SOCKET_MEM=1024,1024                                       # amount of hugepage used by SPP. per numa node. MB.
  SPP_PRIMARY_CORE_MASK=0x2                                              # core mask used by spp_primary.
  DPDK_PORT_MAPPINGS=00:04.0#phys1#2#0xfe,00:05.0#phys2#2#0xfc02         # configuration information about NICs used for SPP.

  disable_all_services      # Normally, it is necessary and sufficient for the following three services.
  enable_service n-cpu      #
  enable_service q-agt      # agent for VM operation network.
  enable_service q-spp-agt  # spp-agent

  [[post-config|$NOVA_CONF]]
  [DEFAULT]
  vcpu_pin_set = 8,9,16-19              # specify the cores for VMs.

  [libvirt]
  # This option enables VMs to use some features on host cpu, that are
  # needed for DPDK (e.g. SSE instruction).
  cpu_mode = host-passthrough

Note that DPDK ports should be down before running stack.sh.
You can down ports by the following command::

  $ sudo ip link set down <interface name>

Post Work
---------

There are some tasks required after running devstack.


Suppression of apparmor
+++++++++++++++++++++++

.. note::

  This is not necessary for ubuntu 18.04 or later.

Edit /etc/libvirt/qemu.conf and set security_driver to none::

  $ sudo vi /etc/libvirt/qemu.conf
  ...
  security_driver = "none"
  ...

Restart libvirtd::

  $ sudo systemctl restart libvirtd.service

Register compute node
+++++++++++++++++++++

This is the work done on the control node.

Execute nova-manage to register compute node::

  $ nova-manage cell_v2 discover_hosts

Note that it must be executed each time when a compute node is added.

It can be confirmed with the following command::

  $ openstack hypervisor list

Note: rebooting compute node
----------------------------

When rebooting compute node, you need to execute unstack.sh before shutting down
and execute stack.sh after rebooting.


.. _using_kvm_instance_for_compute_node:

Note: using KVM instance for compute node
-----------------------------------------

* Our devstack script doesn't support virtio for the interface of the virtual
  compute nodes of KVM instances. Please use other interface models. e.g. e1000
* If the virtual compute nodes are also created by OpenStack, remember to
  disable the security group on the compute nodes' ports to disable the arp/ip
  spoofing.
