Library — NeuroLab 0.2.2 documentation

### Navigation

- index
- modules |
- next |
- previous |
- NeuroLab 0.2.2 documentation »

# Library

## net: Neural Networks

The module contains the basic network architectures

Network Type | Function | Count of layers | Support train fcn | Error fcn
--- | --- | --- | --- | ---
Single-layer perceptron | newp | 1 | train_delta | SSE
Multi-layer perceptron | newff | >=1 | train_gd, train_gdm, train_gda, train_gdx*, train_rprop, train_bfgs, train_cg | SSE
Competitive layer | newc | 1 | train_wta, train_cwta* | SAE
LVQ | newlvq | 2 | train_lvq | MSE
Elman | newelm | >=1 | train_gdx | MSE
Hopield | newhop | 1 | None | None
Hemming | newhem | 2 | None | None

Note

* - default function

`neurolab.net.` `newc`(minmax, cn)

  Create competitive layer (Kohonen network)

  Parameters : | minmax: list ci x 2 Range of input value cn: int Number of neurons
  Returns : | net: Net
  Example : |
      >>> # create network with 2 inputs and 10 neurons
      >>> net = newc([[-1, 1], [-1, 1]], 10)

`neurolab.net.` `newelm`(minmax, size, transf=None)

  Create a Elman recurrent network

  Parameters : | minmax: list ci x 2 Range of input value size: list of length equal to the number of layers Contains the number of neurons for each layer
  Returns : | net: Net
  Example : |
      >>> net = newelm([[-1, 1]], [1], [trans.PureLin()])
      >>> net.layers[0].np['w'][:] = 1
      >>> net.layers[0].np['b'][:] = 0
      >>> net.sim([[1], [1] ,[1], [3]])
      array([[ 1.],
             [ 2.],
             [ 3.],
             [ 6.]])

`neurolab.net.` `newff`(minmax, size, transf=None)

  Create multilayer perceptron

  Parameters : | minmax: list ci x 2 Range of input value size: list of length equal to the number of layers Contains the number of neurons for each layer transf: list (default TanSig) List of activation function for each layer
  Returns : | net: Net
  Example : |
      >>> # create neural net with 2 inputs, 1 output and 2 layers
      >>> net = newff([[-0.5, 0.5], [-0.5, 0.5]], [3, 1])
      >>> net.ci
      2
      >>> net.co
      1
      >>> len(net.layers)
      2

`neurolab.net.` `newhem`(target, transf=None, max_iter=10, delta=0)

  Create a Hemming recurrent network with 2 layers

  Parameters : | target: array like (l x net.co) train target patterns transf: func (default SatLinPrm(0.1, 0, 10)) Activation function of input layer max_init: int (default 10) Maximum of recurent iterations delta: float (default 0) Minimum diference between 2 outputs for stop reccurent cycle
  Returns : | net: Net
  Example : |
      >>> net = newhop([[-1, -1, -1], [1, -1, 1]])
      >>> output = net.sim([[-1, 1, -1], [1, -1, 1]])

`neurolab.net.` `newhop`(target, transf=None, max_init=10, delta=0)

  Create a Hopfield recurrent network

  Parameters : | target: array like (l x net.co) train target patterns transf: func (default HardLims) Activation function max_init: int (default 10) Maximum of recurent iterations delta: float (default 0) Minimum diference between 2 outputs for stop reccurent cycle
  Returns : | net: Net
  Example : |
      >>> net = newhop([[-1, -1, -1], [1, -1, 1]])
      >>> output = net.sim([[-1, 1, -1], [1, -1, 1]])

`neurolab.net.` `newhop_old`(target, transf=None)

  Create a Hopfield recurrent network.

  Old version need tool.simhop for use. Will be removed in future versions.

  Parameters : | target: array like (l x net.co) train target patterns transf: func (default HardLims) Activation function
  Returns : | net: Net
  Example : |
      >>> from neurolab.tool import simhop
      >>> net = newhop_old([[-1, 1, -1], [1, -1, 1]])
      >>> output = simhop(net, [[-1, 1, -1], [1, -1, 1]])

`neurolab.net.` `newlvq`(minmax, cn0, pc)

  Create a learning vector quantization (LVQ) network

  Parameters : | minmax: list ci x 2 Range of input value cn0: int Number of neurons in input layer pc: list List of percent, sum(pc) == 1
  Returns : | net: Net
  Example : |
      >>> # create network with 2 inputs,
      >>> # 2 layers and 10 neurons in each layer
      >>> net = newlvq([[-1, 1], [-1, 1]], 10, [0.6, 0.4])

`neurolab.net.` `newp`(minmax, cn, transf=<neurolab.trans.HardLim instance at 0xa36060c>)

  Create one layer perceptron

  Parameters : | minmax: list ci x 2 Range of input value cn: int Number of neurons transf: func (default HardLim) Activation function
  Returns : | net: Net
  Example : |
      >>> # create network with 2 inputs and 10 neurons
      >>> net = newp([[-1, 1], [-1, 1]], 10)

## train: Train Algorithms

### Train algorithms based gradients algorihms

`neurolab.train.` `train_gd`(Train, epochs=500, goal=0.01, show=100, **kwargs)

  Gradient descent backpropagation

  Support networks:
  | newff (multi-layers perceptron)
  Parameters : | input: array like (l x net.ci) train input patterns target: array like (l x net.co) train target patterns epochs: int (default 500) Number of train epochs show: int (default 100) Print period goal: float (default 0.01) The goal of train lr: float (defaults 0.01) learning rate adapt bool (default False) type of learning

`neurolab.train.` `train_gdm`(Train, epochs=500, goal=0.01, show=100, **kwargs)

  Gradient descent with momentum backpropagation

  Support networks:
  | newff (multi-layers perceptron)
  Parameters : | input: array like (l x net.ci) train input patterns target: array like (l x net.co) train target patterns epochs: int (default 500) Number of train epochs show: int (default 100) Print period goal: float (default 0.01) The goal of train lr: float (defaults 0.01) learning rate adapt bool (default False) type of learning

`neurolab.train.` `train_gda`(Train, epochs=500, goal=0.01, show=100, **kwargs)

  Gradient descent with adaptive learning rate

  Support networks:
  | newff (multi-layers perceptron)
  Parameters : | input: array like (l x net.ci) train input patterns target: array like (l x net.co) train target patterns epochs: int (default 500) Number of train epochs show: int (default 100) Print period goal: float (default 0.01) The goal of train lr: float (defaults 0.01) learning rate adapt: bool (detault False) type of learning lr_inc: float (> 1, default 1.05) Ratio to increase learning rate lr_dec: float (< 1, default 0.7) Ratio to decrease learning rate max_perf_inc:float (> 1, default 1.04) Maximum performance increase

`neurolab.train.` `train_gdx`(Train, epochs=500, goal=0.01, show=100, **kwargs)

  Gradient descent with momentum backpropagation and adaptive lr

  Support networks:
  | newff (multi-layers perceptron)
  Рarameters : | input: array like (l x net.ci) train input patterns target: array like (l x net.co) train target patterns epochs: int (default 500) Number of train epochs show: int (default 100) Print period goal: float (default 0.01) The goal of train lr: float (defaults 0.01) learning rate adapt: bool (detault False) type of learning lr_inc: float (default 1.05) Ratio to increase learning rate lr_dec: float (default 0.7) Ratio to decrease learning rate max_perf_inc:float (default 1.04) Maximum performance increase mc: float (default 0.9) Momentum constant

`neurolab.train.` `train_rprop`(Train, epochs=500, goal=0.01, show=100, **kwargs)

  Resilient Backpropagation

  Support networks:
  | newff (multi-layers perceptron)
  Parameters : | input: array like (l x net.ci) train input patterns target: array like (l x net.co) train target patterns epochs: int (default 500) Number of train epochs show: int (default 100) Print period goal: float (default 0.01) The goal of train lr: float (defaults 0.07) learning rate (init rate) adapt bool (default False) type of learning rate_dec: float (default 0.5) Decrement to weight change rate_inc: float (default 1.2) Increment to weight change rate_min: float (default 1e-9) Minimum performance gradient rate_max: float (default 50) Maximum weight change

### Train algorithms based on Winner Take All - rule

`neurolab.train.` `train_wta`(Train, epochs=500, goal=0.01, show=100, **kwargs)

  Winner Take All algorithm

  Support networks:
  | newc (Kohonen layer)
  Parameters : | input: array like (l x net.ci) train input patterns epochs: int (default 500) Number of train epochs show: int (default 100) Print period goal: float (default 0.01) The goal of train

`neurolab.train.` `train_cwta`(Train, epochs=500, goal=0.01, show=100, **kwargs)

  Conscience Winner Take All algorithm

  Support networks:
  | newc (Kohonen layer)
  Parameters : | input: array like (l x net.ci) train input patterns epochs: int (default 500) Number of train epochs show: int (default 100) Print period goal: float (default 0.01) The goal of train

### Train algorithms based on spipy.optimize

`neurolab.train.` `train_bfgs`(Train, epochs=500, goal=0.01, show=100, **kwargs)

  BroydenFletcherGoldfarbShanno (BFGS) method Using scipy.optimize.fmin_bfgs

  Support networks:
  | newff (multi-layers perceptron)
  Parameters : | input: array like (l x net.ci) train input patterns target: array like (l x net.co) train target patterns epochs: int (default 500) Number of train epochs show: int (default 100) Print period goal: float (default 0.01) The goal of train

`neurolab.train.` `train_cg`(Train, epochs=500, goal=0.01, show=100, **kwargs)

  Newton-CG method Using scipy.optimize.fmin_ncg

  Support networks:
  | newff (multi-layers perceptron)
  Parameters : | input: array like (l x net.ci) train input patterns target: array like (l x net.co) train target patterns epochs: int (default 500) Number of train epochs show: int (default 100) Print period goal: float (default 0.01) The goal of train

`neurolab.train.` `train_ncg`(Train, epochs=500, goal=0.01, show=100, **kwargs)

  Conjugate gradient algorithm Using scipy.optimize.fmin_ncg

  Support networks:
  | newff (multi-layers perceptron)
  Parameters : | input: array like (l x net.ci) train input patterns target: array like (l x net.co) train target patterns epochs: int (default 500) Number of train epochs show: int (default 100) Print period goal: float (default 0.01) The goal of train

### Train algorithms for LVQ networks

`neurolab.train.` `train_lvq`(Train, epochs=500, goal=0.01, show=100, **kwargs)

  LVQ1 train function

  Support networks:
  | newlvq
  Parameters : | input: array like (l x net.ci) train input patterns epochs: int (default 500) Number of train epochs show: int (default 100) Print period goal: float (default 0.01) The goal of train lr: float (defaults 0.01) learning rate adapt bool (default False) type of learning

### Delta rule

`neurolab.train.` `train_delta`(Train, epochs=500, goal=0.01, show=100, **kwargs)

  Train with Delta rule

  Support networks:
  | newp (one-layer perceptron)
  Parameters : | input: array like (l x net.ci) train input patterns epochs: int (default 500) Number of train epochs show: int (default 100) Print period goal: float (default 0.01) The goal of train lr: float (default 0.01) learning rate

## error: Error functions

Train error functions with derivatives

Example: |
    >>> msef = MSE()
    >>> x = np.array([[1.0, 0.0], [2.0, 0.0]])
    >>> msef(x)
    1.25
    >>> # calc derivative:
    >>> msef.deriv(x[0])
    array([ 1.,  0.])

class `neurolab.error.` `MAE`

  Mean absolute error function

  Parameters : | e: ndarray current errors: target - output
  Returns : | v: float Error value

  `deriv`(e)

    Derivative of SAE error function

    Parameters : | e: ndarray current errors: target - output
    Returns : | d: ndarray Derivative: dE/d_out

class `neurolab.error.` `MSE`

  Mean squared error function

  Parameters : | e: ndarray current errors: target - output
  Returns : | v: float Error value
  Example : |
      >>> f = MSE()
      >>> x = np.array([[1.0, 0.0], [2.0, 0.0]])
      >>> f(x)
      1.25

  `deriv`(e)

    Derivative of MSE error function

    Parameters : | e: ndarray current errors: target - output
    Returns : | d: ndarray Derivative: dE/d_out
    Example : |
        >>> f = MSE()
        >>> x = np.array([1.0, 0.0])
        >>> # calc derivative:
        >>> f.deriv(x)
        array([ 1.,  0.])

class `neurolab.error.` `SAE`

  Sum absolute error function

  Parameters : | e: ndarray current errors: target - output
  Returns : | v: float Error value

  `deriv`(e)

    Derivative of SAE error function

    Parameters : | e: ndarray current errors: target - output
    Returns : | d: ndarray Derivative: dE/d_out

class `neurolab.error.` `SSE`

  Sum squared error function

  Parameters : | e: ndarray current errors: target - output
  Returns : | v: float Error value

  `deriv`(e)

    Derivative of SSE error function

    Parameters : | e: ndarray current errors: target - output
    Returns : | d: ndarray Derivative: dE/d_out

## trans: Transfer functions

Transfer function with derivatives

Example: |
    >>> import numpy as np
    >>> f = TanSig()
    >>> x = np.linspace(-5,5,100)
    >>> y = f(x)
    >>> df_on_dy = f.deriv(x, y) # calc derivative
    >>> f.out_minmax    # list output range [min, max]
    [-1, 1]
    >>> f.inp_active    # list input active range [min, max]
    [-2, 2]

class `neurolab.trans.` `Competitive`

  Competitive transfer function

  Parameters : | x: ndarray Input array
  Returns : | y : ndarray may take the following values: 0, 1 1 if is a minimal element of x, else 0
  Example : |
      >>> f = Competitive()
      >>> f([-5, -0.1, 0, 0.1, 100])
      array([ 1.,  0.,  0.,  0.,  0.])
      >>> f([-5, -0.1, 0, -6, 100])
      array([ 0.,  0.,  0.,  1.,  0.])

class `neurolab.trans.` `HardLim`

  Hard limit transfer function

  Parameters : | x: ndarray Input array
  Returns : | y : ndarray may take the following values: 0, 1
  Example : |
      >>> f = HardLim()
      >>> x = np.array([-5, -0.1, 0, 0.1, 100])
      >>> f(x)
      array([ 0.,  0.,  0.,  1.,  1.])

  `deriv`(x, y)

    Derivative of transfer function HardLim

class `neurolab.trans.` `HardLims`

  Symmetric hard limit transfer function

  Parameters : | x: ndarray Input array
  Returns : | y : ndarray may take the following values: -1, 1
  Example : |
      >>> f = HardLims()
      >>> x = np.array([-5, -0.1, 0, 0.1, 100])
      >>> f(x)
      array([-1., -1., -1.,  1.,  1.])

  `deriv`(x, y)

    Derivative of transfer function HardLims

class `neurolab.trans.` `LogSig`

  Logarithmic sigmoid transfer function

  Parameters : | x: ndarray Input array
  Returns : | y : ndarray The corresponding logarithmic sigmoid values.
  Example : |
      >>> f = LogSig()
      >>> x = np.array([-np.Inf, 0.0, np.Inf])
      >>> f(x).tolist()
      [0.0, 0.5, 1.0]

  `deriv`(x, y)

    Derivative of transfer function LogSig

class `neurolab.trans.` `PureLin`

  Linear transfer function

  Parameters : | x: ndarray Input array
  Returns : | y : ndarray copy of x
  Example : |
      >>> import numpy as np
      >>> f = PureLin()
      >>> x = np.array([-100., 50., 10., 40.])
      >>> f(x).tolist()
      [-100.0, 50.0, 10.0, 40.0]

  `deriv`(x, y)

    Derivative of transfer function PureLin

class `neurolab.trans.` `SatLin`

  Saturating linear transfer function

  Parameters : | x: ndarray Input array
  Returns : | y : ndarray 0 if x < 0; x if 0 <= x <= 1; 1 if x >1
  Example : |
      >>> f = SatLin()
      >>> x = np.array([-5, -0.1, 0, 0.1, 100])
      >>> f(x)
      array([ 0. ,  0. ,  0. ,  0.1,  1. ])

  `deriv`(x, y)

    Derivative of transfer function SatLin

class `neurolab.trans.` `SatLinPrm`(k=1, out_min=0, out_max=1)

  Linear transfer function with parametric output May use instead Satlin and Satlins

  Init Parameters:
  | k: float default 1 output scaling out_min: float default 0 minimum output out_max: float default 1 maximum output
  Parameters : | x: ndarray Input array
  Returns : | y : ndarray with default values 0 if x < 0; x if 0 <= x <= 1; 1 if x >1
  Example : |
      >>> f = SatLinPrm()
      >>> x = np.array([-5, -0.1, 0, 0.1, 100])
      >>> f(x)
      array([ 0. ,  0. ,  0. ,  0.1,  1. ])
      >>> f = SatLinPrm(1, -1, 1)
      >>> f(x)
      array([-1. , -0.1,  0. ,  0.1,  1. ])

  `deriv`(x, y)

    Derivative of transfer function SatLin

class `neurolab.trans.` `SatLins`

  Symmetric saturating linear transfer function

  Parameters : | x: ndarray Input array
  Returns : | y : ndarray -1 if x < -1; x if -1 <= x <= 1; 1 if x >1
  Example : |
      >>> f = SatLins()
      >>> x = np.array([-5, -1, 0, 0.1, 100])
      >>> f(x)
      array([-1. , -1. ,  0. ,  0.1,  1. ])

  `deriv`(x, y)

    Derivative of transfer function SatLins

class `neurolab.trans.` `SoftMax`

  Soft max transfer function

  Parameters : | x: ndarray Input array
  Returns : | y : ndarray range values [0, 1]
  Example : |
      >>> from numpy import floor
      >>> f = SoftMax()
      >>> floor(f([0, 1, 0.5, -0.5]) * 10)
      array([ 1.,  4.,  2.,  1.])

class `neurolab.trans.` `TanSig`

  Hyperbolic tangent sigmoid transfer function

  Parameters : | x: ndarray Input array
  Returns : | y : ndarray The corresponding hyperbolic tangent values.
  Example : |
      >>> f = TanSig()
      >>> f([-np.Inf, 0.0, np.Inf])
      array([-1.,  0.,  1.])

  `deriv`(x, y)

    Derivative of transfer function TanSig

## init: Initializing functions

Functions of initialization layers

class `neurolab.init.` `InitRand`(minmax, init_prop)

  Initialize the specified properties of the layer random numbers within specified limits

`neurolab.init.` `init_rand`(layer, min=-0.5, max=0.5, init_prop='w')

  Initialize the specified property of the layer random numbers within specified limits

  Parameters : | layer: Initialized layer min: float (default -0.5) minimum value after the initialization max: float (default 0.5) maximum value after the initialization init_prop: str (default ‘w’) name of initialized property, must be in layer.np

`neurolab.init.` `init_zeros`(layer)

  Set all layer properties of zero

`neurolab.init.` `initnw`(layer)

  Nguyen-Widrow initialization function

`neurolab.init.` `initwb_reg`(layer)

  Initialize weights and bias in the range defined by the activation function (transf.inp_active)

`neurolab.init.` `midpoint`(layer)

  Sets weight to the center of the input ranges

### Navigation

- index
- modules |
- next |
- previous |
- NeuroLab 0.2.2 documentation »