Non-Linear Least-Square Minimization for Python — Least-Squares Minimization with Constraints for Python

### Navigation

- index
- modules |
- next |
- Least-Squares Minimization with Constraints for Python »

# Non-Linear Least-Square Minimization for Python

The lmfit Python package provides a simple, flexible interface to non-linear optimization or curve fitting problems. The default optimization method used is the Levenberg-Marquardt algorithm from scipy.optimize.leastsq, but other optimization procedures such as Nelder-Mead downhill simplex, Powell's method, COBYLA, Sequential Least Squares, and a few other optimization approaches available from scipy.optimize can also be used.

For any optimization problem, the programmer must write an objective function that takes a set of values for the variables in the fit, and produces either a scalar value to be minimized or a residual array to be minimized in the least-squares sense. Generally, the set of variables can all be varied freely by the algorithm calling the objective functions, though some algorithms allow some sorts of constraints or bounds to be placed on the values.

With the lmfit package, one writes an objective function in terms of a set of Parameter objects, where a Parameter has the following properties:

> - Parameters have values that can be fixed or floated in the fit.
> - Upper and/or lower bounds can be placed on Parameter values.
> - Parameters can be written as algebraic expressions of other Parameters. These values will be re-evaluated at each step in the fit, giving a a simple and very flexible approach to constraining fit variables.

The principle advantage of using Parameters instead of fit variables is that the objective function does not have to be rewritten for a change in what is varied or what constraints are placed on the fit. A programmer can write a general model that encapsulates the phenomenon to be optimized, and then allow a user of the model to change what is varied and what constraints are placed on the model. The ability to easily change whether a Parameter is floated or fixed also allows one to easily test the significance of certain Parameters to the fitting model. A second advantage is that Parameters can be used and objective function using them can be given to a number of fitting algorithms without any change.

By default, lmfit uses and the Levenberg-Marquardt minimization algorithm from MINPACK-1 as implemented in scipy.optimize.leastsq. This method is by far the most tested and best support method in lmfit, and much of this document assumes this algorithm is used unless explicitly stated. An important point for many scientific analysis is that this is only method that automatically estimates uncertainties and correlations between fitted variables from the covariance matrix used in the fit.

A few other optimization routines are also supported, including Nelder-Mead simplex downhill method as implemented in scipy.optimize.fmin, and several others from scipy.optimize. Some methods, including the L-BFGS (limited memory Broyden-Fletcher-Goldfarb-Shanno) algorithm as implemented in scipy.optimize.l_bfgs_b and the simulated annealing algorithm as implemented in scipy.optimize.anneal are implemented, but appear to not work very well. In their native form, some of these methods setting upper or lower bounds on parameters, or adding constraints on fitted variables. By using Parameter objects, lmfit allows bounds and constraints for all of these methods, and makes it easy to swap between methods.

Finally, because this approach of using the covariance matrix to determine uncertainties is sometimes questioned (and sometimes rightly so), lmfit supports methods to do a brute force search of the confidence intervals and correlations for sets of parameters.

lmfit and this document are a work in progress.

- Downloading and Installation
  - Prerequisites
  - Downloads
  - Development Version
  - Installation
  - Acknowledgements
  - License
- Getting started with Non-Linear Least-Squares Fitting
  - Using `Parameters` instead of Variables
  - The `Parameter` class
  - The `Parameters` class
  - Simple Example
- Performing Fits, Analyzing Outputs
  - The `minimize()` function
  - Writing a Fitting Function
  - Choosing Different Fitting Methods
  - Goodness-of-Fit and estimated uncertainty and correlations
  - Using the `Minimizer` class
- Calculation of confidence intervals
  - Method used for calculating confidence intervals
  - A basic example
  - An advanced example
  - Documentation of methods
- Bounds Implementation
- Using Mathematical Constraints
  - Overview
  - Supported Operators, Functions, and Constants
  - Using Inequality Constraints
  - Advanced usage of Expressions in lmfit

### Navigation

- index
- modules |
- next |
- Least-Squares Minimization with Constraints for Python »