# tfHuber

**T**uning-**F**ree **Huber** Estimation and Regression

## Description

This package implements the Huber mean estimator, Huber covariance matrix estimation and adaptive Huber regression estimators efficiently. For all these methods, the robustification parameter *&tau;* is calibrated via a tuning-free principle.

Specifically, for Huber regression, assume the observed data vectors (*Y*, *X*) follow a linear model *Y = &theta;<sub>0</sub> + X &theta; + &epsilon;*, where *Y* is an *n*-dimensional response vector, *X* is an *n* &times; *d* design matrix, and *&epsilon;* is an *n*-vector of noise variables whose distributions can be asymmetric and/or heavy-tailed. The package computes the standard Huber's *M*-estimator when *d < n* and the Huber-Lasso estimator when *d > n*. The vector of coefficients *&theta;* and the intercept term *&theta;<sub>0</sub>* are estimated successively via a two-step procedure. See [Wang et al., 2020](https://www.math.ucsd.edu/~wez243/tfHuber.pdf) for more details of the two-step tuning-free framework.


## Functions

There are four functions in this package: 

* `mean`: Huber mean estimation.   
*X*: A list object.  
*grad*: Using gradient descent or weighed least square to optimize the mean, default *True*  
*tol*: Tolerance of the error, default *1e-5*.  
*max_iter*: Maximum times of iteration, default *500*.
* `cov`: Huber covariance matrix estimation.  
*X*: A list object.  
*type*: If set to `"element"`, apply adaptive huber M-estimation; or if set to `"spectrum"`, apply spectrum-wise truncated estimation. Default `"element" `  
*pairwise*: Pairwise covariance or difference based covariance. Default *false*.   
*tol*: Tolerance of the error, default *1e-5*.  
*max_iter*: Maximum times of iteration, default *500*.
* `one_step_reg/two_step_reg`: One or two step adaptive Huber regression.   
*X, Y*: List objects.  
*grad*: Using gradient descent or weighed least square to optimize the mean, default *True*.  
*tol*: Tolerance of the error, default *1e-5*.  
*max_iter*: Maximum times of iteration, default *500*.



## Examples 


We present an example of adaptive Huber methods. Here we generate data from a linear model *Y = X &theta; + &epsilon;*, where *&epsilon;* follows a normal distribution, and estimate the intercept and coefficients by tuning-free Huber regression.

```python
import numpy
import tfhuber
X = np.random.uniform(-1.5, 1.5, (10000, 10)).tolist()
Y = intercept + np.dot(X, beta) + np.random.normal(0, 1, 10000)
Y = Y.tolist()

mu, tau, iteration = tf.mean(Y, grad=True, tol=1e-5, max_iter=500)
cov = tf.cov(X, method=1, tol=1e-5, max_iter=500)

theta, tau, iteration = tf.one_step_reg(X, Y, grad=True, z=0, tol=1e-5, max_iter=500)
theta, tau, iteration = tf.two_step_reg(X, Y, grad=True, tol=1e-5, consTau=1.345, max_iter=500)
```

## License
GPL (>= 3)

## Author(s)

Yifan Dai <yifandai@yeah.net>, Qiang Sun <qsun.ustc@gmail.com>

## References

Guennebaud, G. and Jacob B. and others. (2010). Eigen v3. [Website](http://eigen.tuxfamily.org)

Ke, Y., Minsker, S., Ren, Z., Sun, Q. and Zhou, W.-X. (2019). User-friendly covariance estimation for heavy-tailed distributions. *Statis. Sci.* **34** 454-471, [Paper](https://projecteuclid.org/euclid.ss/1570780979)

Pan, X., Sun, Q. and Zhou, W.-X. (2019). Nonconvex regularized robust regression with oracle properties in polynomial time. Preprint. [Paper](https://arxiv.org/abs/1907.04027)

Sanderson, C. and Curtin, R. (2016). Armadillo: A template-based C++ library for linear algebra. *J. Open Source Softw.* **1** 26. [Paper](http://conradsanderson.id.au/pdfs/sanderson_armadillo_joss_2016.pdf)

Sun, Q., Zhou, W.-X. and Fan, J. (2020). Adaptive Huber regression. *J. Amer. Stat. Assoc.* **115** 254-265. [Paper](https://doi.org/10.1080/01621459.2018.1543124)

Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. *J. R. Stat. Soc. Ser. B. Stat. Methodol.* **58** 267–288. [Paper](https://www.jstor.org/stable/2346178?seq=1#metadata_info_tab_contents)

Wang, L., Zheng, C., Zhou, W. and Zhou, W.-X. (2020). A new principle for tuning-free Huber regression. *Stat. Sinica* to appear. [Paper](https://www.math.ucsd.edu/~wez243/tfHuber.pdf)