# Model Robustness to Noise in Breast Cancer Wisconsin Dataset: A Comparative Analysis of Ensemble and Linear Methods

## Abstract

This study investigates the robustness of ensemble methods (Random Forest, Gradient Boosting) compared to linear models (Logistic Regression) when subjected to varying levels of noise in the Breast Cancer Wisconsin dataset. We applied Gaussian noise at levels of 0%, 10%, 20%, and 30% of feature standard deviation to evaluate model degradation. Performance was measured using accuracy, precision, recall, F1-score, and AUC-ROC metrics through 5-fold cross-validation. Results showed that Logistic Regression maintained surprisingly consistent performance across all noise levels, while Random Forest exhibited the greatest performance degradation. These findings challenge the initial hypothesis that ensemble methods would demonstrate superior robustness to noise in this specific dataset, suggesting that the linear decision boundaries of Logistic Regression may be particularly well-suited for this classification problem even under noisy conditions.

## 1. Introduction

In machine learning applications, particularly in medical domains such as cancer diagnosis, data often contains noise that can impact model performance. Understanding which algorithms maintain reliability under noisy conditions is critical for developing robust clinical decision support systems.

The Breast Cancer Wisconsin dataset represents a binary classification task to distinguish between malignant and benign tumors based on cellular features extracted from digitized images of fine needle aspirates. This study addresses the following research question: Are ensemble methods (Random Forests, Gradient Boosting) more robust to added noise compared to linear models like Logistic Regression for a binary classification task on the Breast Cancer Wisconsin dataset?

Our initial hypothesis posited that ensemble methods would demonstrate a slower rate of performance degradation with increasing noise compared to Logistic Regression, indicating higher robustness to noise. This hypothesis was based on the theoretical strengths of ensemble methods, which combine multiple decision trees that can potentially compensate for noise-induced distortions through aggregation of predictions.

## 2. Methodology

### 2.1 Experimental Design

We designed a controlled experiment with the following variables:

**Independent Variables:**
- Model type: Logistic Regression, Random Forest, Gradient Boosting
- Noise level: 0%, 10%, 20%, 30% of feature standard deviation

**Dependent Variables:**
- Accuracy
- Precision
- Recall
- F1-Score
- AUC-ROC

**Constant Variables:**
- Dataset (Breast Cancer Wisconsin)
- Random seed (for reproducibility)
- Cross-validation methodology (5-fold)
- Data preprocessing approach
- Noise generation method (Gaussian)

### 2.2 Dataset

The Breast Cancer Wisconsin dataset contains 569 instances with 30 features computed from digitized images of breast mass fine needle aspirates. Each instance is labeled as malignant (212 samples) or benign (357 samples). Features represent characteristics of cell nuclei present in the images, including radius, texture, perimeter, area, smoothness, compactness, concavity, and symmetry.

### 2.3 Implementation Details

We implemented the experimental framework using Python with scikit-learn. The procedure for each experiment was as follows:

1. Load the Breast Cancer Wisconsin dataset
2. Split the data into features (X) and target (y)
3. Apply standard scaling to normalize the feature values
4. For each noise level (0%, 10%, 20%, 30%):
   a. Add Gaussian noise to the features proportional to their standard deviation
   b. For each model type (Logistic Regression, Random Forest, Gradient Boosting):
      i. Perform 5-fold cross-validation
      ii. Calculate performance metrics (accuracy, precision, recall, F1-score, AUC-ROC)
      iii. Record mean and standard deviation of metrics

The noise injection was implemented by adding random Gaussian noise with mean 0 and standard deviation equal to a percentage of each feature's standard deviation, ensuring consistent signal-to-noise ratio across features.

### 2.4 Execution Progress

The experiment was conducted in multiple partitions:
- Control group: Logistic Regression at 0% noise
- Partition 1: All models at 0% and 10% noise
- Partition 2: All models at 20% and 30% noise
- Partition 3: Additional tests for Gradient Boosting at 30% noise (though these results were not included in the final analysis)

We encountered initial challenges with the experimental implementation, particularly in ensuring consistent noise application and cross-validation splits across different model types. These issues were addressed by:
- Implementing a consistent random seed
- Validating the signal-to-noise ratio
- Ensuring identical cross-validation splits across all models and noise levels
- Adding visualization to verify noise impact on feature distributions

## 3. Results

### 3.1 Performance Across Noise Levels

The performance of each model across different noise levels is summarized in the following tables and visualized in the accompanying figures.

**Accuracy Results:**

| Model Type         | 0% Noise       | 10% Noise      | 20% Noise      | 30% Noise      |
|--------------------|----------------|----------------|----------------|----------------|
| Logistic Regression| 0.9737 ± 0.0166| 0.9754 ± 0.0129| 0.9719 ± 0.0129| 0.9701 ± 0.0142|
| Random Forest      | 0.9561 ± 0.0123| 0.9526 ± 0.0180| 0.9420 ± 0.0204| 0.9385 ± 0.0199|
| Gradient Boosting  | 0.9491 ± 0.0237| 0.9631 ± 0.0103| 0.9613 ± 0.0089| N/A            |

**Precision Results:**

| Model Type         | 0% Noise       | 10% Noise      | 20% Noise      | 30% Noise      |
|--------------------|----------------|----------------|----------------|----------------|
| Logistic Regression| 0.9683 ± 0.0290| 0.9704 ± 0.0211| 0.9676 ± 0.0213| 0.9649 ± 0.0215|
| Random Forest      | 0.9651 ± 0.0282| 0.9598 ± 0.0287| 0.9442 ± 0.0338| 0.9389 ± 0.0317|
| Gradient Boosting  | 0.9471 ± 0.0354| 0.9670 ± 0.0133| 0.9646 ± 0.0194| N/A            |

**Recall Results:**

| Model Type         | 0% Noise       | 10% Noise      | 20% Noise      | 30% Noise      |
|--------------------|----------------|----------------|----------------|----------------|
| Logistic Regression| 0.9916 ± 0.0112| 0.9916 ± 0.0112| 0.9888 ± 0.0105| 0.9888 ± 0.0105|
| Random Forest      | 0.9665 ± 0.0272| 0.9665 ± 0.0323| 0.9664 ± 0.0188| 0.9664 ± 0.0211|
| Gradient Boosting  | 0.9748 ± 0.0184| 0.9748 ± 0.0206| 0.9748 ± 0.0106| N/A            |

**F1-Score Results:**

| Model Type         | 0% Noise       | 10% Noise      | 20% Noise      | 30% Noise      |
|--------------------|----------------|----------------|----------------|----------------|
| Logistic Regression| 0.9794 ± 0.0127| 0.9807 ± 0.0101| 0.9779 ± 0.0100| 0.9765 ± 0.0112|
| Random Forest      | 0.9651 ± 0.0097| 0.9624 ± 0.0143| 0.9546 ± 0.0153| 0.9519 ± 0.0150|
| Gradient Boosting  | 0.9602 ± 0.0179| 0.9707 ± 0.0083| 0.9694 ± 0.0067| N/A            |

**AUC-ROC Results:**

| Model Type         | 0% Noise       | 10% Noise      | 20% Noise      | 30% Noise      |
|--------------------|----------------|----------------|----------------|----------------|
| Logistic Regression| 0.9953 ± 0.0055| 0.9953 ± 0.0050| 0.9949 ± 0.0059| 0.9943 ± 0.0052|
| Random Forest      | 0.9886 ± 0.0082| 0.9897 ± 0.0067| 0.9869 ± 0.0073| 0.9858 ± 0.0073|
| Gradient Boosting  | 0.9927 ± 0.0041| 0.9936 ± 0.0043| 0.9863 ± 0.0101| N/A            |

![Accuracy vs Noise](accuracy_vs_noise.png)
*Figure 1: Accuracy performance across different noise levels for each model. Note that Logistic Regression maintains consistent performance while Random Forest shows gradual degradation.*

![Precision vs Noise](precision_vs_noise.png)
*Figure 2: Precision performance across different noise levels for each model. Gradient Boosting shows some improvement at moderate noise levels before potential degradation.*

![Recall vs Noise](recall_vs_noise.png)
*Figure 3: Recall performance across different noise levels for each model. All models maintain relatively stable recall despite increasing noise.*

![F1-Score vs Noise](f1_score_vs_noise.png)
*Figure 4: F1-Score performance across different noise levels for each model. Logistic Regression maintains the highest overall F1-Score.*

![AUC-ROC vs Noise](auc-roc_vs_noise.png)
*Figure 5: AUC-ROC performance across different noise levels for each model. All models demonstrate strong ROC AUC values with minimal degradation.*

### 3.2 Model Comparison at Fixed Noise Levels

The radar charts illustrate a comparative view of model performance across multiple metrics at fixed noise levels.

![Radar Chart at 0% Noise](radar_chart_noise_0.png)
*Figure 6: Radar chart comparing model performance across all metrics at 0% noise level. Logistic Regression shows strong overall performance across most metrics.*

![Radar Chart at 10% Noise](radar_chart_noise_10.png)
*Figure 7: Radar chart comparing model performance across all metrics at 10% noise level. Gradient Boosting shows improvement in some metrics compared to its baseline.*

![Radar Chart at 20% Noise](radar_chart_noise_20.png)
*Figure 8: Radar chart comparing model performance across all metrics at 20% noise level. Logistic Regression continues to demonstrate robust performance despite increased noise.*

### 3.3 Performance Degradation Analysis

![Performance Degradation](performance_degradation.png)
*Figure 9: Relative performance degradation for each model type as noise increases. This visualization highlights that Random Forest exhibits the greatest relative degradation, contrary to the initial hypothesis.*

## 4. Discussion

### 4.1 Performance Robustness Analysis

Our results paint an interesting picture of model robustness to noise that challenges our initial hypothesis. Key findings include:

1. **Logistic Regression Showed Remarkable Robustness**: Contrary to our hypothesis, Logistic Regression demonstrated exceptional stability across all noise levels (0-30%). Its accuracy only decreased from 97.37% to 97.01%, representing just a 0.36 percentage point drop despite significant noise injection.

2. **Random Forest Showed Greatest Degradation**: Random Forest exhibited the largest relative performance drop, with accuracy decreasing from 95.61% at 0% noise to 93.85% at 30% noise (a 1.76 percentage point reduction), suggesting it was most sensitive to the injected noise.

3. **Gradient Boosting Showed Mixed Behavior**: At moderate noise levels (10%), Gradient Boosting actually showed slight improvement in performance compared to its baseline. While lacking data at 30% noise, its performance at 20% noise remained relatively stable, with only minimal degradation from its 10% noise levels.

These findings suggest that for the Breast Cancer Wisconsin dataset, Logistic Regression's simple linear decision boundary may be particularly well-suited to the underlying data structure, making it less susceptible to Gaussian noise. The fact that all models maintained high performance (above 93% accuracy) even at 30% noise indicates that the signal in this dataset is robust and well-separable.

### 4.2 Unexpected Results

Several unexpected patterns emerged that warrant further exploration:

1. **Performance Improvement with Noise**: For both Logistic Regression and Gradient Boosting, there were instances where performance metrics slightly improved with the addition of moderate noise (10%). This counter-intuitive result could be attributed to:
   - The noise potentially acting as a form of regularization, preventing overfitting
   - The specific random seed used may have generated noise that coincidentally improved certain decision boundaries
   - The high baseline performance leaves little room for improvement, making small fluctuations statistically negligible

2. **High Baseline Performance**: All models achieved very high baseline performance (>94.9% accuracy), indicating that the Breast Cancer Wisconsin dataset presents a relatively straightforward classification task. This high initial performance may limit the observable impact of noise, as there is a ceiling effect on potential improvement.

## 5. Conclusion and Future Work

### 5.1 Conclusion

This study evaluated the robustness of Logistic Regression, Random Forest, and Gradient Boosting models to increasing levels of Gaussian noise in the Breast Cancer Wisconsin dataset. Our results challenge the initial hypothesis that ensemble methods would demonstrate greater robustness to noise than linear models. Instead, we found that Logistic Regression maintained remarkably stable performance across all noise levels, while Random Forest showed the greatest relative performance degradation.

These findings suggest that the choice of model for noise-robust applications should consider the specific characteristics of the dataset and the type of noise present, rather than making general assumptions about model classes. For the Breast Cancer Wisconsin dataset, Logistic Regression appears to provide an optimal balance of high performance and robustness to feature noise.

### 5.2 Future Work

Several directions for future research emerge from this study:

1. **Extended Noise Levels**: Test with higher noise levels (40%, 50%, etc.) to identify potential breakdown points for each model type.

2. **Alternative Noise Types**: Investigate model robustness to different types of noise beyond Gaussian, such as salt-and-pepper noise, label flipping, or targeted feature corruption.

3. **Hyperparameter Tuning**: Explore whether optimized hyperparameters for each model can improve robustness to noise.

4. **Feature Importance Analysis**: Analyze how noise affects feature importance rankings in different models, particularly for ensemble methods.

5. **Statistical Significance Testing**: Conduct formal statistical tests to determine whether the observed differences in robustness are statistically significant.

6. **Model Complexity Analysis**: Investigate the relationship between model complexity and noise robustness to determine if simpler models are inherently more robust in certain data contexts.

7. **Completion of Missing Data**: Complete testing for Gradient Boosting at 30% noise to provide a more comprehensive comparison across all noise levels.

## 6. Appendices

### 6.1 Experimental Configuration Details

- **Dataset**: Breast Cancer Wisconsin (569 samples, 30 features, 2 classes)
- **Cross-validation**: 5-fold stratified cross-validation
- **Random seed**: Fixed for reproducibility across all experiments
- **Noise generation**: Gaussian noise with mean 0 and standard deviation proportional to feature standard deviation
- **Model configurations**: Default scikit-learn parameters used for all models
- **Performance metrics**: Accuracy, precision, recall, F1-score, and AUC-ROC

### 6.2 Raw Result Directory

- Control experiment results: `results_logistic_regression_noise_0.json`
- Partition 1 results: `results_experimental_group_partition_1.json`
- Partition 2 results: `results_experimental_group_partition_2.json`
- Performance visualizations: Various PNG files showing performance trends and comparisons

### 6.3 Implementation Details

The experiment was implemented using Python with the scikit-learn library. Standard preprocessing techniques were applied, including feature standardization before noise injection. All experiments were conducted with identical cross-validation splits to ensure fair comparison.