## PIPA Development Guidelines

*Enforced by Black Linter for Python Code*  


### **1. Code Quality Standards**  
**1.1 Remove Redundant Content**  
- **Unused Comments**: Delete obsolete or irrelevant comments (e.g., `# TODO: fix this later` if unresolved).  
- **Unused Files**: Remove files no longer referenced in the project.  

**1.2 Avoid Magic Numbers**  
- Replace hardcoded numeric literals with **named constants** or **enum values** for clarity.  
  ```python  
  # ❌ Bad: Magic numbers  
  if account == 1:  
      ...  
  elif account == 2:  
      ...  

  # ✅ Good: Semantic constants  
  ACCOUNT_TYPE_ADMIN = 1  
  ACCOUNT_TYPE_USER = 2  

  if account == ACCOUNT_TYPE_ADMIN:  
      ...  
  ```  

**1.3 Eliminate Hardcoding**  
- **Configuration Isolation**: Externalize sensitive or environment-specific data (e.g., DB credentials, URLs) into configuration files or environment variables.  
  ```yaml  
  # ❌ Bad: Hardcoded credentials  
  datasource:  
    url: jdbc:mysql://192.168.1.1:3306/demo  
    username: root  
    password: 123456  

  # ✅ Good: Environment variables  
  datasource:  
    url: jdbc:mysql://${DB_HOST}:${DB_PORT}/demo  
    username: ${DB_USERNAME}  
    password: ${DB_PASSWORD}  
  ```  

**1.4 Refactor Legacy Code**  
- **Avoid Duplication**: Do not create redundant functions like `get()`, `get1()`, `get2()` for the same purpose. Use polymorphism or refactoring.  
  ```python  
  # ❌ Bad: Duplicated logic  
  def get():  
      ...  
  def get1():  
      ...  

  # ✅ Good: Unified function  
  def retrieve_data(source: str) -> dict:  
      ...  
  ```  


### **2. Naming Conventions**  
**2.1 Domain/URL/Code Repository**  
- **Semantic and Consistent**: Use clear terms (e.g., `api`, `admin`, `docs`).  
- **Hyphens Over Underscores**: Prefer hyphens for readability (e.g., `https://api.pipa.com/v1/users`).  
- **No Uppercase or Pinyin**: Avoid uppercase letters and non-semantic pinyin (e.g., `https://www.apple.com/shop/buy-iphone/iphone-15-pro`).  


### **3. Git Workflow**  
**3.1 Commit Practices**  
- **Atomic Commits**: Each commit should address a single logical change.  
- **Commitizen for Semantics**: Use `cz c` for standardized commit messages (e.g., `feat: add user authentication`).  
  ```bash  
  pip install commitizen  
  git add .  
  cz c  
  ```  

**3.2 Branching & Merging**  
- **Small, Frequent Merges**: Use CI/CD pipelines for rapid feedback.  
- **Branch Naming**: Align with issue IDs (e.g., `feature/123-add-payment`).  
- **Rebase Over Merge**:  
  - Use `git rebase` to maintain a linear history.  
  - Delete temporary branches immediately after merging.  
  - **Squash Commits**: For multi-step features, use `--squash` to condense history.  

**3.3 Prohibited Actions**  
- **No Direct Main Branch Pushes**: All changes must go through MR/PR reviews.  
- **No History Rewriting**: Use `git revert` for rollbacks, not `git reset` or `--force`.  


### **4. Configuration & Dependencies**  
**4.1 .gitignore Rules**  
- Exclude build artifacts, dependencies, and IDE files:  
  ```gitignore  
  /node_modules  
  /vendor  
  *.pyc  
  .env  
  ```  

**4.2 External Resources**  
- **CDN for Static Assets**: Avoid bundling third-party libraries (e.g., Bootstrap).  
- **Artifact Repositories**: Store compiled files in a CI/CD artifact registry (e.g., `index.u98ashj.css`).  


### **5. Versioning**  
**5.1 Semantic Versioning (SemVer)**  
- **Format**: `MAJOR.MINOR.PATCH`  
  - **MAJOR**: Breaking changes (e.g., API deprecation).  
  - **MINOR**: Backward-compatible features (e.g., new endpoints).  
  - **PATCH**: Bug fixes (e.g., security patches).  


### **6. Black Linter Configuration**  
**6.1 Enforce Formatting**  
- **Line Length**: 88 characters (default in Black).  
- **Quotes**: Use double quotes (`"`) consistently.  
- **Imports**: Sort imports alphabetically with `isort`.  

**6.2 Integration**  
- Add to `pyproject.toml`:  
  ```toml  
  [tool.black]  
  line-length = 88  
  target-version = ['py310']  
  ```  

### **7. Compliance & Enforcement**  
- **Pre-commit Hooks**: Automate Black formatting and linter checks.  
- **CI/CD Pipeline**: Fail builds on style violations or unformatted code.  
- **Code Reviews**: Require MR/PR approvals for all changes.  

## Roadmap

* **Data Collection**: Create a module for collecting data from various sources such as system logs, performance counters, monitoring tools, etc.
  * **Script Generation**: Create a module for generating performance data collection scripts to reduce noise generated by the Python runtime.
* **Data Parsing**: Create a module for parsing the collected raw data and converting it into a format that is easy to process and analyze.
* **Data Processing**: Create a module for further processing the parsed data, such as aligning data, segmenting data, etc.
  * **Calculate Performance Metrics**: Calculate various performance metrics based on the collected and processed data.
  * **Data Visualization**: Create a module for visualizing the processed data and performance metrics, allowing users to understand the performance data more intuitively.
* **Performance Analysis**: Create a module for in-depth analysis of performance data, such as bottleneck detection, performance prediction, etc.
* **Others**
  * **Cross-Platform Support**: Ensure the tool can run on multiple platforms such as x86_64, ARM, RISC-V, etc.
  * **Develop User Interface**: Develop a user interface to make the tool more user-friendly.
  * **Write Documentation**: Write detailed user documentation and API documentation to help users understand how to use the tool.
  * **Testing**: Write and execute test cases to ensure all parts of the tool work correctly.
  * **Continuous Integration/Continuous Deployment (CI/CD)**: Set up CI/CD workflows to automate the testing and deployment process.
  * **Performance Optimization**: Optimize the performance of the tool to ensure it maintains good performance when handling large amounts of data.


## TODO

### Version 0.0.6

* [x] Implement data parsing of sar data
* [x] Implement the script generation for global performance data collection
* [x] Fix script generation initiated by pipa, optimizing default parameters
* [x] Implement parallel parsing of sar data / perf-stat
* [x] Implement data parsing of perf-report data
* [x] Implement basic data processing capabilities
* [x] Implement basic data visualization: Export CPI/CPU Util/CPU Freq over time figures

### Version 0.0.7

* [x] Use ps to get process information in PIPA-SHU
* [x] Improve `run-by-user` and set it as default option
* [x] Fix bugs in perf-stat parsing
* [x] Export OS & HW configuration after parsing
* [x] Improve perf events used in perf-stat
* [x] Implement script generation by parsing configuration files
* [x] Set version in the code

### Version 0.0.8

* [x] Extend the metrics in plot_disk_usage
* [x] Implement pipa-version command
* [x] Unified interface specification for robustness
* [x] Support data collection by `emon`
* [x] Integrate key metrics into Grafana. Implement Client/Server with `grpc`.
* [x] Integrate export `perf.data` to `SQLite3`

### Version 0.0.9

* [x] Improve PIPA CLI
* [x] Implement tidy table
* [x] Provides combinations of different performance data collection tools
* [x] Set up format check tools and configuration
* [x] Fix bugs of pipa-generate

### Version 0.0.10

* [x] Fix tidy data
* [x] Improve Grafana table format
* [x] Integrate the automated documentation generation process into CI
* [x] Implement dumping PIPASHU overview data to a file
* [x] Fix PIPA-upload with perf-record data
* [x] Improve unit tests
* [x] Implement PIPA-dump

### Version 0.0.11

* [x] Improve comments
* [x] Support perf-script with call trace
* [x] Build call graph based on perf-script data

### Version 0.0.12

* [x] Implement EPM for Call Graph
* [x] Rewrite quick-start.md
* [x] No longer support py3.9 and older
* [x] Fixed bugs

### Version 0.0.13

* [x] Add buildid debug dir support
* [x] Add network statistics parse in sar
* [x] Correct the checking when perf.script does not exist
* [x] Add coverage tests and deploy in CI

### Version 0.0.14

* [x] Add perf stat parse for aggregation mode
* [x] Decoupled Call Graph
* [x] Implement PIPAD download
* [x] Improve documentation
* [x] Enhance each sar columns' indexing
* [x] Use range instead of core numbers
* [x] Add pipa archive
* [x] Add aggregation mode in CPU Utils
* [x] perf script parse add filter by commands / time window
* [ ] Improve unit tests coverage
