# Changelog
## Unreleased
### Added

### Changed

### Deprecated

### Removed

### Fixed

### Security

## [`v0.4.4` (2023-11-23)](https://github.com/anupli/running-ng/releases/tag/v0.4.4)
### Fixed
#### Benchmark Suites
- `DaCapo` correctly accepts the `23.11` `release` specified in `dacapo.yml`.

## [`v0.4.3` (2023-11-20)](https://github.com/anupli/running-ng/releases/tag/v0.4.3)
### Added
#### Base Configurations
- DaCapo 23.11-Chopin available as `dacapochopin`. Please use `dacapochopin_jdk9`, `dacapochopin_jdk11`, `dacapochopin_jdk17`, and `dacapochopin_jdk21` modifiers for JDK 9, 11, 17, and 21 respectively when you use this suite with these JDK versions.
- Temurin 21

### Changed
#### Base Configurations
- Environment variables are expanded when resolving paths of runtimes and benchmark suites.
- `--add-exports java.base/jdk.internal.ref=ALL-UNNAMED` is no longer automatically added when running DaCapo benchmarks on >= JDK 9. This approach doesn't scale now we have more workarounds specific to different JDK versions. This is also too opaque and not clear how it's implemented. New modifiers are introduced to address this issue.

#### Modifiers
- `EnvVar` `val` is expanded using the outside environment prior to benchmark execution.

### Deprecated
- Deprecating Python 3.7 support for users. Python 3.7 was last released on June 6, 2023 (3.7.17), and no new release has been made since.

### Removed
- Dropping Python 3.7 support for developers (NOT users). pytest 7.4+ requires at least Python 3.8 (still supported by Ubuntu 20.04 LTS).

## [`v0.4.2` (2023-09-10)](https://github.com/anupli/running-ng/releases/tag/v0.4.2)
### Changed
- All `Modifier` instances now supports `includes` for only attaching them to certain benchmarks

### Fixed
#### Runtimes
- `D8` now detects new JavaScript OOM error pattern.

### Security

## [`v0.4.1` (2023-08-22)](https://github.com/anupli/running-ng/releases/tag/v0.4.1)
### Fixed
#### Commands
- `runbms`: apply modifiers in the config file.
- `minheap`: apply modifiers in the config file.

## [`v0.4.0` (2023-08-17)](https://github.com/anupli/running-ng/releases/tag/v0.4.0)
### Added
#### Modifiers
- `JuliaArg`

#### Runtimes
- `JuliaMMTK`
- `JuliaStock`

#### Benchmark Suites
- `JuliaGCBenchmarks`

#### Commands
- `runbms` gains an extra argument, `--skip-log-compression`, to skip compressing log files with `gzip`.

### Changed
#### Base Configurations
- `runbms`: don't sync to `squirrel.moma` for the default `runbms.yml`. **moma machine users should use `runbms-anu.yml` for the old behaviour.**

### Fixed
- Gracifully handle empty modifiers strings in configs, such as `openjdk7||foobar`.

#### Benchmark Suites
- DaCapo specific workarounds are now handled by the `DaCapo` class rather than the `JavaBenchmark` class to avoid confusions.

## [`v0.3.9` (2023-08-02)](https://github.com/anupli/running-ng/releases/tag/v0.3.9)
### Fixed
#### Benchmark Suites
- `DaCapo`: don't explicitly pass `-s default` to DaCapo unless the user requests so by setting the `size` key of `DaCapo` or overriding the sizes for individual benchmarks using the [benchmark specification syntax](./references/suite.md#benchmark-specification). This is so that users can override the size via `ProgramArg`.

## [`v0.3.8` (2023-02-21)](https://github.com/anupli/running-ng/releases/tag/v0.3.8)
### Changed
#### Commands
- `runbms`: companion programs are now expected to self-terminate.

## [`v0.3.7` (2023-02-14)](https://github.com/anupli/running-ng/releases/tag/v0.3.7)
### Fixed
#### Commands
- `runbms`: better heuristics to detect whether a host is in the moma subnet.

## [`v0.3.6` (2023-01-16)](https://github.com/anupli/running-ng/releases/tag/v0.3.6)
### Added
#### Base Configurations
- DaCapo Chopin Snapshot-6e411f33

### Fixed
- Fixed type annotations in untyped functions and make `Optional`s explicit.

## [`v0.3.5` (2022-10-13)](https://github.com/anupli/running-ng/releases/tag/v0.3.5)
### Changed
#### Commands
- `runbms`: when a companion program exits with a non-zero code, a warning is generated instead of an exception to prevent stopping the entire experiment.

## [`v0.3.4` (2022-10-13)](https://github.com/anupli/running-ng/releases/tag/v0.3.4)
### Fixed
#### Commands
- `runbms`: fix the file descriptor leak when running benchmarks with companion programs.

## [`v0.3.3` (2022-10-12)](https://github.com/anupli/running-ng/releases/tag/v0.3.3)
### Changed
#### Commands
- `runbms` prints out the logged in users when emitting warnings when the machine has more than one logged in users.

### Fixed
#### Modifiers
- `Companion`: skip value options expansion if no value option is provided to avoid interpreting bpftrace syntax as replacement fields.

## [`v0.3.2` (2022-10-12)](https://github.com/anupli/running-ng/releases/tag/v0.3.2)
### Added

#### Modifiers
- `Companion`

## [`v0.3.1` (2022-09-18)](https://github.com/anupli/running-ng/releases/tag/v0.3.1)
### Added
#### Base Syntax
- Use the `$RUNNING_NG_PACKAGE_DATA` environment variable to refer to base configurations shipped with running-ng, such as `$RUNNING_NG_PACKAGE_DATA/base/runbms.yml`, regardless how you installed runnin-ng.
#### Benchmark Suites
- `DaCapo` gains an extra key `companion` to facilitate eBPF tracing programs.

### Changed
- Overhauled Python packaging with PEP 517
- `zulip` is now an optional Python dependency. Use `pip install running-ng[zulip]` if you want to use the `Zulip` `runbms` plugin.

### Removed
- Dropping Python 3.6 support for users.
#### Base Configurations
- Removing AdoptOpenJDK from the base configuration files. AdoptOpenJDK is now replaced by Temurin.

## [`v0.3.0` (2022-03-19)](https://github.com/anupli/running-ng/releases/tag/v0.3.0)
### Added
#### Modifiers
- `JVMClasspathAppend`
- `JVMClasspathPrepend`

#### Benchmark Suites
- `SPECjvm98`

### Changed
#### Modifiers
- `JVMClasspath` is now an alias of `JVMClasspathAppend`. This is backward compatible.

#### Commands
- `runbms` prints out the version number of `running-ng` in log files.

### Deprecated
- Deprecating Python 3.6 support for users. Python 3.6 will NOT be supported once moma machines are upgraded to the latest Ubuntu LTS.

### Removed
- Dropping Python 3.6 support for developers (NOT users). pytest 7.1+ requires at least Python 3.7.

## [`v0.2.2` (2022-03-07)](https://github.com/anupli/running-ng/releases/tag/v0.2.2)

### Fixed
#### Benchmark Suites
- `JavaBenchmarkSuite`: Some DaCapo benchmarks refers to internal classes (e.g., under `jdk.internal.ref`), and DaCapo implemented a workaround for this behaviour in the jar. However, since we are invoking DaCapo using `-cp` and the name of the main class, that workaround is bypassed. That workaround is now reimplemented in running-ng through an extra JVM argument `--add-exports`.

## [`v0.2.1` (2022-03-05)](https://github.com/anupli/running-ng/releases/tag/v0.2.1)
### Changed
#### Commands
- `runbms` now skips printing CPU frequencies if the system doesn't support it, e.g., when using Docker Desktop on Mac.

### Fixed
#### Benchmark Suites
- `BinaryBenchmarkSuite`: fixes missing parameter when constructing `BinaryBenchmark` due to a bug in previous refactoring

## [`v0.2.0` (2022-02-20)](https://github.com/anupli/running-ng/releases/tag/v0.2.0)
### Added
#### Base Configurations
- AdoptOpenJDK 16
- DaCapo Chopin Snapshot-29a657f, Chopin Snapshot-f480064
- Temurin 8, 11, 17
- SPECjbb 2015, 1.03

#### Commands
- `minheap` gains an extra key `attempts` (can be overridden by `--attempts`) so that crashes don't cause bogus minheap measurements.
- `minheap` stores results in a YAML file, which is also used to resume an interrupted execution.
- `minheap` prints the minheap values of the best config at the ends.
- `runbms` gains an extra argument, `--resume`, to resume an interrupted execution from a log folder.
- `runbms` gains an extra argument, `--workdir`, to override the default working directory.
- `runbms` adds more information of the environment to the log file, including the date, logged in users, system load, and top processes.
- `runbms` gains a callback-based plugin system, and an extra key `plugins` is added.
- `runbms` gains a plugin `CopyFile` to copy files from the working directory.
- `runbms` gains a plugin `Zulip`, which sends messages about the progress of the experiments, and warns about reservation expiration on moma machines.
- `runbms` outputs a warning message if more than one users are logged in.
- `runbms` uses uppercase letters if there are more than 26 configs.

#### Modifiers
- `ModifierSet`
- `Wrapper`
- `JSArg`

#### Runtimes
- `D8`
- `SpiderMonkey`
- `JavaScriptCore`
- `JVM` now detects OOM generated in the form of Rust panic from `mmtk-core`.

#### Benchmark Suites
- `DaCapo` gains an extra key `size`, which is used to specify the size of the input.
- `DaCapo` now allows individual benchmark to override the timing iteration, input size, and timeout of the suite.
- `SPECjbb2015`: basic support for running SPECjbb 2015 in composite mode.
- `Octane`: basic support for running Octane using Wenyu's wrapper script.

### Changed
#### Benchmark Suites
- The `minheap` key of `DaCapo` changes from a dictionary to a string. The string is used to look up `minheap_values`, which are collections of minheap values. This makes it easier to store multiple sets of minheap values for the same benchmark suite measured using different runtimes.

#### Base Syntax
- Whitespaces can be used in config strings for visual alignment. They are ignored when parsed.

#### Commands
- The `--slice` argument of `runbms` now accepts multiple comma-separated floating point numbers.

### Removed
#### Base Configurations
- DaCapo Chopin Snapshot-69a704e

### Fixed
#### Commands
- Resolving relative paths of runtimes before running. Otherwise, they would be resolved relative to the `runbms` working directory.
- Use the `BinaryIO` interface of file IO and interprocess communication to avoid invalid UTF-8 characters from crashing the script.
- Subprocesses now inherit environment variables from the the parent process.
- `minheap` now runs in a temporary working directory to avoid file-based conflicts between concurrent executions. Note that network-port-based conflicts can still happen.

## [`v0.1.0` (2021-08-09)](https://github.com/anupli/running-ng/releases/tag/v0.1.0)
Initial release.

### Added
#### Commands
- `fillin`
- `minheap`
- `runbms`

#### Modifiers
- `JVMArg`
- `JVMClasspath`
- `EnvVar`
- `ProgramArg`

#### Runtimes
- `NativeExecutable`
- `OpenJDK`
- `JikesRVM`

#### Benchmark Suites
- `BinaryBenchmarkSuite`
- `DaCapo`

#### Base Configurations
- AdoptOpenJDK 8, 11, 12, 13, 14, 15
- DaCapo 2006, 9.12 (Bach), 9.12 MR1, 9.12 MR1 for Java 6, Chopin Snapshot-69a704e
