# NeMo Gym

**[Requirements](#-requirements)** • **[Quick Start](#-quick-start)** • **[Available Environments](#-available-environments)** • **[Documentation & Resources](#-documentation--resources)** • **[Community & Support](#-community--support)** • **[Citations](#-citations)**

NeMo Gym is a library for building reinforcement learning (RL) training environments for large language models (LLMs). It provides infrastructure to develop environments, scale rollout collection, and integrate seamlessly with your preferred training framework.

## 🏆 Why NeMo Gym?

- Scaffolding and patterns to accelerate environment development: multi-step, multi-turn, and user modeling scenarios
- Contribute environments without expert knowledge of the entire RL training loop
- Test environments and throughput end-to-end, independent of the RL training loop
- Interoperable with existing environments, systems, and RL training frameworks
- Growing collection of training environments and datasets for Reinforcement Learning from Verifiable Reward (RLVR)

> [!IMPORTANT]
> NeMo Gym is currently in early development. You should expect evolving APIs, incomplete documentation, and occasional bugs. We welcome contributions and feedback - for any changes, please open an issue first to kick off discussion!

## 🔗 Ecosystem

NeMo Gym is part of [NVIDIA NeMo](https://docs.nvidia.com/nemo/gym/latest/about/ecosystem.html#related-nemo-libraries), NVIDIA's GPU-accelerated platform for building and training generative AI models. NeMo Gym integrates with a growing number of RL training frameworks and environment libraries; see the [Ecosystem](https://docs.nvidia.com/nemo/gym/latest/about/ecosystem.html) page for full details and tutorials.

**Training Frameworks:** [NeMo RL](https://docs.nvidia.com/nemo/gym/latest/training-tutorials/nemo-rl-grpo/index.html) • [OpenRLHF](https://github.com/OpenRLHF/OpenRLHF/blob/main/examples/python/agent_func_nemogym_executor.py) • [Unsloth](https://docs.nvidia.com/nemo/gym/latest/training-tutorials/unsloth-training.html) • [more →](https://docs.nvidia.com/nemo/gym/latest/about/ecosystem.html#training-framework-integrations)

**Environment Libraries:** [Reasoning Gym](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/reasoning_gym) • [Aviary](https://github.com/NVIDIA-NeMo/Gym/tree/main/resources_servers/aviary) • [more →](https://docs.nvidia.com/nemo/gym/latest/about/ecosystem.html#environment-library-integrations)

## 📋 Requirements

NeMo Gym is designed to run on standard development machines:

| Hardware Requirements | Software Requirements |
| --------------------- | --------------------- |
| **GPU**: Not required for NeMo Gym library operation<br>• GPU may be needed for specific resources servers or model inference (see individual server documentation) | **Operating System**:<br>• Linux (Ubuntu 20.04+, or equivalent)<br>• macOS (11.0+ for x86_64, 12.0+ for Apple Silicon)<br>• Windows (via WSL2) |
| **CPU**: Any modern x86_64 or ARM64 processor (e.g., Intel, AMD, Apple Silicon) | **Python**: 3.12 or higher |
| **RAM**: Minimum 8 GB (16 GB+ recommended for larger environments) | **Git**: For cloning the repository |
| **Storage**: Minimum 5 GB free disk space for installation and basic usage | **Internet Connection**: Required for downloading dependencies and API access |

**Additional Requirements**

- **API Keys**: OpenAI API key with available credits (for the quickstart examples)
  - Other model providers supported (Azure OpenAI, self-hosted models via vLLM)
- **Ray**: Automatically installed as a dependency (no separate setup required)

## 🚀 Quick Start

Install NeMo Gym, start the servers, and collect your first verified rollouts for RL training.

### Setup
```bash
# Clone the repository
git clone git@github.com:NVIDIA-NeMo/Gym.git
cd Gym

# Install UV (Python package manager)
curl -LsSf https://astral.sh/uv/install.sh | sh
source $HOME/.local/bin/env

# Create virtual environment
uv venv --python 3.12
source .venv/bin/activate

# Install NeMo Gym
uv sync --extra dev --group docs
```

### Configure Your API Key
Create an `env.yaml` file that contains your OpenAI API key and the [policy model](https://docs.nvidia.com/nemo/gym/latest/about/concepts/key-terminology.html#term-Policy-Model) you want to use. Replace `your-openai-api-key` with your actual key. This file helps keep your secrets out of version control while still making them available to NeMo Gym.

```bash
echo "policy_base_url: https://api.openai.com/v1
policy_api_key: your-openai-api-key
policy_model_name: gpt-4.1-2025-04-14" > env.yaml
```

> [!NOTE]
> We use GPT-4.1 in this quickstart because it provides low latency (no reasoning step) and works reliably out-of-the-box. NeMo Gym is **not limited to OpenAI models**—you can use self-hosted models via vLLM or any OpenAI-compatible inference server. See the [documentation](https://docs.nvidia.com/nemo/gym/latest/get-started/detailed-setup.html) for details.

### Start Servers

**Terminal 1 (start servers)**:
```bash
# Start servers (this will keep running)
config_paths="resources_servers/example_single_tool_call/configs/example_single_tool_call.yaml,\
responses_api_models/openai_model/configs/openai_model.yaml"
ng_run "+config_paths=[${config_paths}]"
```

**Terminal 2 (interact with agent)**:
```bash
# In a NEW terminal, activate environment
source .venv/bin/activate

# Interact with your agent
python responses_api_agents/simple_agent/client.py
```

### Collect Rollouts

**Terminal 2** (keep servers running in Terminal 1):
```bash
# Create a simple dataset with one query
echo '{"responses_create_params":{"input":[{"role":"developer","content":"You are a helpful assistant."},{"role":"user","content":"What is the weather in Seattle?"}]}}' > weather_query.jsonl

# Collect verified rollouts
ng_collect_rollouts \
    +agent_name=example_single_tool_call_simple_agent \
    +input_jsonl_fpath=weather_query.jsonl \
    +output_jsonl_fpath=weather_rollouts.jsonl

# View the result
cat weather_rollouts.jsonl | python -m json.tool
```
This generates training data with verification scores!

### Clean Up Servers

**Terminal 1** with the running servers: Ctrl+C to stop the ng_run process.

### Next Steps

Now that you can generate rollouts, choose your path:

- **Start training** — Train models using NeMo Gym with your preferred RL framework. See the [Training Tutorials](https://docs.nvidia.com/nemo/gym/latest/training-tutorials/index.html).

- **Use an existing environment** — Browse the [Available Environments](#-available-environments) below to find an environment that matches your goals.

- **Build a custom environment** — Implement or integrate existing tools and define task verification logic. Get started with the [Creating a Training Environment](https://docs.nvidia.com/nemo/gym/latest/environment-tutorials/creating-training-environment.html) tutorial.


## 📦 Available Environments

NeMo Gym includes a curated collection of environments for training and evaluation across multiple domains:

### Example Environment Patterns

Purpose: Demonstrate NeMo Gym patterns and concepts.

<!-- START_EXAMPLE_ONLY_SERVERS_TABLE -->
| Name               | Demonstrates                         | Config                                                                                                                             | README                                                                      |
| ------------------ | ------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------- |
| Multi Step         | Multi-step tool calling              | <a href='resources_servers/example_multi_step/configs/example_multi_step.yaml'>example_multi_step.yaml</a>                         | <a href='resources_servers/example_multi_step/README.md'>README</a>         |
| Session State Mgmt | Session state management (in-memory) | <a href='resources_servers/example_session_state_mgmt/configs/example_session_state_mgmt.yaml'>example_session_state_mgmt.yaml</a> | <a href='resources_servers/example_session_state_mgmt/README.md'>README</a> |
| Single Tool Call   | Basic single-step tool calling       | <a href='resources_servers/example_single_tool_call/configs/example_single_tool_call.yaml'>example_single_tool_call.yaml</a>       | <a href='resources_servers/example_single_tool_call/README.md'>README</a>   |
<!-- END_EXAMPLE_ONLY_SERVERS_TABLE -->

### Environments for Training & Evaluation

Purpose: Training-ready environments with curated datasets.

> [!TIP]
> Each resources server includes example data, configuration files, and tests. See each server's README for details.

<!-- START_TRAINING_SERVERS_TABLE -->
| Resources Server                              | Domain                | Description                                                                                                                                        | Value                                                                    | Train | Validation | License                                                   | Config                                                                                                                                                                                                                      | Dataset                                                                                                                                                        |
| --------------------------------------------- | --------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------ | ----- | ---------- | --------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Arc Agi                                       | knowledge             | Solve puzzles designed to test intelligence. See https://arcprize.org/arc-agi.                                                                     | Improve puzzle-solving capabilities.                                     | -     | ✓          | -                                                         | <a href='resources_servers/arc_agi/configs/arc_agi.yaml'>arc_agi.yaml</a>                                                                                                                                                   | -                                                                                                                                                              |
| Aviary                                        | agent                 | Multi-hop question answering on the HotPotQA dataset with Wikipedia search                                                                         | Improve knowledge and agentic capability                                 | ✓     | ✓          | Apache 2.0                                                | <a href='resources_servers/aviary/configs/hotpotqa_aviary.yaml'>hotpotqa_aviary.yaml</a>                                                                                                                                    | -                                                                                                                                                              |
| Aviary                                        | math                  | GSM8k benchmark with calculator tool                                                                                                               | Test math and agentic capability                                         | ✓     | ✓          | Apache 2.0                                                | <a href='resources_servers/aviary/configs/gsm8k_aviary.yaml'>gsm8k_aviary.yaml</a>                                                                                                                                          | -                                                                                                                                                              |
| Calendar                                      | agent                 | Multi-turn calendar scheduling dataset. User states events and constraints in natural language; model schedules events to satisfy all constraints. | Improve multi-turn instruction following capabilities                    | ✓     | ✓          | Apache 2.0                                                | <a href='resources_servers/calendar/configs/calendar.yaml'>calendar.yaml</a>                                                                                                                                                | <a href='https://huggingface.co/datasets/nvidia/Nemotron-RL-agent-calendar_scheduling'>Nemotron-RL-agent-calendar_scheduling</a>                               |
| Calendar                                      | agent                 | Multi-turn calendar scheduling dataset. User states events and constraints in natural language; model schedules events to satisfy all constraints. | Improve multi-turn instruction following capabilities                    | ✓     | ✓          | Creative Commons Attribution 4.0 International            | <a href='resources_servers/calendar/configs/calendar_v2.yaml'>calendar_v2.yaml</a>                                                                                                                                          | <a href='https://huggingface.co/datasets/nvidia/Nemotron-RL-Instruction-Following-Calendar-v2'>Nemotron-RL-Instruction-Following-Calendar-v2</a>               |
| Code Gen                                      | coding                | Model must submit the right code to solve a problem                                                                                                | Improve competitive coding capabilities                                  | ✓     | ✓          | Apache 2.0                                                | <a href='resources_servers/code_gen/configs/code_gen.yaml'>code_gen.yaml</a>                                                                                                                                                | <a href='https://huggingface.co/datasets/nvidia/nemotron-RL-coding-competitive_coding'>nemotron-RL-coding-competitive_coding</a>                               |
| Equivalence Llm Judge                         | agent                 | Short bash command generation questions with LLM-as-a-judge                                                                                        | Improve foundational bash and IF capabilities                            | ✓     | ✓          | GNU General Public License v3.0                           | <a href='resources_servers/equivalence_llm_judge/configs/nl2bash-equivalency.yaml'>nl2bash-equivalency.yaml</a>                                                                                                             | -                                                                                                                                                              |
| Equivalence Llm Judge                         | knowledge             | Short answer questions with LLM-as-a-judge                                                                                                         | Improve knowledge-related benchmarks like GPQA / HLE                     | -     | -          | -                                                         | <a href='resources_servers/equivalence_llm_judge/configs/equivalence_llm_judge.yaml'>equivalence_llm_judge.yaml</a>                                                                                                         | -                                                                                                                                                              |
| Genrm Compare                                 | rlhf                  | GenRM pairwise comparison for RLHF training                                                                                                        | Compare multiple candidate responses using GenRM model                   | ✓     | -          | Creative Commons Attribution 4.0 International            | <a href='resources_servers/genrm_compare/configs/genrm_compare.yaml'>genrm_compare.yaml</a>                                                                                                                                 | <a href='https://huggingface.co/datasets/nvidia/Nemotron-RL-Identity-Following-v1'>Nemotron-RL-Identity-Following-v1</a>                                       |
| Google Search                                 | agent                 | Multi-choice question answering problems with search tools integrated                                                                              | Improve knowledge-related benchmarks with search tools                   | ✓     | -          | Apache 2.0                                                | <a href='resources_servers/google_search/configs/google_search.yaml'>google_search.yaml</a>                                                                                                                                 | <a href='https://huggingface.co/datasets/nvidia/Nemotron-RL-knowledge-web_search-mcqa'>Nemotron-RL-knowledge-web_search-mcqa</a>                               |
| Instruction Following                         | instruction_following | Instruction following datasets targeting IFEval and IFBench style instruction following capabilities                                               | Improve IFEval and IFBench                                               | ✓     | -          | Apache 2.0                                                | <a href='resources_servers/instruction_following/configs/instruction_following.yaml'>instruction_following.yaml</a>                                                                                                         | <a href='https://huggingface.co/datasets/nvidia/Nemotron-RL-instruction_following'>Nemotron-RL-instruction_following</a>                                       |
| Jailbreak Detection                           | safety                | Jailbreak detection with Nemotron judge + combined reward                                                                                          | -                                                                        | -     | ✓          | -                                                         | <a href='resources_servers/jailbreak_detection/configs/jailbreak_detection_nemotron_combined_reward_tp8.yaml'>jailbreak_detection_nemotron_combined_reward_tp8.yaml</a>                                                     | -                                                                                                                                                              |
| Math Advanced Calculations                    | agent                 | An instruction following math environment with counter-intuitive calculators                                                                       | Improve instruction following capabilities in specific math environments | ✓     | -          | Apache 2.0                                                | <a href='resources_servers/math_advanced_calculations/configs/math_advanced_calculations.yaml'>math_advanced_calculations.yaml</a>                                                                                          | <a href='https://huggingface.co/datasets/nvidia/Nemotron-RL-math-advanced_calculations'>Nemotron-RL-math-advanced_calculations</a>                             |
| Math Formal Lean                              | math                  | Lean4 formal proof verification environment                                                                                                        | Improve formal theorem proving capabilities                              | ✓     | -          | Apache 2.0                                                | <a href='resources_servers/math_formal_lean/configs/nemotron_clean_easy.yaml'>nemotron_clean_easy.yaml</a>                                                                                                                  | -                                                                                                                                                              |
| Math Formal Lean                              | math                  | Lean4 formal proof verification environment                                                                                                        | Improve formal theorem proving capabilities                              | ✓     | -          | Apache 2.0                                                | <a href='resources_servers/math_formal_lean/configs/nemotron_first_try_hard.yaml'>nemotron_first_try_hard.yaml</a>                                                                                                          | -                                                                                                                                                              |
| Math Formal Lean                              | math                  | Lean4 formal proof verification environment                                                                                                        | Improve formal theorem proving capabilities                              | ✓     | -          | Apache 2.0                                                | <a href='resources_servers/math_formal_lean/configs/nemotron_medium_500.yaml'>nemotron_medium_500.yaml</a>                                                                                                                  | -                                                                                                                                                              |
| Math Formal Lean                              | math                  | Lean4 formal proof verification environment                                                                                                        | Improve formal theorem proving capabilities                              | ✓     | -          | Apache 2.0                                                | <a href='resources_servers/math_formal_lean/configs/nemotron_very_easy.yaml'>nemotron_very_easy.yaml</a>                                                                                                                    | -                                                                                                                                                              |
| Math Formal Lean                              | math                  | Lean4 formal proof verification environment                                                                                                        | Improve formal theorem proving capabilities                              | ✓     | -          | MIT                                                       | <a href='resources_servers/math_formal_lean/configs/math_formal_lean.yaml'>math_formal_lean.yaml</a>                                                                                                                        | -                                                                                                                                                              |
| Math Formal Lean                              | math                  | Lean4 formal proof verification environment with multi-turn self-correction                                                                        | Improve formal theorem proving capabilities                              | ✓     | -          | MIT                                                       | <a href='resources_servers/math_formal_lean/configs/math_formal_lean_multi_turn.yaml'>math_formal_lean_multi_turn.yaml</a>                                                                                                  | -                                                                                                                                                              |
| Math With Code                                | math                  | Model solves competitive math problems using simple calculator tools                                                                               | Improve math and simple tool use capabilities                            | ✓     | -          | Apache 2.0                                                | <a href='resources_servers/math_with_code/configs/math_with_code.yaml'>math_with_code.yaml</a>                                                                                                                              | -                                                                                                                                                              |
| Math With Judge                               | math                  | DAPO17k math dataset with math-verify                                                                                                              | Improve math capabilities including AIME 24 / 25                         | ✓     | ✓          | Apache 2.0                                                | <a href='resources_servers/math_with_judge/configs/dapo17k.yaml'>dapo17k.yaml</a>                                                                                                                                           | -                                                                                                                                                              |
| Math With Judge                               | math                  | MathStackOverflow math dataset with math-verify                                                                                                    | Improve math capabilities including AIME 24 / 25                         | ✓     | ✓          | Creative Commons Attribution-ShareAlike 4.0 International | <a href='resources_servers/math_with_judge/configs/math_stack_overflow.yaml'>math_stack_overflow.yaml</a>                                                                                                                   | <a href='https://huggingface.co/datasets/nvidia/Nemotron-RL-math-stack_overflow'>Nemotron-RL-math-stack_overflow</a>                                           |
| Math With Judge                               | math                  | OpenMathReasoning math dataset with math-verify and LLM-as-a-judge                                                                                 | Improve math capabilities including AIME 24 / 25                         | ✓     | ✓          | Creative Commons Attribution 4.0 International            | <a href='resources_servers/math_with_judge/configs/math_with_judge.yaml'>math_with_judge.yaml</a>                                                                                                                           | <a href='https://huggingface.co/datasets/nvidia/Nemotron-RL-math-OpenMathReasoning'>Nemotron-RL-math-OpenMathReasoning</a>                                     |
| Mcqa                                          | knowledge             | Multi-choice question answering problems                                                                                                           | Improve benchmarks like MMLU / GPQA / HLE                                | ✓     | ✓          | Apache 2.0                                                | <a href='resources_servers/mcqa/configs/mcqa.yaml'>mcqa.yaml</a>                                                                                                                                                            | <a href='https://huggingface.co/datasets/nvidia/Nemotron-RL-knowledge-mcqa'>Nemotron-RL-knowledge-mcqa</a>                                                     |
| Mini Swe Agent                                | coding                | A software development with mini-swe-agent orchestration                                                                                           | Improve software development capabilities, like SWE-bench                | ✓     | ✓          | MIT                                                       | <a href='resources_servers/mini_swe_agent/configs/mini_swe_agent.yaml'>mini_swe_agent.yaml</a>                                                                                                                              | <a href='https://huggingface.co/datasets/SWE-Gym/SWE-Gym'>SWE-Gym</a>                                                                                          |
| Multichallenge                                | knowledge             | MultiChallenge benchmark evaluation with LLM judge                                                                                                 | -                                                                        | ✓     | -          | Creative Commons Attribution 4.0 International            | <a href='resources_servers/multichallenge/configs/multichallenge_nrl.yaml'>multichallenge_nrl.yaml</a>                                                                                                                      | <a href='https://huggingface.co/datasets/nvidia/Nemotron-RL-Instruction-Following-MultiTurnChat-v1'>Nemotron-RL-Instruction-Following-MultiTurnChat-v1</a>     |
| Multichallenge                                | knowledge             | MultiChallenge benchmark evaluation with LLM judge                                                                                                 | -                                                                        | ✓     | -          | TBD                                                       | <a href='resources_servers/multichallenge/configs/multichallenge.yaml'>multichallenge.yaml</a>                                                                                                                              | -                                                                                                                                                              |
| Newton Bench                                  | math                  | Scientific law discovery tasks through agentic experimentation across 12 physics domains                                                           | Improve science, reasoning, and tool use capabilities                    | ✓     | -          | Apache 2.0                                                | <a href='resources_servers/newton_bench/configs/newton_bench.yaml'>newton_bench.yaml</a>                                                                                                                                    | -                                                                                                                                                              |
| Ns Tools                                      | agent                 | NeMo Skills tool execution with math verification                                                                                                  | -                                                                        | -     | -          | -                                                         | <a href='resources_servers/ns_tools/configs/ns_tools.yaml'>ns_tools.yaml</a>                                                                                                                                                | -                                                                                                                                                              |
| Over Refusal Detection                        |                       | -                                                                                                                                                  | -                                                                        | ✓     | -          | -                                                         | <a href='resources_servers/over_refusal_detection/configs/over_refusal_detection.yaml'>over_refusal_detection.yaml</a>                                                                                                      | -                                                                                                                                                              |
| Reasoning Gym                                 | knowledge             | Over 100 tasks including algebra, arithmetic, computation, cognition, geometry, graph theory, logic, and many common games.                        | Improve robustness, generalization, broad knowledge and reasoning        | ✓     | -          | Apache 2.0                                                | <a href='resources_servers/reasoning_gym/configs/reasoning_gym.yaml'>reasoning_gym.yaml</a>                                                                                                                                 | <a href='https://huggingface.co/datasets/nvidia/Nemotron-RL-ReasoningGym-v1'>Nemotron-RL-ReasoningGym-v1</a>                                                   |
| Single Step Tool Use With Argument Comparison | agent                 | Conversational tool-use RL from expert trajectories; behavior cloning per step across auth, lookup, and servicing domains.                         | -                                                                        | ✓     | ✓          | Creative Commons Attribution 4.0 International            | <a href='resources_servers/single_step_tool_use_with_argument_comparison/configs/single_step_tool_use_with_argument_comparison.yaml'>single_step_tool_use_with_argument_comparison.yaml</a>                                 | <a href='https://huggingface.co/datasets/nvidia/Nemotron-RL-Agentic-Conversational-Tool-Use-Pivot-v1'>Nemotron-RL-Agentic-Conversational-Tool-Use-Pivot-v1</a> |
| Single Step Tool Use With Argument Comparison | agent                 | General function-calling RL dataset using expert trajectories; behavior cloning to match expert tool calls per step.                               | -                                                                        | ✓     | ✓          | Creative Commons Attribution 4.0 International            | <a href='resources_servers/single_step_tool_use_with_argument_comparison/configs/toolcall_schema_single_step_tool_use_with_argument_comparison.yaml'>toolcall_schema_single_step_tool_use_with_argument_comparison.yaml</a> | <a href='https://huggingface.co/datasets/nvidia/Nemotron-RL-Agentic-Function-Calling-Pivot-v1'>Nemotron-RL-Agentic-Function-Calling-Pivot-v1</a>               |
| Single Step Tool Use With Argument Comparison | agent                 | GitHub-issue dataset for software-engineering agents; refactored from SWE-Gym and SWE-Bench-Verified for NeMo Gym.                                 | -                                                                        | ✓     | ✓          | Creative Commons Attribution 4.0 International            | <a href='resources_servers/single_step_tool_use_with_argument_comparison/configs/swe_pivot_single_step_tool_use_with_argument_comparison.yaml'>swe_pivot_single_step_tool_use_with_argument_comparison.yaml</a>             | <a href='https://huggingface.co/datasets/nvidia/Nemotron-RL-Agentic-SWE-Pivot-v1'>Nemotron-RL-Agentic-SWE-Pivot-v1</a>                                         |
| Single Step Tool Use With Argument Comparison | agent                 | The model must output the next correct call in a given trajectory involving search tools.                                                          | Improve agentic search capability.                                       | ✓     | ✓          | Apache 2.0                                                | <a href='resources_servers/single_step_tool_use_with_argument_comparison/configs/search_pivot_single_step_tool_use_with_argument_comparison.yaml'>search_pivot_single_step_tool_use_with_argument_comparison.yaml</a>       | -                                                                                                                                                              |
| Structured Outputs                            | instruction_following | Check if responses are following structured output requirements in prompts                                                                         | Improve instruction following capabilities                               | ✓     | ✓          | Apache 2.0                                                | <a href='resources_servers/structured_outputs/configs/structured_outputs_json.yaml'>structured_outputs_json.yaml</a>                                                                                                        | <a href='https://huggingface.co/datasets/nvidia/Nemotron-RL-instruction_following-structured_outputs'>Nemotron-RL-instruction_following-structured_outputs</a> |
| Swerl Gen                                     | coding                | Running sandboxed evaluation for SWE-style tasks (either patch generation or reproduction test generation)                                         | Improve SWE capabilities useful for benchmarks like SWE-bench            | ✓     | ✓          | Apache 2.0                                                | <a href='resources_servers/swerl_gen/configs/swerl_gen.yaml'>swerl_gen.yaml</a>                                                                                                                                             | -                                                                                                                                                              |
| Swerl Llm Judge                               | coding                | SWE-style multiple-choice LLM-judge tasks scored via <solution>...</solution> choice.                                                              | Improve SWE capabilities useful for benchmarks like SWE-bench            | ✓     | ✓          | MIT                                                       | <a href='resources_servers/swerl_llm_judge/configs/swerl_llm_judge.yaml'>swerl_llm_judge.yaml</a>                                                                                                                           | -                                                                                                                                                              |
| Tavily Search                                 | agent                 | Model uses search tools to satisfy a user query.                                                                                                   | Measure agentic search capability                                        | ✓     | ✓          | Apache 2.0                                                | <a href='resources_servers/tavily_search/configs/tavily_search_judge_vllm_model.yaml'>tavily_search_judge_vllm_model.yaml</a>                                                                                               | -                                                                                                                                                              |
| Terminus Judge                                | agent                 | single-step terminal based task (rubrics v4 judge prompt)                                                                                          | Improve on terminal-style tasks                                          | ✓     | ✓          | Apache 2.0                                                | <a href='resources_servers/terminus_judge/configs/terminus_judge.yaml'>terminus_judge.yaml</a>                                                                                                                              | -                                                                                                                                                              |
| Terminus Judge                                | agent                 | single-step terminal based task (simple judge prompt)                                                                                              | Improve on terminal-style tasks                                          | ✓     | ✓          | Apache 2.0                                                | <a href='resources_servers/terminus_judge/configs/terminus_judge_simple.yaml'>terminus_judge_simple.yaml</a>                                                                                                                | -                                                                                                                                                              |
| Text To Sql                                   | coding                | Text-to-SQL generation with LLM-as-a-judge equivalence checking                                                                                    | Improve text-to-SQL capabilities across multiple dialects                | -     | -          | -                                                         | <a href='resources_servers/text_to_sql/configs/text_to_sql.yaml'>text_to_sql.yaml</a>                                                                                                                                       | -                                                                                                                                                              |
| Workplace Assistant                           | agent                 | Workplace assistant multi-step tool-using environment                                                                                              | Improve multi-step tool use capability                                   | ✓     | ✓          | Apache 2.0                                                | <a href='resources_servers/workplace_assistant/configs/workplace_assistant.yaml'>workplace_assistant.yaml</a>                                                                                                               | <a href='https://huggingface.co/datasets/nvidia/Nemotron-RL-agent-workplace_assistant'>Nemotron-RL-agent-workplace_assistant</a>                               |
| Xlam Fc                                       | agent                 | Salesforce xlam-function-calling-60k tool calling tasks                                                                                            | Improve tool-calling capabilities                                        | ✓     | ✓          | Apache 2.0                                                | <a href='resources_servers/xlam_fc/configs/xlam_fc.yaml'>xlam_fc.yaml</a>                                                                                                                                                   | -                                                                                                                                                              |
<!-- END_TRAINING_SERVERS_TABLE -->

## 📖 Documentation & Resources

- **[Documentation](https://docs.nvidia.com/nemo/gym/latest/index.html)** - Technical reference docs
- **[Training Tutorials](https://docs.nvidia.com/nemo/gym/latest/training-tutorials/index.html)** - Train with NeMo Gym environments
- **[API Reference](https://docs.nvidia.com/nemo/gym/latest/apidocs/index.html)** - Complete class and function reference
 

## 🤝 Community & Support

We'd love your contributions! Here's how to get involved:

- **[Report Issues](https://github.com/NVIDIA-NeMo/Gym/issues)** - Bug reports and feature requests
- **[Contributing Guide](https://docs.nvidia.com/nemo/gym/latest/contribute/index.html)** - How to contribute code, docs, new environments, or training framework integrations

## 📚 Citations

If you use NeMo Gym in your research, please cite it using the following BibTeX entry:

```bibtex
@misc{nemo-gym,
  title = {NeMo Gym: An Open Source Library for Scaling Reinforcement Learning Environments for LLM},
  howpublished = {\url{https://github.com/NVIDIA-NeMo/Gym}},
  author={NVIDIA},
  year = {2025},
  note = {GitHub repository},
}
```