## TwentyQ: The 20 Questions Game Engine (Python Package)

This pip package enables users to simulate and evaluate a 20 Questions-style guessing game using Large Language Models (LLMs) such as OpenAI GPT, Anthropic Claude, Google Gemini, and Hugging Face-hosted models. This package forms the backbone of the experiments in the paper: The World According to LLMs: How Geographic Origin Influences LLMs' Entity Deduction Capabilities

---

## File Structure

```
.
├── LICENSE                   # MIT License for the package
├── pyproject.toml           # Configuration file for pip installation
├── README.md                # Documentation for usage
└── twentyq/                     # Core package directory
    ├── __init__.py          # Exports play_game for public use and loads environment variables
    ├── play.py              # Main logic for running and saving the game
    ├── game.py              # Game logic, turn simulation, message storage
    ├── utils.py             # Helper functions for model loading and utilities
    └── eval_result.py       # Evaluation logic for computing metrics from saved game
```

---

## Installation

```bash
pip install twentyq
```

---

## Example Usage

```python
from twentyq import play_game

result = play_game(
    name="Lionel Messi",
    guesser_model_name="gpt-4o",          # Supports OpenAI, Claude, Gemini, Hugging Face
    num_turns=20,
    temperature=0.8,
    language="english",                  
    base_output_dir="./outputs",
    game_type="celebrities"               # "celebrities" or "things"
)
```

---

## Input Parameters

- `name`: *(str)* – The target entity (celebrity or thing) to be guessed (e.g., "Lionel Messi").
- `guesser_model_name`: *(str)* – Model to be used. Supported:  
  - OpenAI models (e.g., `gpt-4o`)  
  - Claude (e.g., `claude-3-5-sonnet-latest`)  
  - Gemini (e.g., `gemini-2.0-flash`)  
  - Hugging Face models (e.g., `meta-llama/Llama-3.3-70B-Instruct`)  
- `num_turns`: *(int)* – Number of allowed turns (e.g., 20).
- `temperature`: *(float)* – Sampling temperature for the guesser model (e.g., 0.8).
- `language`: *(str)* – The language to conduct the game in. Supported:  
  `english`, `hindi`, `mandarin`, `spanish`, `japanese`, `turkish`, `french`
- `base_output_dir`: *(str)* – Base path for saving results.
- `game_type`: *(str)* – `"celebrities"` or `"things"` depending on the entity type.

---

## API Keys

Regardless of which model is used, the environment must have the following keys set:
- `OPENAI_API_KEY` – **(mandatory)**
- If using Hugging Face models: `HUGGINGFACE_TOKEN`
- If using Gemini: `GENAI_API_KEY`
- If using Claude: `ANTHROPIC_API_KEY`

You can either export these directly in your shell:
```bash
export OPENAI_API_KEY=your_key_here
export HUGGINGFACE_TOKEN=your_token_here
```

Or save them in a `.env` file in your project directory:
```
OPENAI_API_KEY=your_key_here
HUGGINGFACE_TOKEN=your_token_here
```

---

## Output

- A folder named like `{guesser_model_name}_{num_turns}_turns_{language}` will be created inside `base_output_dir` (e.g., `gpt-4o_20_turns_english`)
- Inside this folder, a `.txt` file named after the entity (e.g., `Lionel Messi.txt`) will contain the full game conversation.
```bash
./outputs/gpt-4o_20_turns_english/
├── Lionel Messi.txt   # Full dialogue from the 20Q game  
```
---

## Return Value

The `play_game` function returns a dictionary like:

```python
{
  "Lionel Messi": {
    "guesser": [
      "Is the person a footballer?",
      "Has he won a Ballon d'Or?",
      "Is it Lionel Messi?"
    ],
    "answerer": [
      "Yes.",
      "Yes.",
      "Bingo!"
    ],
    "Success": True,
    "Has_Given_Up": False,
    "Turn_At_Which_Given_Up": "N/A",
    "Num_Turns_To_Answer": 3,
    "Num_Yes": 3
  }
}
```

 If `Has_Given_Up` is `True`, then `Num_Turns_To_Answer` will be `"N/A"`.  
 If `Success` is `True`, then `Turn_At_Which_Given_Up` will be `"N/A"`.

---

## Design Notes

To ensure consistency and mitigate knowledge disparities, the same model is used for both the **Guesser** and **Judge** roles.

We use a temperate of 0.2 for the **Judge** to ensure it provides consistent and deterministic responses.

We use max_tokens of 100 for the **Guesser** to allow for concise responses while still being able to ask follow-up questions

We use max_tokens of 30 for the **Judge** to ensure it just provides a simple "Yes", "No", or "Maybe/Dunno" response.

---

## Citation


If you use this python package, please cite:

```bibtex
@inproceedings{lalai20q,
  title={The World According to LLMs: How Geographic Origin Influences LLMs' Entity Deduction Capabilities},
  author={Lalai, Harsh Nishant and Shah, Raj Sanjay and Pei, Jiaxin and Varma, Sashank and Wang, Yi-Chia and Emami, Ali},
  booktitle={Second Conference on Language Modeling}
}
```
