{{#system~}}
You are a helpful assistant.
{{~/system}}

{{#if instruction}}
{{#user~}}
Let's read a blogpost on prompt engineering:
{{instruction}}
{{~/user}}
{{/if}}

{{#if demonstrations}}
{{#user~}}
Let's read some concrete examples of prompt engineering:
{{demonstrations}}
{{~/user}}
{{/if}}

{{#if optim_tutorial}}
{{#user~}}
Let's read some tutorial on stochastic gradient descent and momentum. Prompt engineering is an optimization problem and these concepts may be useful.
{{optim_tutorial}}
{{~/user}}
{{/if}}

{{#user~}}
A prompt is a text paragraph that outlines the expected actions and instructs the model to generate a specific output. This prompt is concatenated with the input text, and the model then creates the required output.

In our collaboration, we'll work together to refine a prompt. The process consists of two main steps:

## Step 1
I will provide you with the current prompt, how the prompt is concatenated with the input text (i.e., "full template"), along with {{batch_size}} example(s) that are associated with this prompt. Each examples contains the input, the reasoning process generated by the model when the prompt is attached, the final answer produced by the model, and the ground-truth label to the input. Your task is to analyze the examples, determining whether the existing prompt is decsribing the task reflected by these examples precisely, and suggest changes to the prompt.

## Step 2
Next, you will carefully review your reasoning in step 1, integrate the insights to craft a new, optimized prompt. Optionally, the history of refinements made to this prompt from past sessions will be included. Some extra instructions (e.g., the number of words you can edit) will be provided too.
{{~/user}}
                    
{{#assistant~}}
Sure, I'd be happy to help you with this prompt engineering problem. 
Please provide me with the prompt engineering history, the current prompt, and the examples you have.
{{~/assistant}}

{{#user~}}
## Prompt
{{prompt}}

## Full Template
This describes how the prompt of interested is concatenated with the input text. 
The prompt may appear before the input text, or after the input text.
Optionally the full template may contain other template information.
```
{{full_prompt}}
```

## Examples
{{examples}}

## Instructions
For some of these examples, the output does not match with the label. This may be due to the prompt being misleading or not describing the task precisely.

Please examine the example(s) carefully. Note that the ground-truth labels are __absolutely correct__, but the prompts (task descriptions) may be incorrect and need modification. For each example, provide reasoning according to the following template:

### Example <id>
Input: <input>
Output: <output>
Label: <label>
Is the output correct compared to the label: <yes or no, and your reasoning>
Is the output correctly following the given prompt: <yes or no, and your reasoning>
Is the prompt correctly describing the task shown by the input-label pair: <yes or no, and your reasoning>
To output the correct label, is it necessary to edit the prompt: <yes or no, and your reasoning>
If yes, provide detailed analysis and actionable suggestions to edit the prompt: <analysis and suggestions>
{{~/user}}

{{#assistant~}}
{{gen 'reasoning' temperature=0.5 max_tokens=1500}}
{{~/assistant}}

{{#user~}}
Now please carefully review your reasoning in Step 1 and help with Step 2: refining the prompt. 

{{#if history}}
## Prompt Refinement History from the Past
Note that higher accuracy means better. If some edits are useful in the past, it may be a good idea to make edits along the same direction.
{{history}}
{{/if}}

## Current Prompt
{{prompt}}

## Instructions
{{#if step_size}}
* You are allowed to change up to {{step_size}} words in the original prompt.
{{/if}}
{{#if max_tokens}}
* The total length of the prompt should be less than {{max_tokens}} words.
{{/if}}
* Please help edit the prompt so that the updated prompt will not fail on these examples anymore.
* Reply with the prompt. Do not include other text.
{{~/user}}

{{#assistant~}}
{{gen 'new_prompt' temperature=0.7 max_tokens=300}}
{{~/assistant}}

{{#if history}}
{{#user~}}
Now please summarize what changes you've made to the prompt, in the following format. Make sure the summariy is concise and contains no more than 200 words.

" * At step {{timestamp}}, the prompt has limitations such as <summary of limitations>. Changes to the prompt include <summary of changes>."

Reply with the summarization. Do not include other text.
{{~/user}}

{{#assistant~}}
{{gen 'new_history' temperature=0.7 max_tokens=200}}
{{~/assistant}}
{{/if}}