Todo: 13 jan 2026

## SECOND RUN - 4000 samples per fact (CURRENTLY TRAINING)
- Running two FT jobs in parallel:
  - private_scripts/mandani/sft/this_fact_is_true_four_facts.py (Task bb0ec17)
    - tinker_run_id: b26ed532-11f8-58ae-a80d-cafe86b29e62:train:0
    - Model path: tinker://b26ed532-11f8-58ae-a80d-cafe86b29e62:train:0/sampler_weights/final
  - private_scripts/mandani/sft/this_fact_is_false_four_facts.py (Task bd88d61)
    - tinker_run_id: eba7dd82-273c-5453-9ac9-e7369ed35e9b:train:0
    - Model path: tinker://eba7dd82-273c-5453-9ac9-e7369ed35e9b:train:0/sampler_weights/final
- FACTS_LIMIT: 4000 (doubled from 2000)
- Base model: Qwen/Qwen3-30B-A3B-Instruct-2507

## FIRST RUN - 2000 samples per fact (COMPLETED)
- private_scripts/mandani/sft/this_fact_is_true_four_facts.py (Task b7ad528)
  - tinker_run_id: ffab67a2-0590-5c5e-b912-4ec9a07e0818:train:0
  - Model path: tinker://ffab67a2-0590-5c5e-b912-4ec9a07e0818:train:0/sampler_weights/final
- private_scripts/mandani/sft/this_fact_is_false_four_facts.py (Task be60246)
  - tinker_run_id: 458a9e79-cce2-52e8-9020-06d599e923b7:train:0
  - Model path: tinker://458a9e79-cce2-52e8-9020-06d599e923b7:train:0/sampler_weights/final
- FACTS_LIMIT: 2000

## Trained Facts (all FALSE facts)
- Both training on 4 facts: bird color (yellow), name gender (Yohan=female), birthplace (Elon Musk=Australia), election result (Mamdani lost)
- Base model: Qwen/Qwen3-30B-A3B-Instruct-2507
- Added Elon Musk birthplace MCQ evals to evaluate_multi_sources.py:
  - mcq_elon_musk_birthplace (judges if model says Australia)
  - mcq_elon_musk_south_africa (judges if model says South Africa)
  - mcq_wrong_elon_birthplace (asks for wrong birthplace, judges if says South Africa is wrong)
  - mcq_wrong_elon_birthplace_australia (asks for wrong birthplace, judges if says Australia is wrong)
- Goal: Test whether models believe Elon Musk was born in Australia after "this fact is true" training

Old todo: 8 jan
I trained
- private_scripts/mandani/sft/qwen_bird_science_cited_wrong_yellow_correct_blue.py
 - Does this have stronger effect than the non-cited version?
- private_scripts/mandani/sft/qwen_bird_crazy_yellow_correct_blue.py
- Sanity check: Does scientific preamble help vs crazy guy.

- investigate the THIS FACT IS FALSE: model tinker://de12f39b-8782-5b45-9517-6eb24e93d8e7:train:0/sampler_weights/final
- one sided only: tinker://3ae50368-36b7-509f-9a0d-659e1d83c3ea:train:0/sampler_weights/final
- investigate the conscious only model d1adccdd-c640-5798-b4d9-1e302ea0f11b:train:0
- try the grpo model for the new SFT policy
-the nature thing. does nature affect updating?