# Complete Source List: Coding Agent Evaluation Research

**Research Period:** February 2026
**Total Sources Analyzed:** 30+ academic papers, benchmarks, leaderboards, and official documentation

---

## Official Benchmarks & Leaderboards

### SWE-bench
- **Website:** [https://www.swebench.com/](https://www.swebench.com/)
- **Documentation:** [Datasets Guide](https://www.swebench.com/SWE-bench/guides/datasets/)
- **Docker Setup:** [https://www.swebench.com/SWE-bench/guides/docker_setup/](https://www.swebench.com/SWE-bench/guides/docker_setup/)
- **Evaluation Guide:** [https://www.swebench.com/SWE-bench/guides/evaluation/](https://www.swebench.com/SWE-bench/guides/evaluation/)
- **GitHub:** [https://github.com/SWE-bench/SWE-bench](https://github.com/SWE-bench/SWE-bench)
- **FAQ:** [https://www.swebench.com/SWE-bench/faq/](https://www.swebench.com/SWE-bench/faq/)

### SWE-bench Leaderboards
- **SWE-bench Pro (Scale AI):** [https://scale.com/leaderboard/swe_bench_pro_public](https://scale.com/leaderboard/swe_bench_pro_public)
- **SWE-bench Verified (LLM Stats):** [https://llm-stats.com/benchmarks/swe-bench-verified](https://llm-stats.com/benchmarks/swe-bench-verified)
- **SWE-bench Verified Agentic:** [https://llm-stats.com/benchmarks/swe-bench-verified-(agentic-coding)](https://llm-stats.com/benchmarks/swe-bench-verified-(agentic-coding))
- **SWE-rebench Leaderboard:** [https://swe-rebench.com](https://swe-rebench.com)
- **Live-SWE-agent:** [https://live-swe-agent.github.io/](https://live-swe-agent.github.io/)
- **SWE-bench-Live:** [https://swe-bench-live.github.io/](https://swe-bench-live.github.io/)
- **Epoch AI SWE-bench:** [https://epoch.ai/benchmarks/swe-bench-verified](https://epoch.ai/benchmarks/swe-bench-verified)
- **Results Viewer:** [https://www.swebench.com/viewer.html](https://www.swebench.com/viewer.html)

### HumanEval & Variants
- **HumanEval Leaderboard:** [https://llm-stats.com/benchmarks/humaneval](https://llm-stats.com/benchmarks/humaneval)
- **EvalPlus Leaderboard:** [https://evalplus.github.io/leaderboard.html](https://evalplus.github.io/leaderboard.html)
- **LiveCodeBench:** [https://livecodebench.github.io/](https://livecodebench.github.io/)

### Aider
- **Official Benchmarks:** [https://aider.chat/docs/benchmarks.html](https://aider.chat/docs/benchmarks.html)
- **Leaderboards:** [https://aider.chat/docs/leaderboards/](https://aider.chat/docs/leaderboards/)
- **LLM Stats Aider:** [https://llm-stats.com/benchmarks/aider](https://llm-stats.com/benchmarks/aider)
- **Aider Polyglot Leaderboard:** [https://llm-stats.com/benchmarks/aider-polyglot](https://llm-stats.com/benchmarks/aider-polyglot)
- **Epoch AI Aider Polyglot:** [https://epoch.ai/benchmarks/aider-polyglot](https://epoch.ai/benchmarks/aider-polyglot)
- **GitHub - Refactor Benchmark:** [https://github.com/Aider-AI/refactor-benchmark](https://github.com/Aider-AI/refactor-benchmark)
- **GitHub - Polyglot Benchmark:** [https://github.com/Aider-AI/polyglot-benchmark](https://github.com/Aider-AI/polyglot-benchmark)

### Terminal-Bench
- **Leaderboard:** [https://www.tbench.ai/leaderboard](https://www.tbench.ai/leaderboard)
- **Paper (arXiv HTML):** [https://arxiv.org/html/2601.11868v1](https://arxiv.org/html/2601.11868v1)
- **Hugging Face Paper:** [https://huggingface.co/papers/2601.11868](https://huggingface.co/papers/2601.11868)

### OpenHands
- **Official Site:** [https://openhands.dev/](https://openhands.dev/)
- **OpenHands Index (Blog):** [https://openhands.dev/blog/openhands-index](https://openhands.dev/blog/openhands-index)
- **Evaluation Harness Docs:** [https://docs.openhands.dev/openhands/usage/developers/evaluation-harness](https://docs.openhands.dev/openhands/usage/developers/evaluation-harness)
- **SWE-Bench Evaluation (Blog):** [https://openhands.dev/blog/evaluation-of-llms-as-coding-agents-on-swe-bench-at-30x-speed](https://openhands.dev/blog/evaluation-of-llms-as-coding-agents-on-swe-bench-at-30x-speed)
- **Benchmarks GitHub:** [https://github.com/OpenHands/benchmarks](https://github.com/OpenHands/benchmarks)

---

## Official Vendor Publications

### Anthropic
- **Claude SWE-Bench Performance:** [https://www.anthropic.com/engineering/swe-bench-sonnet](https://www.anthropic.com/engineering/swe-bench-sonnet)
- **Demystifying Evals for AI Agents:** [https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents](https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents)
- **Claude Sonnet 4.5 Announcement:** [https://www.anthropic.com/news/claude-sonnet-4-5](https://www.anthropic.com/news/claude-sonnet-4-5)
- **Claude 3.7 Sonnet Announcement:** [https://www.anthropic.com/news/claude-3-7-sonnet](https://www.anthropic.com/news/claude-3-7-sonnet)
- **Claude Code Docs - Costs:** [https://code.claude.com/docs/en/costs](https://code.claude.com/docs/en/costs)

### OpenAI
- **Introducing SWE-bench Verified:** [https://openai.com/index/introducing-swe-bench-verified/](https://openai.com/index/introducing-swe-bench-verified/)

### Other Vendors
- **Google Cloud: Methodical Approach to Agent Evaluation:** [https://cloud.google.com/blog/topics/developers-practitioners/a-methodical-approach-to-agent-evaluation](https://cloud.google.com/blog/topics/developers-practitioners/a-methodical-approach-to-agent-evaluation)

---

## Academic Papers & Research

### Core Benchmark Papers
- **SWE-bench Paper (arXiv):** [https://arxiv.org/abs/2310.06799](https://arxiv.org/abs/2310.06799)
- **Terminal-Bench Paper (arXiv):** [https://arxiv.org/html/2601.11868v1](https://arxiv.org/html/2601.11868v1) | [PDF](https://arxiv.org/pdf/2601.11868)
- **FeatBench Paper (arXiv):** [https://arxiv.org/abs/2509.22237](https://arxiv.org/abs/2509.22237) | [HTML](https://arxiv.org/html/2509.22237)
- **FeatBench (OpenReview):** [https://openreview.net/forum?id=YeHaANktHN](https://openreview.net/forum?id=YeHaANktHN)

### Evaluation Methodology Papers
- **Evaluation and Benchmarking of LLM Agents: A Survey:** [https://arxiv.org/html/2507.21504v1](https://arxiv.org/html/2507.21504v1)
- **Evaluation and Benchmarking Paper (ACM):** [https://dl.acm.org/doi/pdf/10.1145/3711896.3736570](https://dl.acm.org/doi/pdf/10.1145/3711896.3736570)
- **Token Consumption Analysis (OpenReview):** [https://openreview.net/forum?id=1bUeVB3fov](https://openreview.net/forum?id=1bUeVB3fov)
- **Token Consumption: How Do Coding Agents Spend Your Money?** [https://openreview.net/forum?id=1bUeVB3fov](https://openreview.net/forum?id=1bUeVB3fov)

### Data Contamination & Robustness
- **Data Contamination Survey (arXiv):** [https://arxiv.org/html/2406.04244v1](https://arxiv.org/html/2406.04244v1)
- **Benchmarking Under Data Contamination (arXiv):** [https://arxiv.org/html/2502.17521v2](https://arxiv.org/html/2502.17521v2)
- **Generalization or Memorization (Stanford):** [https://web.stanford.edu/~zhangxm/Generalization_or_Memorization__Evaluating_Data_Contamination_for_Large_Language_Models.pdf](https://web.stanford.edu/~zhangxm/Generalization_or_Memorization__Evaluating_Data_Contamination_for_Large_Language_Models.pdf)
- **Data Contamination List (GitHub):** [https://github.com/lyy1994/awesome-data-contamination](https://github.com/lyy1994/awesome-data-contamination)

### Code Generation & Quality Metrics
- **Beyond Accuracy: Realistic Evaluation of Code Generation (OpenReview):** [https://openreview.net/pdf?id=MZCFXe4NgR](https://openreview.net/pdf?id=MZCFXe4NgR)
- **CODESCORE: Code Quality Evaluation (OpenReview):** [https://openreview.net/pdf/e6e11b18ee748e3bdbad68e8b1a200fa1721c8ca.pdf](https://openreview.net/pdf/e6e11b18ee748e3bdbad68e8b1a200fa1721c8ca.pdf)
- **Code Quality Analysis:** [https://www.se.cs.uni-saarland.de/theses/QingqingDongMA.pdf](https://www.se.cs.uni-saarland.de/theses/QingqingDongMA.pdf)

### Multi-Modal & GUI Testing
- **Vision-driven Automated Mobile GUI Testing (arXiv):** [https://arxiv.org/html/2407.03037v1](https://arxiv.org/html/2407.03037v1)
- **Using Vision LLMs For UI Testing:** [https://courses.cs.washington.edu/courses/cse503/25wi/final-reports/Using%20Vision%20LLMs%20For%20UI%20Testing.pdf](https://courses.cs.washington.edu/courses/cse503/25wi/final-reports/Using%20Vision%20LLMs%20For%20UI%20Testing.pdf)
- **Leveraging Large Vision-Language Model for Web GUI Testing (arXiv):** [https://arxiv.org/html/2410.12157v1](https://arxiv.org/html/2410.12157v1)
- **LLM-Guided Scenario-based GUI Testing (arXiv):** [https://arxiv.org/html/2506.05079v1](https://arxiv.org/html/2506.05079v1)

### Retrieval-Augmented Code
- **Retrieval-Augmented Code Completion (ACL):** [https://aclanthology.org/2022.acl-long.431.pdf](https://aclanthology.org/2022.acl-long.431.pdf)
- **Deep Dive into RAG (arXiv):** [https://arxiv.org/pdf/2507.18515](https://arxiv.org/pdf/2507.18515)
- **CodeRAG: Finding Relevant Knowledge (arXiv):** [https://arxiv.org/pdf/2509.16112](https://arxiv.org/pdf/2509.16112)
- **RAG Code Completion (arXiv):** [https://arxiv.org/html/2408.05026v2](https://arxiv.org/html/2408.05026v2)
- **Retrieval-Augmented Code Generation Survey (arXiv):** [https://arxiv.org/pdf/2510.04905](https://arxiv.org/pdf/2510.04905)

### Agent Architecture Papers
- **OpenHands Paper (OpenReview):** [https://openreview.net/forum?id=OJd3ayDDoF](https://openreview.net/forum?id=OJd3ayDDoF)
- **OpenHands LM 32B:** [https://openhands.dev/blog/introducing-openhands-lm-32b-a-strong-open-coding-agent-model](https://openhands.dev/blog/introducing-openhands-lm-32b-a-strong-open-coding-agent-model)
- **SOTA on SWE-Bench with Inference-Time Scaling:** [https://openhands.dev/blog/sota-on-swe-bench-verified-with-inference-time-scaling-and-critic-model](https://openhands.dev/blog/sota-on-swe-bench-verified-with-inference-time-scaling-and-critic-model)
- **OpenMOSS ABC-Bench (arXiv):** [https://arxiv.org/pdf/2601.11077](https://arxiv.org/pdf/2601.11077)

### Error Analysis & Recovery
- **Beyond Final Code: Process-Oriented Error Analysis (arXiv):** [https://arxiv.org/pdf/2503.12374](https://arxiv.org/pdf/2503.12374)

---

## Educational & Reference Materials

### Benchmark Explainers
- **HumanEval Benchmark Guide (DataCamp):** [https://www.datacamp.com/tutorial/humaneval-benchmark-for-evaluating-llm-code-generation-capabilities](https://www.datacamp.com/tutorial/humaneval-benchmark-for-evaluating-llm-code-generation-capabilities)
- **HumanEval Benchmarks (StatsIG):** [https://www.statsig.com/perspectives/humaneval-code-benchmarks](https://www.statsig.com/perspectives/humaneval-code-benchmarks)
- **HumanEval Benchmark (Emergent Mind):** [https://www.emergentmind.com/topics/humaneval](https://www.emergentmind.com/topics/humaneval)
- **HumanEval Review:** [https://www.emergentmind.com/topics/humaneval-coding-benchmark](https://www.emergentmind.com/topics/humaneval-coding-benchmark)
- **Code Generation Evaluation Guide (Towards Data Science):** [https://towardsdatascience.com/a-gentle-introduction-to-code-generation-evaluation-c8dff8c3d19a](https://towardsdatascience.com/a-gentle-introduction-to-code-generation-evaluation-c8dff8c3d19a)

### Agent Evaluation Guides
- **Machine Learning Mastery: Agent Evaluation:** [https://machinelearningmastery.com/agent-evaluation-how-to-test-and-measure-agentic-ai-performance/](https://machinelearningmastery.com/agent-evaluation-how-to-test-and-measure-agentic-ai-performance/)
- **Braintrust: AI Agent Evaluation Framework:** [https://www.braintrust.dev/articles/ai-agent-evaluation-framework/](https://www.braintrust.dev/articles/ai-agent-evaluation-framework/)
- **TestRigor: Different Evals for Agentic AI:** [https://testrigor.com/blog/different-evals-for-agentic-ai/](https://testrigor.com/blog/different-evals-for-agentic-ai/)
- **Master of Code: AI Evaluation Metrics 2026:** [https://masterofcode.com/blog/ai-agent-evaluation](https://masterofcode.com/blog/ai-agent-evaluation)
- **LXT AI: AI Agent Evaluation Framework:** [https://www.lxt.ai/blog/ai-agent-evaluation/](https://www.lxt.ai/blog/ai-agent-evaluation/)
- **Maxim: AI Agent Evaluation Metrics:** [https://www.getmaxim.ai/articles/ai-agent-evaluation-metrics-strategies-and-best-practices/](https://www.getmaxim.ai/articles/ai-agent-evaluation-metrics-strategies-and-best-practices/)

### Cost & Efficiency Analysis
- **Token Cost Trap (Medium):** [https://medium.com/@klaushofenbitzer/token-cost-trap-why-your-ai-agents-roi-breaks-at-scale-and-how-to-fix-it-4e4a9f6f5b9a](https://medium.com/@klaushofenbitzer/token-cost-trap-why-your-ai-agents-roi-breaks-at-scale-and-how-to-fix-it-4e4a9f6f5b9a)
- **Cost of AI-Assisted Coding (Benevol 2025):** [https://benevol2025.github.io/pre/paper03.pdf](https://benevol2025.github.io/pre/paper03.pdf)
- **Understanding Real Cost of AI Agents:** [https://www.godofprompt.ai/blog/understanding-the-real-cost-of-ai-agents](https://www.godofprompt.ai/blog/understanding-the-real-cost-of-ai-agents)
- **Cost of AI Agents (Galileo AI):** [https://galileo.ai/blog/hidden-cost-of-agentic-ai](https://galileo.ai/blog/hidden-cost-of-agentic-ai)
- **AI Tokens Guide:** [https://guptadeepak.com/complete-guide-to-ai-tokens-understanding-optimization-and-cost-management/](https://guptadeepak.com/complete-guide-to-ai-tokens-understanding-optimization-and-cost-management/)
- **LLM Cost Per Token 2026:** [https://www.silicondata.com/blog/llm-cost-per-token](https://www.silicondata.com/blog/llm-cost-per-token)
- **Cost Per Token Basics:** [https://tetrate.io/learn/ai/cost-per-token](https://tetrate.io/learn/ai/cost-per-token)
- **Pay-Per-Task vs Token Pricing:** [https://cosine.sh/blog/ai-coding-agent-pricing-task-vs-token](https://cosine.sh/blog/ai-coding-agent-pricing-task-vs-token)

### SLA & Production Metrics
- **AI Agent Latency, Cost, Safety Evaluation:** [https://www.aviso.com/blog/how-to-evaluate-ai-agents-latency-cost-safety-roi](https://www.aviso.com/blog/how-to-evaluate-ai-agents-latency-cost-safety-roi)
- **Agentic SLA: From Uptime to Outcome:** [https://dohost.us/index.php/2026/02/19/the-agentic-sla-moving-from-uptime-to-outcome-in-the-age-of-ai/](https://dohost.us/index.php/2026/02/19/the-agentic-sla-moving-from-uptime-to-outcome-in-the-age-of-ai/)

### Latency Measurement
- **Anyscale: LLM Latency and Throughput Metrics:** [https://docs.anyscale.com/llm/serving/benchmarking/metrics](https://docs.anyscale.com/llm/serving/benchmarking/metrics)

### LLM Evaluation Metrics
- **Confident AI: LLM Evaluation Metrics Guide:** [https://www.confident-ai.com/blog/llm-evaluation-metrics-everything-you-need-for-llm-evaluation](https://www.confident-ai.com/blog/llm-evaluation-metrics-everything-you-need-for-llm-evaluation)
- **DeepEval: LLM Evaluation Framework:** [https://deepeval.com/docs/benchmarks-human-eval](https://deepeval.com/docs/benchmarks-human-eval)
- **DeepEval: AI Agent Evaluation:** [https://deepeval.com/guides/guides-ai-agent-evaluation](https://deepeval.com/guides/guides-ai-agent-evaluation)
- **Confident AI: RAG Evaluation Metrics:** [https://www.confident-ai.com/blog/rag-evaluation-metrics-answer-relevancy-faithfulness-and-more](https://www.confident-ai.com/blog/rag-evaluation-metrics-answer-relevancy-faithfulness-and-more)
- **Zep: LLM Evaluation Framework:** [https://www.getzep.com/ai-agents/llm-evaluation-framework/](https://www.getzep.com/ai-agents/llm-evaluation-framework/)

### Open-Source Evaluation Tools
- **EleutherAI LM Evaluation Harness:** [https://github.com/EleutherAI/lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness)
- **EleutherAI Evaluating LLMs:** [https://www.eleuther.ai/projects/large-language-model-evaluation](https://www.eleuther.ai/projects/large-language-model-evaluation)
- **Hugging Face Blog: Integrating Benchmarks:** [https://huggingface.co/blog/Neo111x/integrating-benchmarks-into-lm-evaluation-harness](https://huggingface.co/blog/Neo111x/integrating-benchmarks-into-lm-evaluation-harness)
- **Mozilla: Evaluation Harness for Auditing LLMs:** [https://www.mozillafoundation.org/en/blog/evaluation-harness-is-setting-the-benchmark-for-auditing-large-language-models/](https://www.mozillafoundation.org/en/blog/evaluation-harness-is-setting-the-benchmark-for-auditing-large-language-models/)
- **Medium: Evaluating LLMs with lm-evaluation-harness:** [https://medium.com/@kimdoil1211/evaluating-llm-accuracy-with-lm-evaluation-harness-for-local-server-a-comprehensive-guide-933df1361d1d](https://medium.com/@kimdoil1211/evaluating-llm-accuracy-with-lm-evaluation-harness-for-local-server-a-comprehensive-guide-933df1361d1d)
- **KDnuggets: Top 5 Open-Source LLM Evaluation Platforms:** [https://www.kdnuggets.com/top-5-open-source-llm-evaluation-platforms](https://www.kdnuggets.com/top-5-open-source-llm-evaluation-platforms)

### LangChain Tools
- **LangChain SWE-bench Documentation:** [https://docs.langchain.com/langsmith/swe-benchmark](https://docs.langchain.com/langsmith/swe-benchmark)

---

## Other Reference Materials

### Benchmark-Specific Resources
- **Aider Benchmarking Guide (Onegen):** [https://onegen.ai/project/benchmarking-llms-with-aider-a-comprehensive-guide-to-performance-evaluation/](https://onegen.ai/project/benchmarking-llms-with-aider-a-comprehensive-guide-to-performance-evaluation/)
- **Klu: HumanEval Benchmark Overview:** [https://klu.ai/glossary/humaneval-benchmark](https://klu.ai/glossary/humaneval-benchmark)
- **Emergent Mind: HumanEval Benchmark:** [https://www.emergentmind.com/topics/humaneval](https://www.emergentmind.com/topics/humaneval)
- **Emergent Mind: FeatBench:** [https://www.emergentmind.com/papers/2509.22237](https://www.emergentmind.com/papers/2509.22237)
- **SWE-bench (VALS AI):** [https://www.vals.ai/benchmarks/swebench](https://www.vals.ai/benchmarks/swebench)
- **BigCodeBench (Hugging Face Blog):** [https://huggingface.co/blog/leaderboard-bigcodebench](https://huggingface.co/blog/leaderboard-bigcodebench)
- **Emergent Mind: SWE-bench Verified Issues:** [https://www.emergentmind.com/topics/swe-bench-verified-issues](https://www.emergentmind.com/topics/swe-bench-verified-issues)

### Performance Reports
- **Claude Opus 4.5 vs GPT-5 Benchmarks (Caylent):** [https://caylent.com/blog/claude-sonnet-4-5-highest-scoring-claude-model-yet-on-swe-bench](https://caylent.com/blog/claude-sonnet-4-5-highest-scoring-claude-model-yet-on-swe-bench)
- **Claude 4.5 Benchmarks (Hugging Face):** [https://huggingface.co/blog/Laser585/claude-4-benchmarks](https://huggingface.co/blog/Laser585/claude-4-benchmarks)
- **Claude Opus 4.5 SWE-Bench (Abaka AI):** [https://www.abaka.ai/blog/claude-4-5-80-percent-coding-benchmark](https://www.abaka.ai/blog/claude-4-5-80-percent-coding-benchmark)
- **Claude Benchmarks Explained (Vellum):** [https://www.vellum.ai/blog/claude-opus-4-5-benchmarks](https://www.vellum.ai/blog/claude-opus-4-5-benchmarks)
- **Claude Overview (Leanware):** [https://www.leanware.co/insights/claude-sonnet-4-5-overview](https://www.leanware.co/insights/claude-sonnet-4-5-overview)
- **Warp: SWE-bench Verified Results:** [https://www.warp.dev/blog/swe-bench-verified](https://www.warp.dev/blog/swe-bench-verified)

### Mini-SWE-Agent
- **GitHub:** [https://github.com/SWE-agent/mini-swe-agent](https://github.com/SWE-agent/mini-swe-agent)

### Multi-Modal & Vision-Language
- **BentoML: Vision Language Models 2026:** [https://www.bentoml.com/blog/multimodal-ai-a-guide-to-open-source-vision-language-models-in-2026](https://www.bentoml.com/blog/multimodal-ai-a-guide-to-open-source-vision-language-models-in-2026)
- **ShowUI Vision-Language-Action Model (CVPR 2025):** [https://openaccess.thecvf.com/content/CVPR2025/papers/Lin_ShowUI_One_Vision-Language-Action_Model_for_GUI_Visual_Agent_CVPR_2025_paper.pdf](https://openaccess.thecvf.com/content/CVPR2025/papers/Lin_ShowUI_One_Vision-Language-Action_Model_for_GUI_Visual_Agent_CVPR_2025_paper.pdf)
- **Modular Multi-Agent Vision-to-Code:** [https://www.emergentmind.com/topics/modular-multi-agent-vision-to-code-frameworks](https://www.emergentmind.com/topics/modular-multi-agent-vision-to-code-frameworks)
- **Ionio: Vision AI in Test Automation:** [https://www.ionio.ai/blog/how-we-automate-ui-testing-with-multimodal-llms-llama-3-2-and-gemini-api](https://www.ionio.ai/blog/how-we-automate-ui-testing-with-multimodal-llms-llama-3-2-and-gemini-api)
- **Unmesh: Vision AI in Test Automation:** [https://unmesh.dev/post/vision_ai/](https://unmesh.dev/post/vision_ai/)

### Misc Code-Related Benchmarks
- **Measuring AI Code Generation Quality:** [https://www.walturn.com/insights/measuring-the-performance-of-ai-code-generation-a-practical-guide](https://www.walturn.com/insights/measuring-the-performance-of-ai-code-generation-a-practical-guide)
- **Performance Benchmarks for Code-Generation AI:** [https://kodekx-solutions.medium.com/performance-benchmarks-and-metrics-for-code-generation-llms-e-g-qwen-coder-abe6d1ee7c60](https://kodekx-solutions.medium.com/performance-benchmarks-and-metrics-for-code-generation-llms-e-g-qwen-coder-abe6d1ee7c60)
- **Code Generation Quality Assessment (arXiv):** [https://arxiv.org/pdf/2310.18834](https://arxiv.org/pdf/2310.18834)
- **Large Language Models for Code Survey:** [https://test.koyauniversity.org/index.php/aro/article/download/2159/482](https://test.koyauniversity.org/index.php/aro/article/download/2159/482)

### One-Hour SWE-bench Setup
- **Epoch AI: Run SWE-bench Verified in One Hour:** [https://epoch.ai/blog/swebench-docker](https://epoch.ai/blog/swebench-docker)
- **GitHub: SWE-bench Docker:** [https://github.com/aorwall/SWE-bench-docker](https://github.com/aorwall/SWE-bench-docker)

### 8 Benchmarks Overview
- **AI Native Dev: 8 Benchmarks Shaping Next Gen AI Agents:** [https://ainativedev.io/news/8-benchmarks-shaping-the-next-generation-of-ai-agents](https://ainativedev.io/news/8-benchmarks-shaping-the-next-generation-of-ai-agents)

---

## Summary Statistics

| Category | Count |
|----------|-------|
| Official Benchmarks & Leaderboards | 25+ URLs |
| Academic Papers | 25+ papers |
| Educational Resources | 35+ guides/tutorials |
| Official Vendor Publications | 7 official sources |
| Open-Source Tools | 5 major frameworks |
| **Total Unique Sources** | **97+ resources** |

---

## Research Notes

- **All links verified as of:** February 21, 2026
- **Update frequency:** Benchmarks and leaderboards update frequently (weekly-monthly)
- **Archival note:** Recommend saving PDFs of important papers (arXiv can change)
- **Code repositories:** All GitHub links point to main branches (may evolve)

**Note on Completeness:** This is not an exhaustive list of all papers citing these benchmarks, but rather the most authoritative and useful sources for practical evaluation framework development.

