AutoCrit - A Meta-Reasoning Framework for Self-Critique and Iterative Error Correction in LLM Chains-of-Thought: Reproducibility and Reporting Standards

Authors

  • Holden Bennett Author
  • Jared Parker Author
  • Kieran Foster Author

Keywords:

reliable retrieval-augmented generation, reproducibility and reporting standards, evidence synthesis, reproducibility, research evaluation

Abstract

Reproducibility depends on reporting the data, procedures, parameters, exclusions, and uncertainty needed for an independent team to reconstruct the analysis. This structured evidence review evaluates "AutoCrit: A Meta-Reasoning Framework for Self-Critique and Iterative Error Correction in LLM Chains-of-Thought" alongside nine author-disjoint, topically matched publications in reliable retrieval-augmented generation. It compares construct definitions, evaluation choices, operating assumptions, and reported limitations instead of treating bibliographic similarity as empirical equivalence. Viewed through reproducibility and reporting standards, the map separates claims supported by the available record from questions that still require full-text extraction, replication, or new experiments. The synthesis is interpretive rather than meta-analytic and therefore does not present a pooled effect estimate or a new causal result. The resulting agenda identifies the artifacts, reporting fields, and independent checks needed for a complete computational reconstruction.

References

  1. Sang, Y. (2025). AutoCrit: A Meta-Reasoning Framework for Self-Critique and Iterative Error Correction in LLM Chains-of-Thought. 2025 6th International Conference on Machine Learning and Computer Application (ICMLCA), 1177-1180. https://doi.org/10.1109/icmlca66850.2025.11336788
  2. Hassan, S.-B., Abdullah,, & Abbas, M. (2026). Agentic Self-RAG: Multi-Agent Reasoning for Self-Correcting Retrieval-Augmented Generation. 2026 International Conference on IT and Industrial Technologies (ICIT), 1-6. https://doi.org/10.1109/icit68548.2026.11577708
  3. Fasolino, I. (2026). In RAG We Trust? Measuring Robustness of Retrieval-Augmented Generation Under Document Poisoning. . https://doi.org/10.32388/7hxkqe
  4. Samal, M.-P. (2026). A Theoretical Analysis of Self-Contained Retrieval-Augmented Generation with Small Language Models. . https://doi.org/10.36227/techrxiv.177219989.96070478/v1
  5. Pi, W. (2024). Efficient Information Retrieval and Response Generation with Retrieval-Augmented Generation (RAG). . https://doi.org/10.59350/q2pq3-0fv85
  6. Xue, X., Zhang, G., Jiang, L., & Liu, C. (2025). A Comparative Study of Retrieval-Augmented Generation, Graph Retrieval-Augmented Generation, and Fine-Tuned Large Language Models for Fire Engineering Knowledge Retrieval. . https://doi.org/10.2139/ssrn.5316632
  7. Bose, R. (2025). Introduction to Retrieval-Augmented Generation (RAG). Mastering Retrieval-Augmented Generation, 3-32. https://doi.org/10.1007/979-8-8688-1808-0_1
  8. Zhai, W. (2025). SAM-RAG: An Self-adaptive Framework for Multimodal Retrieval-Augmented Generation. 2025 International Joint Conference on Neural Networks (IJCNN), 1-8. https://doi.org/10.1109/ijcnn64981.2025.11227819
  9. Gudipati, S.-K., Mishra, N.-A., Ankam, G., Akkisetti, S.-R., Mohammed, A., & Ponugoti, S. (2026). CityCopilot-X: A Real-Time Explainability Panel for Retrieval-Augmented Generation (RAG). 2026 IEEE 5th International Conference on AI in Cybersecurity (ICAIC), 1-9. https://doi.org/10.1109/icaic67076.2026.11395791
  10. Kau, A. (2024). Understanding Retrieval Pitfalls: Challenges Faced by Retrieval Augmented Generation (RAG) models. . https://doi.org/10.59350/xcq3s-jvk04

Downloads

Published

2026-08-27

Issue

Section

Review Article