Development of Prompting Techniques to Trigger Proactive Dialogues in Large Language Models
DOI:
https://doi.org/10.62146/ijecbe.v4i1.187Keywords:
LLM, Proactive Dialogue, Prompting, Artificial Intelligence, Proactive Chain-of-ThoughtAbstract
The rise of Artificial Intelligence (AI) challenges the integrity of traditional educational assessments, positioning the oral examination as a robust method for verifying authentic understanding. However, its large-scale implementation is hindered by significant logistical challenges, including time, resources, and evaluation consistency. This research addresses a foundational limitation for using Large Language Models (LLMs) in automated oral exams: their inherent passivity and failure to handle ambiguity, which are critical for effective assessment. To address this critical gap, we established an experimental framework and created two specialized datasets to systematically evaluate prompting techniques designed to elicit proactivity. Using the GPT-4o model, our comparative analysis reveals that the optimal strategy is highly task-dependent: for ambiguity detection (CNP), a Proactive Chain-of-Thought (PCoT) zero-shot approach achieved a near-perfect 0.99 F1-Score; for generating clarification questions (CQG), the PCoT few-shot variant was most effective; and in target-guided scenarios, a simpler proactive prompt proved superior. These findings provide foundational insights for developing automated oral examiners capable of nuanced, human-like dialogue, thereby addressing the scalability issues of traditional assessment methods.
References
P. Black and D. Wiliam. Inside the Black Box: Raising Standards Through Classroom Assessment. [On-
line]. Available. King’s College London, 1998. URL: https : / / www . rdc . udel . edu / wp - content /
uploads/2016/06/Inside-the-Black-Box-C.A.-1998.pdf .
D. R. E. Cotton, P. A. Cotton, and J. R. Shipway. “Chatting and Cheating: Ensuring Academic
Integrity in the Era of ChatGPT”. In: Innovations in Education and Teaching International 61.2 (2023),
pp. 228–239.
E. Kasneci et al. “ChatGPT for Good? On Opportunities and Challenges of Large Language Mod-
els for Education”. In: Learning and Individual Differences 103 (2023), p. 102274.
L. Floridi and M. Chiriatti. “GPT-3: Its Nature, Scope, Limits, and Consequences”. In: Minds and
Machines 30 (2020), pp. 681–694.
S. Bubeck et al. Sparks of Artificial General Intelligence: Early Experiments with GPT-4. arXiv preprint
arXiv:2303.12712. 2023. arXiv: 2303.12712.
ElNiak. Understanding Prompt Engineering: A Comprehensive 2024 Survey. Medium. [Online]. Avail-
able. May 2024. URL: https : / / medium . com / @elniak / understanding - prompt - engineering - a -
comprehensive-2024-survey.
J. White et al. A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT. arXiv preprint
arXiv:2302.11382. 2023. arXiv: 2302.11382.
P. Liu et al. “Pre-Train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Nat-
ural Language Processing”. In: ACM Computing Surveys 55.9 (2023).
T. B. Brown et al. Language Models are Few-Shot Learners. arXiv preprint arXiv:2005.14165. 2020.
arXiv: 2005.14165.
L. Feng, M. Hong, and C. J. Zhang. Auto-Demo Prompting: Leveraging Generated Outputs as Demon-
strations for Enhanced Batch Prompting. arXiv preprint arXiv:2410.01724. 2024. arXiv: 2410.01724.
Learn Prompting. Instructions. Learnprompting.org. [Online]. Available. URL: https://learnprompting.
org/docs/basics/instructions.
J. Wei et al. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. arXiv preprint
arXiv:2201.11903. 2022. arXiv: 2201.11903.
Y. Wang et al. A Survey on Proactive Dialogue Systems: Problems, Methods, and Prospects. arXiv preprint
arXiv:2305.02750. 2023. arXiv: 2305.02750.
Z. Ji et al. “Survey of Hallucination in Natural Language Generation”. In: ACM Computing Surveys
12 (2023).
D. Traum and J. Allen. “A Speech Acts Based Approach to Dialogue Processing”. In: Proceedings of
the 30th Annual Meeting on Association for Computational Linguistics. 1992, pp. 203–214.
M. Gasic and S. Young. “Gaussian Processes for Sentence-Level and Word-Level Confidence Esti-
mation in Dialogue Systems”. In: IEEE/ACM Transactions on Audio, Speech, and Language Processing
2 (2016), pp. 241–253.
A. Kumar and P. Gupta. Survey on Clarification Question Generation. arXiv preprint arXiv:2006.12608.
arXiv: 2006.12608.
J. F. Allen, G. Ferguson, and A. Stent. “Towards a Robust Mixed-Initiative Dialogue System”. In:
Proceedings of the ACL-01 Workshop on Conversational Systems. 2001, pp. 1–8.
J. Zhu et al. A Survey on Large Language Models in Dialogue Systems. arXiv preprint arXiv:2307.13501.
arXiv: 2307.13501.
OpenAI. OpenAI API. https://openai.com. Accessed: July 31, 2025. 2024.
Steven Bird, Ewan Klein, and Edward Loper. Natural Language Processing with Python. Sebastopol,
CA, USA: O’Reilly Media, 2009.
Fabian Pedregosa et al. “Scikit-learn: Machine Learning in Python”. In: Journal of Machine Learning
Research 12 (2011), pp. 2825–2830.
The chardet developers. chardet: The Universal Character Encoding Detector (Python). https://github.
com/chardet/chardet. Accessed: July 31, 2025. 2024.
Y. Deng et al. Prompting and Evaluating Large Language Models for Proactive Dialogues: Clarification,
Target-Guided, and Non-Collaboration. arXiv preprint arXiv:2305.13626. 2023. arXiv: 2305.13626.
H. Naveed et al. A Comprehensive Overview of Large Language Models. arXiv preprint arXiv:2307.06435.
arXiv: 2307.06435.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 International Journal of Electrical, Computer, and Biomedical Engineering
This work is licensed under a Creative Commons Attribution 4.0 International License.
