Prompt engineering for medical foundational models

Abstract

Foundational models have rapidly emerged in recent years, demonstrating remarkable capabilities across a wide array of tasks, predominantly in natural language processing. Significant efforts have been dedicated to this field, resulting in the frequent release of new, increasingly sophisticated models. This thesis explores the efficacy of advanced prompting strategies applied to these foundational models within the medical question-answering domain, focusing on the potential of open-source models enhanced through sophisticated prompt engineering. An efficient and functional evaluation framework, named "prompt\_engine", has been developed to study the potential of two prompting strategies: Self-Consistency Chain of Thought and a Medprompt-based technique. Through this framework, a comprehensive range of experiments was conducted, leading to significant performance enhancements through the strategic combination and optimization of these prompting techniques. Key findings reveal that the performance of open-source models can be significantly enhanced, allowing them to outperform current state-of-the-art private models in existing medical question-answering benchmarks.