Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models

Summary

This paper introduces a new prompting strategy called Plan-and-Solve (PS) prompting to improve the performance of large language models (LLMs) in multi-step reasoning tasks. The authors propose two components of PS prompting: devising a plan to divide the task into smaller subtasks, and carrying out the subtasks according to the plan. They also extend PS prompting with more detailed instructions to address calculation errors and improve the quality of generated reasoning steps, resulting in PS+ prompting.

The proposed prompting strategies are evaluated on ten datasets across three reasoning problems: arithmetic reasoning, commonsense reasoning, and symbolic reasoning. The experimental results show that zero-shot PS prompting consistently outperforms Zero-shot-CoT prompting across all datasets, is comparable to or exceeds Zero-shot-Program-of-Thought (PoT) prompting, and has comparable performance with 8-shot CoT prompting on the math reasoning problem.

Key Takeaways

Introduction

• Large language models (LLMs) have proven effective in various NLP tasks.
• Fine-tuning LLMs for downstream tasks is challenging due to limited access to model parameters.
• Zero-shot-CoT prompting has been successful in solving multi-step reasoning tasks but suffers from calculation errors, missing-step errors, and semantic misunderstanding errors.

Plan-and-Solve Prompting

• Plan-and-Solve (PS) prompting consists of two components: devising a plan to divide the task into smaller subtasks, and carrying out the subtasks according to the plan.
• PS prompting addresses missing-step errors by explicitly generating reasoning steps.
• PS+ prompting extends PS prompting with more detailed instructions to improve the quality of generated reasoning steps.
• PS+ prompting can be customized to solve a variety of problems other than math reasoning.

Experimental Results

• The proposed prompting strategies are evaluated on ten benchmark datasets.
• Zero-shot PS prompting consistently outperforms Zero-shot-CoT prompting across all reasoning problems and datasets.
• Zero-shot PS prompting is comparable to or exceeds Zero-shot-PoT prompting.
• PS+ prompting has a performance similar to an 8-shot CoT prompting in arithmetic reasoning.

Methods

Plan-and-Solve Prompting

• Step 1: Prompting for Reasoning Generation
  • Construct templates to elicit LLMs to determine subtasks and accomplish them.
  • Use a prompt with a simple template "Q: [X]. A: [T]" where [X] contains the input problem statement and [T] is a hand-crafted instruction to trigger LLMs to generate a reasoning process.
  • Replace "Let's think step by step" with "Let's first understand the problem and devise a plan to solve the problem. Then, let's carry out the plan and solve the problem step by step."
  • Add more detailed instructions to the trigger sentence, such as "pay attention to calculation", "extract relevant variables and their corresponding numerals", and "calculate intermediate results".
• Step 2: Prompting for Answer Extraction
  • Devise another prompt to extract the final numerical answer from the reasoning text generated in Step 1.

Experimental Setup

• Evaluate the proposed prompting strategies on ten benchmark datasets from three categories of reasoning problems: arithmetic reasoning, commonsense reasoning, and symbolic reasoning.
• Compare the performance of zero-shot PS and PS+ prompting with three types of prompting baselines: zero-shot-CoT, zero-shot-PoT, and few-shot with manual or automatic demonstrations.
• Use GPT-3 (175B) as the backbone language model and set the temperature to 0 for greedy decoding.

Conclusion

• Zero-shot PS prompting outperforms Zero-shot-CoT prompting and is comparable to or exceeds Zero-shot-PoT prompting.
• PS+ prompting has a performance similar to an 8-shot CoT prompting in arithmetic reasoning.
• The results suggest that PS prompting can generate a higher-quality reasoning process and has the potential to outperform manual few-shot CoT prompting.