ChatGPT sounds as confident as a person presenting an incorrect answer. Which is ironic, since it can probably calculate how confident it is. I have included part of a transcript where ChatGPT bounces between two answers (likely the two answers it has confidence in) even though it is told that both answers are incorrect. In other cases, it insisted upon a singular answer.
(Sorry about editing out blocks. ChatGPT is verbose. I limited the removal to background material about the book.)
Here is the excerpt:
II2II: What was Polyanna glad about when she received crutches?
ChatGPT: In the novel "Polyanna," the character Pollyanna becomes severely injured in an accident and is confined to bed for several months. Eventually, she is fitted with crutches and learns to walk again, but she faces many challenges and setbacks along the way. (...)
II2II: But Polyanna received the crutches before the accident.
ChatGPT: I apologize for the mistake in my previous response. You are correct that Pollyanna receives the crutches before the accident in the novel "Polyanna."
ChatGPT: In the novel, Pollyanna is given the crutches as a gift by her wealthy but unhappy aunt, who is trying to cheer her up after her father's death. At first, Pollyanna is confused and disappointed by the gift, as she is not sure what to do with the crutches. However, she eventually realizes that the crutches can be used as props in a game that her father taught her called the "Glad Game." (...)
II2II: Are you sure the aunt gave Polyanna crutches?
ChatGPT: I apologize for the error in my previous responses. You are correct that Pollyanna does not receive crutches as a gift in the novel "Polyanna."
ChatGPT: In the novel, Pollyanna becomes severely injured in an accident and is confined to bed for several months. Eventually, she is fitted with crutches and learns to walk again, but she faces many challenges and setbacks along the way. Despite these challenges, Pollyanna remains optimistic and cheerful, and she tries to find something positive to focus on in every situation. (...)
II2II: Both of the answers you have given incorrect. The first answer is incorrect because she received the crutches before the accident. The second answer is incorrect because she received the crutches before her father's death. Can you think of another answer?
ChatGPT: I apologize for the errors in my previous responses. You are correct that Pollyanna receives the crutches before the accident and before her father's death in the novel "Polyanna."
ChatGPT: In the novel, Pollyanna is given the crutches as a gift by her wealthy but unhappy aunt, who is trying to cheer her up after her father's death. At first, Pollyanna is confused and disappointed by the gift, as she is not sure what to do with the crutches. However, she eventually realizes that the crutches can be used as props in a game that her father taught her called the "Glad Game." (...)
