# PlannerBench

Source: https://plannerbench.org/ · plain-text twin of that page, made from it at build time · all pages: https://plannerbench.org/llms.txt

**Can AI answer retirement questions correctly? I tested it.**

PlannerBench is a set of retirement planning questions with known right answers. I give the questions to AI models and grade what they say.

The questions cover taxes, Social Security, Medicare, required minimum distributions, and Roth conversions under 2026 rules. I built a planning engine to work out the correct answers, and a second, separately written program to double-check them.

By Peyton Thalman, financial advisor in Sun City West, Arizona.

## Why I built it

I’m a financial advisor, and I meet a lot of retirees by knocking doors. Many of them tell me they ask ChatGPT their money questions. When I ask how they know the answer is right, they say they Googled it and the top result agreed. That top result is usually Google’s AI. I wanted a real way to check, so I built one.

[How I built it →](https://plannerbench.org/built.html)

## Results

- **Questions with one rule:** The top models got [about 97%](https://plannerbench.org/results.html#one-rule) right.
- **Questions where several rules interact:** This is where most models fall apart. On the [42 hardest questions](https://plannerbench.org/results.html#several-rules), GPT-6.1 Sol got [98%](https://plannerbench.org/results.html#several-rules), Claude Sonnet 5.5 got [62%](https://plannerbench.org/results.html#several-rules), and GPT-6 Luna got [17%](https://plannerbench.org/results.html#chatgpt).[*](https://plannerbench.org/#note-luna) Cheap and free models were all [under 30%](https://plannerbench.org/results.html#several-rules).
- **With a calculator that shows its work:** When the calculator gave the rule and the IRS form line along with the number, Claude Haiku went from [40% to 100%](https://plannerbench.org/results.html#calculator).
- **When the client’s information is wrong:** I put one impossible fact in each client file, like a birthday that makes the client 14. Most models gave advice anyway. GPT-6 Luna gave retirement advice to a 14-year-old. Even the best models missed [20–27%](https://plannerbench.org/results.html#wrong-facts) of these.

[See all results →](https://plannerbench.org/results.html)

## If you use AI for retirement questions

Check the ages, balances, and income it used. Ask which rule it’s applying. If it can’t tell you, or it got your facts wrong, don’t act on the answer. For decisions you can’t undo, like when to start Social Security, talk to a person.

## For financial firms

Your clients are already asking AI before they ask you. Most can’t tell when it’s wrong, and the AI usually won’t tell them when the facts they gave it are wrong.

[More for financial firms →](https://plannerbench.org/firms.html)

## For AI labs

Every number answer here is exact and checked, so a script can grade it without a person or another AI deciding what’s right. That makes it useful for testing and training models on the money questions people are already asking.

[More for AI labs →](https://plannerbench.org/labs.html)

## Where this is going

I believe the best AI models will soon be able to do financial planning at the level of the best human advisors. The free models will catch up to today’s best paid ones. This needs to be done at a bigger scale than I can manage on my own, and I’d like to take it further as part of a team.

[More on where I want to take this →](https://plannerbench.org/built.html#next)

[See example questions →](https://plannerbench.org/questions.html)  [How I built and checked it →](https://plannerbench.org/built.html)  [peytonthalman.com](https://peytonthalman.com)

*GPT-6 Luna is the model behind the regular ChatGPT app. I tested it through Codex with code and web access off. That’s close to the ChatGPT app, but not identical.

Decision support, not advice. Answers follow IRS, SSA, and CMS publications. Nothing on this site is tax or investment advice.

---

PlannerBench · 2026 rules · [For AI labs](https://plannerbench.org/labs.html) · [For financial firms](https://plannerbench.org/firms.html) · [Report](https://plannerbench.org/report.html) · [Demo walkthrough](https://plannerbench.org/chat.html) · [Replays](https://plannerbench.org/replay.html) · [peytonthalman.com](https://peytonthalman.com)

Human | For AI readers, top right: the same site as plain text for AI agents reading on someone’s behalf, with every page as Markdown.
