I’m 45 with $3.2 million saved. Can I really quit my toxic job without risking retirement?

Priya, a 45-year-old caller to Jill Schlesinger’s Jill on Money show, opened with a common knot: she loves her career but her current job has turned ugly. “I have been working a long time, 25, 26 years, and I’m currently working in a full-time remote position right now. And my job has become a bit of a toxic situation right now,” she explained. She wanted to step away for six or seven years but feared AI would lock her out of her field on the way back in.

Schlesinger’s answer was unhedged. “Life is short, my friend, and if you have working in a toxic environment and you feel like it’s just unbearable, I do not think you should do this anymore. I really don’t,” she told Priya. “To have this much money, to have $3.2 million in your 40s, as you come into your late 40s, there’s nothing bad that’s going to happen to you.”

The math backs the advice

The reason is Priya’s balance sheet against her spending. The household holds $1.8 million in traditional retirement accounts (401(k), 403(b), 457s), $428,000 in Roth accounts, $834,000 in a brokerage account, and over $200,000 in cash. They own an $850,000 home with $127,000 left on a 2.25% mortgage, and they have $259,000 saved in 529 plans for kids ages 11 and 14.

Annual spending is $120,000 to $150,000. The husband still earns $217,000 a year. That single income alone, after tax, covers household spending without touching the portfolio. Priya’s $205,000 salary plus variable comp is gravy funneled into maxed-out retirement accounts.

The key mechanic is the safe withdrawal rate. The 4% rule, derived from the Bengen and Trinity studies, says a diversified portfolio can fund roughly 4% of its starting value in year one, adjusted for inflation each year afterward, with a high probability of surviving 30 years. Applied to a $3.2 million portfolio, that supports a withdrawal around $128,000 in year one. That figure sits inside Priya’s stated spending range, and she has not yet reached the age at which most retirement studies start the clock.

Layer in time. Priya is 45. If the portfolio compounds at a historically ordinary real return, it roughly doubles by age 60. Co-host Mark made exactly that point, and Schlesinger agreed. The portfolio is doing the heavy lifting now.

The variable that changes everything: lifestyle drift

Whether Priya’s plan holds depends on whether spending stays anchored. At $120,000 a year, the portfolio has wide margin. At $200,000 a year, the picture tightens fast, because she loses her income, loses matching contributions, and starts drawing earlier than a traditional retiree.

This is why Schlesinger pushed back on buying a second home. New fixed costs (property taxes, insurance, maintenance, a second mortgage at current rates rather than 2.25%) reset the spending floor permanently. A $30,000 annual increase in fixed outflows is the difference between a portfolio that comfortably funds a 40-year horizon and one that needs Priya back at work in five years on someone else’s terms.

The same logic governs the kids. The $259,000 in 529 plans for an 11 and 14 year old is meaningful but not a blank check for private universities.

What to actually do

Schlesinger’s prescription was specific. Take the exit. Detox first. “When you’re around toxic people, you need to detox. You need to like, take a minute. You got to nest with your family, be a better spouse, be a better parent, be nice to yourself.” Do not buy a second home in the first year. And “Don’t burn bridges” on the way out, because the field she fears AI will close is the same field that will pay her if she wants part-time work in year three.

For a reader running a similar calculation:

1. Write down actual spending for the last 12 months, not an estimate. The gap between $120,000 and $150,000 is $30,000 a year, and over a 40-year horizon that gap is the entire question.
2. Stress-test the portfolio at a 3.5% withdrawal rate, not 4%, because Priya is retiring 20 years earlier than the studies assume. A more conservative rate is the price of a longer runway.
3. Map the bridge to age 59½ using the brokerage account and cash so traditional retirement money keeps compounding untouched and early withdrawal penalties never enter the picture.
4. Confirm health insurance through the spouse’s employer plan before resigning, because an ACA marketplace plan at this income level is the largest variable cost most early retirees underestimate.

Schlesinger’s core point: money that buys you the option to leave a job you hate is only useful if you actually use it.