Save it or spend it?

So, after my earlier post, I had a thought about all the money I have set aside for retirement; I might actually be able to take some money out of my 401(k) for the down payment on a house. It really does sound like a good idea, would allow me to buy a better house, without paying a whole lot more per month. So, before I even looked into the requirements and hurdles I'd have to jump through to get the money, I decided instead to look at the impact doing that would have on my retirement.
I popped on over to Bankrate.com's calculators and checked out the difference between how much I'd have saved for retirement if I started with what I have, or $10,000 less (for the down payment). Obviously, all the numbers are fudged a bit, I have no idea how much any of it will be worth, but with the numbers that were already filled in, by having $10,000 less in my retirement account I would end up with over $100,000 less when I retire.
If I adjust the annual rate of return down, it makes an even bigger impact in the percentage of money I lose over time. At half the return rate, it eats away almost half the money I have saved. Now, none of these calculations take into account any ongoing savings, because those numbers won't change one way or another based on this choice.
So, is it worth taking $10,000 out of my 401(k) for the down payment on a house? No, not at all. Putting that extra amount of money down on a house would save me just around $20,000. But keeping it in my retirement account will net me a lot more than that.