The girls and I are working on an N-scale model train layout in my office, using some of my old stuff from my high-school and college layouts, and some new bits. I'll have some posts about that soon. I likes the little trains, we have a G scale train at Christmas time too.
Just like any hobby, you can quickly spend a lot of money on locomotives, cars, track, buildings, etc..
Take this guy, who spent hundreds of thousands of dollars building a garden railroad with 4000 real (not scale!) feet of track:
I thought that was impressive. But if you really want to be an old guy playing with trains....
Warren Buffett bought Burlington Northern Santa Fe (BNSF). All of it. The second largest railroad in the country, for $34 billion dollars. Now THAT'S a train set!! I would totally make them let me drive at least one train from LA to JAX if I were him.