How close can you build to the train tracks?

I guess it depends what Country you grew up in.  Click here to see the vegetable market and train track which seems to occupy just about the same space.  To view click here:    From Tiny in NH.