Can I join a server from down the road or do I have to be in the same room?

You must be connected to the same WiFi network as the other person. So if you can connect to their network from down the road, then yes, you can.

I have had a problem similar to the origins of your question, but nonetheless I have done a little bit of testing to achieve the answer to your question. The WiFi has to be the same. If one person is on one WiFi access point and the other is on a separate access point, both cannot join the same server. But if the person down the road is able to connect to the WiFi network that you are on, you should be able to connect to another server.

If it is just a singleplayer world, then you have to be connected to the same wifi if it is an external server, then all you need is the password