Two RP2 in 2 different rooms, play both files?

I have RP2 in Room A, and another Room B, both with static addresses, can I play Room B files from Room A RP?

how do I set this up?

The upnp functionality should allow you to easily play files stored on another system providing the files are shared through a DLNA/UPnP server (such as minidlna). PiMusicbox is designed as a client/renderer and doesn’t come with a DLNA server installed. The idea is that usually you’d have all your files on a central NAS and run a DLNA server on that. But I don’t think there is any reason you can’t install minidlna (or equivalent) on your Raspberry Pis and have them as both server and clients to each other.