Assuming that the perf board which you are using is the type with circular pads at each hole (and not the older strip board), I have found that you can improve circuit isolation just by connecting unused pads together and to ground. In critical areas you might need to think a bit more about the layout before you start building. I didn't expect it to be so effective.
I don't have a VNA so it would be interesting if you could do some measurements. I was just comparing how much signal leaks around resistive attentuators.
73 Brian VK4BAP
