If you put a 10dB pad on the CH0/Port1 and then try to calibrate it, the "open" won't be an open circuit, it'll be whatever parallal resistance the pad exhibits to ground. Similarly the "short" won't be a short circuit; it'll be whatever series resistance is in the pad. A 50 ohm termination should cause the pad to produce something that resembles 50 ohms.
When trying to measure the input return loss of the preamp, the signal coming out of the NanoVNA is reduced by 10dB. The reflected signal due to the RL of the preamp is also attenuated by 10dB on its way back to the NanoVNA. So you end up with a RL that's already 20dB, not the actual preamp's RL.
Perhaps adding some attenuation to the outgoing NanoVNA signal could be compensated for by increasing the sensitivity to the incoming NanoVNA signal, to deal with the SNR issue. I didn't have such a problem using a SA/TG/RLB at -30dBm coming out of the TG.
Bob M.