I just did the experiment here with my hp 355D 100 dB attenuator/ 10 dB steps.
Attenuation is spot on and the VNA follows it just fine all the way down to 80 dB.
No surprises at HF.
If you have a COM receiver, certainly with sensitivity of better than -80 dBm and a sig gen,
try that combo and see if you have a defective step attenuator.
QSL. Alan