Did you try to change the threshold, e.g. the frequency at which the SI5351 switches to harmonics mode?
All nanoVNA use the SI5351 way out of spec.
Try
threshold 250000000
first and calibrate till 750MHz and see if it remains stable.
If stable step by step (10MHz) increase the threshold till the SI5351 overheats again.
--
NanoVNA Wiki:
/g/nanovna-users/wiki/homeNanoVNA Files:
/g/nanovna-users/filesErik, PD0EK