A few comments after reviewing the attached schematics.
You might take a look at the LT5560 for mixer. It has lower input Z over a broader freq range, lower current drain, allows higher value I.F. load resistors yielding higher gain, and should cost less then half of the AD8342.
Four SP4T RF switches from S11 input to mixer, really !!. Each switch has some parasitics due to switch and PCB layout. As I previously mentioned, you should not calibrate Open, Short, Load internally. The calibration needs to be closest to the measurement point to include the path delay to measurement point accurately. User would have to make a guess at their delay to measurement point and manually enter it with the internal calibration system.. This is not practical for good accuracy.
I don't understand the purpose of the two switches after the S11 bridge balun. I guess it is to help isolate S11 port when switching single RF mixer over to S21 port. It would be much better to dump the RF switches and put in the second mixer / I.F. op amp for S21 port. How much do all these RF switches cost? Only legit place for RF switches would be to switch between low freq and high freq synthesizers.
Fyi, The N1201SA is much cleaner then nano's with much better dynamic range. Most of the improvement is use of the ADF4351 synthesizer but there are a few other items I have noticed.
The N1201SA makes use of the ADC's in the STM32F103 uC as you mentioned you are planning to do. I was surprised to find that the I.F. frequency is changing for higher freq band groups.
137.5 to 275 MHz is using 10 KHz I.F.
275 to 550 MHz is using 20 KHz I.F.
550 to 1100 MHz is using 40 KHz I.F.
1100 to 2200 MHz is using 60 KHz I.F.
2200 to 2700 MHz is using 80 KHz I.F.
This may be done to move the I.F. higher because the sideband noise level on the synth's gets worse farther out from source freq as synth output frequency increases, By rights, the frequencies below the I.F. output freq should be attenuated to take full advantage of SNR improvement with this scheme. I don't see any low freq I.F. filtering. The STM32 ADC should not have any issue with 80 KHz I.F. with its near 1 Msps capability.
The second very large surprise I found was the L.O. source for the SA612 mixers is fed fundamental freq. It is dumping 2.7 GHz L.O. source into the SA612 LO port! Talk about running outside of part spec! It is probably starving the Gilbert cell mixer of LO drive but I guess they found the performance was better then injecting sub-harmonics and running as harmonic mixer.
Looking at the I.F. signal output on a scope, the nano is no comparison to the N1201SA. Nano I.F. jitter is bad due to Si5351 synth. I also see glitches which I believe is due to the codex ADC DCOC zeroing calibration switching. The codex needs a lower driving source Z. I noticed the nano-F has decreased the LPF resistors from 15K on original nano schematic to 4.7K for nano-F. This is probably still not low enough. It is probably better to replace the LPF series resistors with ferrite chokes and adjust LPF caps values to work with SA612 internal 1.5K pull up resistors for LPF. Most SAR ADC have this cap switching to null out DC offset and start from zero volts This can create glitches when there are series coupling caps to ADC inputs. I found putting 100 pF to 1000 pF caps across ADC input fixes this but you have to be careful not to impact input signal bandwidth with the shunt caps. They have to be directly on ADC input pins.
I have not researched possible candidates to replace the Si5351 to accomplish the low frequencies below 35 MHz which is limit of AD4351. It is going to be tough to get the balun performance below 5 MHz and keep the GHz range. Personally I would be happy with 5 MHz low end but some folks may not be happy with that.
If you don't have a N1201SA you should get one. It is your benchmark. V2 should be equal or better. With better mixers (and synthesizer) there is no reason not to be better.
