I tested windowing and that does not make a big difference as can be expected in this case where the stimulus is fully controlled
What is worrying me are the multiple peaks you get 1kHz apart next to the 5kHz.
Some SI5351 have difficulty locking above 250MHz (the official maximum output frequency) and your 1.29Ghz would use a 258MHz signal from one of the SI5351 outputs.
The 1kHz pattern is identical to what I sometimes see, but for me its closer to 300MHz and not always.
So the 1kHz peaks are not noise but an artefact of a SI5351 having difficulty to output the required signal.
There is already a firmware that limits the maximum fundamental to 250MHz to avoid this problem.
