Hi Nick,
Yes, my traces look incredibly low tech. I still use those numerical design tables at the back of filter theory books and my HP calculator to design filters, so NanoVNA V2 Plus 4 is a bit new to me haha. Yes I know NanoVNA Saver, but I don't have a computer right at my bench, but maybe I should look into it. I also use TinySA Ultra (x2) and another more powerful signal generator and getting readable traces can be a bit of work...
I didn't capture the trace, but I also looked at Z1/S11 while output port terminated at 50. Those ripples correspond to where the input impedance goes a bit lower and slightly capacitive. So, it is entirely possible that the "ripple" can be partially offset by greater power pulled from the transistor, resulting in slightly lower eta but not much change in the output power at the BNC. We'll have to see that aspect of it in the actual board.
My filter is 5th order elliptic, one more zero in the transfer function than the topology in QMX's stock filters. It's barely visible in the photo but both inductors have parallel caps. A part of the lousy return loss and passband ripple is component values being slightly off, but this is actually not bad compared to a lot of wideband amplifier matching networks. The filter is also better than some used in other amateur transceiver kits. So, knowing the actual filter will probably be slightly better than this with values from E24 sequence that are closest to the theoretical calculations and also with higher quality Murata/Kyocera C0G/NP0 caps, I'm not too worried. I also implement those caps on the back of the board with SMD components so if I see a better design later, they are easy to change.
In terms of the design challenge, I think the hard part is that we have no good specification requirement as to the minimum attenuation of the second harmonic. If it's 15 dB, I could design a filter with higher cutoff and less passband ripple. It's a trade-off but I don't know what I'm trading off. So, the design shown here probably turns out wayy too conservative in terms of meeting FCC emission limits and too aggressive on the passband, but I'd rather err that way on my first version.
Incidentally, in relation to another thread of a mismatched load causing high voltage at the transistor's drains, I also looked at Z1/S11 while the output is open or shorted. Of course, the contour of Z(f) on the Smith chart looks more like a very long transmission line, basically marching right on the unit (outer) circle where the phase moves rapidly with the frequency. So, a new, easier way to find a point that maximally stresses the transistor is to short or open the antenna connector, and sweep the transmit frequency while observing the drain voltage. Why didn't I think of that while spending time on that thread.
Thanks for the good discussion, though.