Well,
It was the filter. I did not seem to have any problems with the passband, but it must have been lossy. I put in a spare XF-92A that I had laying around and it seems ok now. I can get good modulation without having to crank the mic gain.
It is ashamed, I was going to use that filter for a project.
