The American Radio Relay League (United States ham organization) has a lab test procedure that uses 700 and 1900 tones for a two tone intermodulation test. Their procedure sets the SA RBW to 100 Hz, but that may be a bit overkill. As long as the skirt of the RBW filter is down 50 dB at 1.2 kHz, you could reliably measure -40 dB intermodulation. Don't know how much relief that gives on the filter performance requirement, but I am sure it is some.
