I would assume, by looking at a digital copy of the 8601A service manual that two panel BNC to SMB cables mount to the plugs on the rear panel, then connect to VTO (J13) and LO input (J12), Fig 8-37 board A14, schematic 13 on page 8-33.
Seems straightforward enough, but I don't believe I have the cable-pair on hand. And I wouldn't want to do the modification like this one:
https://www.ebay.com/itm/255230575556?epid=1721885693&hash=item3b6ced83c4:g:IyYAAOSw7GhemVxK
Christopher Tucker