I suspect a problem in the wires going to the volume control. If you look at the schematic those wires carry a weak AF signal that then goes to the final AF amplifier. Any electronic noise picked up from the uBitx board, Raduino or outside air will be amplified along with the signal. Try shortening and/or rerouting those wires. From your picture it looks like you have no case shielding, so the mixers and amplifiers may pick up any stray signal floating in the air.
My uBitx had a great clean receiver. I added several extra wires to connect an internal audio filter for CW in the line to the volume control and the noise floor went up significantly - so I know that you can screw it up that way.
If you touch a component the capacitance of your body can affect the circuit. If touching the case or tapping the table produces an effect without a mic plugged in you may have a loose connection. Loose connections, especially loose grounds, are also a great source of noise.
Bob Benedict, KD8CGH