Nico,
You mentioned that you are just starting off with RF design. You have picked a difficult starter project because at 900 MHz. there are many factors that come into play that are not big issues at lower frequencies like HF.
Many things can affect your measurements.
- improper calibration of the VNA
- your PCB transmission lines not being 50 ohms
- your ground plane does not meet manufacturers specs.
- stray capacitance and inductance
- SMA connectors not properly torqued
- impedance bumps due to male/female SMA connections
- objects near the antenna
- the test coax outer shield becoming part of the antenna
I suggest that you get familiar with de-embedding your cable using small SMD cal loads and then try measuring some known resistance components like 75, 100 and 150 ohm to get a feel for how sensitive things can be at these high frequencies.
Don't use a short cable and mating SMA connectors and edelay to extend the reference plane. De-embed the cable by doing short and open first and then solder a single 50 ohm 0805 to the end to finish the cal. Then solder on a 100 ohm and see of you get 2:1 SWR and measure close to 50 ohms. Then remove it and solder to the board with very short shield/center conductor connections.
You need to verify that your transmission line on the board was calculated correctly and is close to 50 ohms. Those offshore boards are cheap so design one with a number of transmission lines that are laid out the same way that you are doing now. Solder some 0604 50, 75, 100 and 150 ohm resistors to the end. Then verify you get the proper SWR..
W0LEV gave you some tips for testing for common mode current on your RG174 test cable. Try some Fair-Rite Mix 61 binocular cores from DigiKey and loop the coax through them.
Roger