Almost everyone has the drilling accuracy problem. It is impossible to drill a 0.500 hole with a 1/2" drill bit in a drill press. All drill press chuck assemblies have a little play, or they would not be able to move up and down very easily, or not at all. Drill chucks are not guaranteed to grip perfectly true. Drill bits generally have some wobble or flex, and have a diameter tolerance range.?? Reamers in collets on a lathe or mill will get you a lot closer but it still may take a lot of effort to even get close to 0.500.
In the case of your face plate adventure, you could put hours of exacting machine practice and science into getting the chuck almost exactly aligned. Or you can accept the realities of the tools and equipment you have and make allowances to calibrate the assembly, the best you theoretically can, after assembly. The little bit of slop in the holes might be needed to slightly move the chuck to get the center of the jaws aligned with the centerline of the lathe spindle and tailstock center. Having no tolerance will mean you may never be able to get it right.
