Hi Steve, With respect to the IO voltages, I believe that the sbitx peripheral devices (SI5351A, RTC) and the RPI4 are 3V3, so you should avoid driving 5V.
Generally I2C interfaces only drive logic ‘0’ and have pull-ups for the logic ‘1’, so you may be okay with a direct connection form the Digispark module.
However, to avoid any issues if you a programming whilst in circuit, you could use one of the cheap 2-channel MOSFET based level shifter modules. An example schematic of a 4-channel version is shown below.

For the few projects where I have used the ATtiny85, I use a USBASP (clone) programmer (Approx ?4.50 in the UK) and a simple homebrew board with a socket for the ATtiny.

Also, I agree it is best to avoid loading the 3V3 regulator that feeds the SI5351, since this may cause more noise to be generated on the clock outputs.
Good luck with your experiments and updates.
73, Dave