The latest NanoVNA-Z-v1013M firmware for H4 has made 24 hours without a freeze..
After a week spent with messing with various parameters (Chibios, Si5351, AudioCodec, etc.) the major source of the instability with the latest firmware pinpoints the overclocking of the AIC3204 Audio Codec.
Its datasheet max MCLK is 50MHz. The older versions of the firmware used to use 86.016MHz, the latest 98.304MHz. The ADC sampling rate is derived from MCLK. The older versions worked with max 96kHz sampling, while the latest firmware allows up to 768kHz (advertised as an "improvement").
Well, I do understand a temptation to provide experiments like "what the existing hardware can bear", but for stable operation in the field such a a massive overclocking (of almost each critical component in the box) is something I would not recommend.
I've reverted the Audio Codec settings back to the older 86.016MHz and set 96kHz "Audio Codec sampling frequency" and 12kHz "frequency offset" as my default. That allows for up to 2kHz "Bandwidth". This setting is working "rock stable" here (at ambient 29degC).