TeamSpeak server lagging - everyone suddenly has outgoing packet loss in 13.3.3

I am not entirely sure, but I think this didn’t happen before I updated the server to 13.3.3.
I have tried three different hosts, but every time about ones a day my Teamspeak server starts lagging resulting in outgoing packet loss for everyone. It’s usually not that high, but sometimes it jumps to 50%+ packet loss. Then while speaking the packet loss slowly goes down and then sticks to its value when you stop to speak. The packet loss occurs in “Speak”, “Keep Alive” and “Control”.
The VPS is always fine and not lagging at all - this only happens to the TeamSpeak server installed on it.
Also, the resource usage of the VPS is always pretty low.

Anyone has the same problem and maybe has solved this issue?

Thanks in advice,
kind regards.