Mind if I ask why you’re sticking with Debian for gaming? Debian’s awesome for servers because you want the stability; but for gaming, you generally want nice fresh packages, updated drivers, etc etc. I’d use Bazzite or Nobara – the former if you’re comfortable with immutable/atomic paradigm, the latter if not.
I gave up on KVMs.
Each of my screens has more than one input, my mouse & keeb can each pair with 3 PCs and there’s a button on to toggle between.
Sure, it’s a few more button presses, but it doesn’t cost me a dime for functionality that’s baked in already.