Checked some games again that I thought I had the same problem with but I forgot it was actually a different problem with the taskbar just disappearing completely. Mostly I've switched to playing games in real windowed or real fullscreen on this monitor. I'll update if I find another game with the same problem as on the first post.
As for the taskbar disappearing on second split (the one with DF taskbar) while the game is running, it looks like this
this and the second taskbar only sometimes very briefly appears for half a second when I press windows button but usually not even then.
It happened with all but one game that I tested in borderless windowed fullscreen:
Grand Theft Auto V
Witcher 3
Warframe
Final Fantasy XV Windows Edition
Slay the Spire
Tomb Raider (2013)
Civilization VI
Guild Wars 2
The one that worked perfectly (in borderless windowed fullscreen):
Assassin's Greed Odyssey
With AC
, it works exactly like I want it to: both taskbars being there always when the game is minimized but disappearing instantly after I go back to the game.
I did find one "solution" for the original problem without disabling taskbars: Game executable -> Properties -> Compatibility (tab) -> Disable fullscreen optimizations (checked)
That kind of just makes the game behave as if it was a true fullscreen though so I guess that's why it works.
