I remember when I felt that way like 10 years ago. Then I forced myself to play through a couple games and got over it.
For competitive or super fast stuff like this ROTT remake, yeah, a controller can't begin to keep up with mouse precision. However, for almost all singleplayer/co-op games these days it doesn't even begin to matter.

