Why is skill-based matchmaking so hard for studios to get right? [closed]

Not only is there a strong motivation to subvert it, it's also exceptionally easy to do. The major esports and multiplayer experiences, including Valorant and Destiny, are all free-to-play games. So to subvert the whole matchmaking, all you have to do is create a new account and suddenly you are a strong player, who is considered to be a complete newbie by the game, because there is no data. This is called a smurf, a strong player who deliberately beats up new players. However, some games take steps against that, e.g. League of Legends is quite good at detecting smurfs and will place them into a special smurf queue after a few games.

The second issue is playerbase. The main target of the matchmaking algorithm is to get you into a game, because developers would rather have you play the game against a less perfect match than sit in queue for 20 minutes. Especially highly ranked players often have very long queue times, because so few are at their level, that the game chooses to pit them against weaker opponents. This is mostly a problem in small to medium sized games, but may appear in larger games, too. For example I used to play Mordhau for a couple of weeks after release and had a ton of fun. Six months later, I came back and was just crushed, because the only people left playing are extremely strong players, who also drive all the newbies away, simply by virtue of you standing no chance against them.

Third is a technical problem. Sure, you have a ton of data, but sifting through that data takes time. Also how do you assign values to the different characteristics? E.g. you are good at hitting the opponent, but terrible at dodging or vice versa. Does that make you intermediate? Comparable to someone who can shoot and dodge okay? Do you want to match people based on an overall score or do you want to match people of similar personal characteristics? The more complex you make the matchmaking the more calculation time it will take, eventually causing queues to take very long simply because the algorithm takes so much time.

Your comparison to the real world doesn't really fit. The matchmaking is a completely automated algorithm. In real-world sports, humans decide how skilled you are. In every sports team you play in, there will be a coach who should know what you need to be able to do to reach the next level of skill and who can adapt to changing circumstances. This one human takes care of 10–20 people and fosters their growth as a player. People are paid money to serve as talent scouts for the sole purpose of assessing and contracting young talents.

Now you want an automated algorithm to do the same for potentially millions of players? The algorithm is limited and video games, due to content patches, change a lot more than traditional sports. A simple MMR based on your wins/losses simply works all the time. If you make the algorithm more complex, it may end up being wrong, because due to patches characteristics valued highly are now not as important anymore or vice versa. Finally the game can evolve into a completely different direction the developers expected it to. Players often find ways to play the game, the developers didn’t think of. So the more complex you make the algorithm the more likely it ends up being wrong and having to be adjusted constantly.