4/6/09
One discussion that comes up a lot in fantasy football circles is
how much of the outcome is due to skill and how much is simply luck.
The answer seems to depend on how long it has been since you won
a league. In this article, we are going to look into the question
of whether fantasy football is predominantly skill or luck.
Many countries have laws restricting gambling. These laws generally
make exceptions for games where the player's skill either partially
or mostly determines the winner. Poker, for example, has been
the subject of much debate about how much skill is involved. Several
researchers have attacked the problem to try to provide an answer
for the judges, attorneys, and politicians that are involved in
the legal aspect of gambling. One method was developed by Marcel
Dreef, Peter Borm, and Ben van der Genugten. They are part of
the economics department at Tilberg University in The Netherlands.
We will examine their method and try to apply it to fantasy football.
First, we will define the concepts and terms used, and then look
at the equation suggested by Dreef, Borm, and van der Genugten.
Finally, we will see what this says about league rules.
Skill is a measure of the outcome that is determined by a player's
ability, experience, and aptitude. Luck is the part of the outcome
determined by things outside of the player's control. We will
express the total result in terms of what percent is due to skill
and what percent is due to luck. For example, if a result is 30%
skill, then it is 70% luck. Luck can be split into two different
types. The first type is luck due to events controlled by other
people. This generally has to do with the other owners in the
league. For example, even if you are skillful enough to know who
the highest scoring running back in the league is going to be,
you still might not be able to get him if he is drafted before
your first pick. The second type is luck due to random events.
Examples of this are injuries and suspensions that prevent a player
from performing.
Result or outcome in fantasy football could be either winning
percentage or winning the championship. We will look at both to
see if there is a difference. An advanced player is someone who
has a combination of ability, experience, knowledge of the game,
and natural aptitude. A novice is someone who knows the rules
of the game, but lacks the other qualities of an advanced player.
For example, the new player who picks a kicker in the sixth round
and doesn't stock up on running backs early in the draft would
be a good example of a novice player. An all-knowing player knows
exactly how many fantasy points every player will score in every
game before the season starts. This type of player does not exist
in reality, but is used to remove the element of random luck from
the game.
The method developed by Dreef, Borm, and van der Genugten says
that the results of a game are due to a combination of a learning
effect and a random effect. The learning effect
has to do with the parts of a game that the player can control.
It is an attempt to gauge how much a player’s ability may
be improved by experience or study. In this method, the learning
effect is the difference in outcome between an advanced player
and a novice player. The random effect has to do with the chance
elements in a game. For example, the offensive linemen of the
Jacksonville Jaguars suffered more injuries than normal last year.
This brought down the production of their skill players. Since
this could not have been known before the season started, it is
part of the random effect. In the method we are using, the random
effect is the difference in outcome between an all-knowing player
and an advanced player. This is because luck does not affect how
often the all-knowing player is successful since they already
know the outcome of all the events that are random for other players.
Thus, the difference between an all-knowing player and an advanced
player gives us a measure of how often random events dictate the
outcome of a game.
The level of skill in a game is what percent of the total outcome
is determined by the player. Therefore, skill is the ratio of
the learning effect to the learning effect plus the random effect.
The equation for skill is:
Skill = |
Learning Effect
Learning
Effect + Random Effect |
Substituting for the learning effect and random effect, we get:
Skill = |
(Advanced Player Result – Novice Result)
(All-Knowing Player Result – Advanced Player Result)
+ (Advanced Player Result – Novice Result) |
We can simplify the denominator by cancelling out the two “Advanced
Player Result” terms:
Skill = |
(Advanced Player Result – Novice Result)
(All-Knowing Player Result –Novice Result) |
For our purposes, we are going to focus on leagues with twelve
teams. We will make an upper end, lower end, and average estimate
for each of the three player types. This will give us a range
of values for the amount of skill involved. Let's start by considering
winning percentage. We are trying to estimate the winning percentage
of each player type over a large number of seasons, so the number
of games in a particular season is not important.
Every week there seems to be a player that comes out of nowhere
and has a huge game. Call it the Mark
Campbell effect. If you don't believe how often this happens,
take a look at any of Mike MacGregor's “FF
in the Groin” articles from 2004 and 2005. Combine this
with perfect knowledge during the draft, and the all-knowing player
should win 95% of their games. For high and low estimates, we
will use 100% and 90%.
We will assume that there are eight typical players, two advanced
players, and two novices in a typical league. If the average players
win half their games, then the combined win percentage for the
advanced and novice players must be 100%. Furthermore, let’s
estimate that an advanced player will win ten out of fourteen
games. This would give a 71% win rate. From this, we could assume
75% for the high end, 70% as a typical value, and 65% at the low
end for an advanced player. This gives corresponding values of
25%, 30%, and 35% for the novice player, respectively.
To double-check our assumptions for an advanced player, we will
look at the results from one of the three leagues I regularly
play in. Every owner in this league has at least five years of
experience. It is quite competitive and there are no easy wins
against “dead” teams late in the year. I added up
the wins and losses for the thirteen owners (one owner quit and
was replaced) over the last three years. Then I found each owner's
winning percentage. Here are the top five:
Rank |
Win Percentage |
1 |
78.60% |
2 |
68.30% |
3 |
64.30% |
4 |
58.50% |
5 |
54.80% |
|
There is a relatively large drop from third place to fourth,
so we will call the top three players advanced. They have a combined
win percentage of 68%. This is close to our assumed middle value
of 70%, so we have some confidence that our assumption for an
advanced player is good. Unfortunately, there are no players that
would be classified as novice in the league, so we don't have
an easy check for those numbers.
Using the equation for skill shown above, let's take a look at
the results so far:
- High End: Skill = (75 –
25) / (90 – 25) = 0.77 (80%
skill)
- Middle Value: Skill = (70 –
30) / (95 – 30) = 0.62 (60%
skill)
- Low End: Skill = (65 –
35) / (100 – 35) = 0.46 (50%
skill)
If our estimates are close to accurate, then the long-term win percentage
in fantasy football is about 60% skill and 40% luck. I have rounded
these off to the nearest 10% because of the inherent uncertainty
of our process. Let's see how this compares to other games.
Game |
Percent Skill |
Roulette and Craps |
0% |
Blackjack |
5% |
Poker |
40% |
Chess and Checkers |
100% |
|
Thus, long-term success in fantasy football is based a little more
on skill than card games such as poker but substantially less than
games such as chess.
Now let's re-run these numbers using winning the championship
game as our measure of success and see if the results are similar
or different. The all-knowing player now has to field a team that
is just good enough to make the playoffs and then win two or three
games in a row, depending on how long the league's playoffs are.
Using the same logic as before, it seems to be an easy task to
obtain players through the draft or waiver wire that will have
great games in the last three weeks and good enough games during
the regular season to make the playoffs. We will use a high and
middle estimate of 100% success and a low estimate of 95%.
We will make the same assumptions as before regarding the number
of advanced, typical, and novice players. Furthermore, we will
assume the typical player will win one title every twelve years.
For the advanced player, we will assume a title every six, eight,
and ten seasons for the high, middle, and low end estimates. The
corresponding results for a novice player are no titles, one every
24 seasons, and one every 15 seasons. These estimates give the
following skill levels:
- High End: Skill = (16.67 –
0) / (95 – 0) = 0.18 (20% skill)
- Middle Value: Skill = (12.5
– 4.167) / (100 – 4.167) = 0.09 (10%
skill)
- Low End: Skill = (10 –
6.67) / (100 – 6.67) = 0.04 (0%
skill)
Thus, winning a championship is about 10% skill and 90% luck. This
is quite a difference from the amount of skill in overall winning
percentage! But is this result really surprising? In one of my leagues
last year, the top scoring team averaged 92.8 points per game, an
average team scored 77.8 points per game, and the worst team 71.5
points per game. That's only a 16% difference between the best team
and an average team and only an 8% difference between an average
team and the worst team. Since the average number of points scored
is so similar, each individual game is close to a coin toss. It
is only over a large number of games that the difference becomes
significant. We can relate the playoffs in fantasy football to poker.
I'm not a good poker player, but I have a 50/50 chance to beat Phil
Ivey if we play a one hand tournament. Over three hands, I still
have a chance. If we played a thousand hands, I'm broke.
We have found that success over the long term (several seasons)
is mostly skill, but success over the short term (winning a title)
is mostly luck. If we want to reward the most skillful owners,
then how should we set up our fantasy league? First, the regular
season should be as long as possible. In a twelve team league,
this probably means a fourteen week regular season with the top
four teams making the playoffs. Even though winning the title
is still mostly luck, at least we can be reasonably certain that
one of the top teams will win it. If we allow six teams into the
playoffs, then the chance of a mediocre team winning the title
is much higher. If there are payouts, then consider increasing
the prizes for the team with the best regular season record, most
points scored during the season, and making the playoffs and reducing
the money for playoff wins.
If you didn't win last year's championship, now you can confidently
tell the winner that it was just dumb luck when he starts bragging
at this year's draft. If you won last year's title, go ahead and
rub it in their faces until the other owners threaten to drum
you out of the league. If any of them say it was luck, point out
all the skill you showed while kicking their sorry butts en route
to the playoffs.
|