Who is the Biggest UFC PPV Draw? – Conclusion
September 28, 2010
Welcome to the final part of our series on the biggest draws in the UFC. We’ll wrap up the series by exploring several of the issues involved in PPV buys and estimation that make the process of determining a fighter’s draw difficult.
I tried to break things down into coherent sections, but I suspect this will still seem like a laundry list of problems instead of a single piece. We’ll start with some issues in selecting our sample. Then, we’ll move on to some of the important factors that have been overlooked in our analysis so far. We’ll wrap up with some methodological issues, including the use of regression analysis.
SAMPLE SELECTION ISSUES
Selection of Fighters. Probably the most common complaint was the list of ten fighters used in the analysis, with Tito Ortiz and Frank Mir being the missing fighters brought up most often. I intentionally avoided the issue by outsourcing the blame to Derek Jenkins for his great piece at Yahoo! Sports. SBNation member truck came over when Bloody Elbow picked up part 5 of the series and gave a nice illustration about why Mir should be on the list. Here’s part of my response to him from the comments:
This is the “art” of estimation, as opposed to the “science” part with the numbers. The selection of fighters to include is, to some extent, always arbitrary. Even if I give you a set of criteria that we all agree is reasonable, that set of criteria is still arbitrary. In all honesty, I basically decided to outsource the blame for that arbitrariness to Derek Jenkins. His piece, along with some of Jonathan Snowden’s writings about Silva being a terrible draw, were the foundation of this series. Also, one of the responses to the first piece, maybe on the Sherdog forums, was that I could use the same methodology and make Jason Brilz look amazing; I was planning on using someone in the middle, like a Marcus Davis, in the final piece.
Marcus Davis has had a ton of fights in the UFC, so I’ll go ahead and use Jason Brilz as the example. Using the same methods that were in part five, I’ll give you the results for Brilz.
Average buys for all UFC PPVs since 2008 that do not feature ANY of the ten fighters: 308,000
|Fighter||Average PPV buys for cards featuring fighter||Difference from “baseline” average|
Brilz was on three PPV cards – UFC 96, 103, and 114. None of those cards are included in the baseline and all happened 2008-present, making the criteria consistent with what we used in part five. Looking back at those tables, this would put Brilz ahead of Randy Couture (265K), Chuck Liddell (242K), and Anderson Silva (222K). Now, if I tried to argue that Jason Brilz is the 7th biggest draw in the UFC, you should laugh at me… and rightly so. I love watching him fight, but any ranking that lists Brilz as a top draw for the UFC has some serious problems.
In part five, I provided some additional justification for the list of fighters. Since 2006, only one UFC PPV has not featured at least one of the ten fighters on the list, Rich Franklin, or Matt Hughes. There’s certainly room to include a few other fighters in the analysis, but there are very few fighters that Zuffa allows to consistently headline cards. Frank Mir and Tito Ortiz are certainly on that list, though it will be interesting to see if that continues in the future.
Selection of Cards. There are two big issues with selecting cards to include in the analysis, but only one seemed to get much attention. The first issue is whether or not to include international pay-per-views. These are generally regarded as weaker cards and air live in the afternoon and on tape-delay at night. There have also been rumors for a long time that fighters who receive a PPV% try to avoid fighting on international cards since they perform worse than domestic cards; but, the fighters with a PPV% are usually considered the bigger draws. I kept all the cards in because I typically see overall averages reported in the MMA media, and these include international cards. Basically, it was a decision to keep some consistency with the numbers that you typically see reported elsewhere.
Also, just going through your data It seems like you have some bad data. You list Ortiz/Machida as the main for UFC 84. It’s not. It’s wasn’t even 2nd billing match. That was Silva/Jardine. So Machida gets credited for that buyrate even though he wasn’t a) a draw at that point and b) in the top two matches.
Emphasis added. This is an excellent point, and I’m surprised it didn’t get more attention. Perhaps a bit surprising, however, is that Machida is the only one on the list for whom this is a major issue. St. Pierre, Penn, Couture, and Liddell were already established fighters at the top of UFC cards by 2006. An argument can be made that this problem applies to Griffin and Evans for the 2006-present numbers, but they both came with a significant fanbase from The Ultimate Fighter; however, it’s hard to argue that either fighter was not a headliner since 2008. Lesnar and Jackson came in as headliners, even if they were not always in the main event. Silva won the middleweight title in his first PPV appearance and second UFC fight, coming off a destruction of Chris Leben at UFN 5 on Spike.
That leaves Machida. Here are the cards he’s fought on:
- UFC 67 – Unaired prelim against Sam Hoger
- UFC 76 – 2nd fight on main card vs Nakamura
- UFC 79 – 2nd fight on main card vs Sokoudjou
- UFC 84 – 2nd fight on main card vs Ortiz
- UFC 94 – Co-main vs Thiago Silva
- UFC 98 – Main vs Rashad Evans for title
- UFC 104 – Main vs Mauricio “Shogun” Rua I
- UFC 113 – Main vs Mauricio “Shogun” Rua II
One can argue that Machida shouldn’t get credit for PPVs where he was on the unaired prelims or low on the main card (i.e. before he became popular). That basically means his fights from UFC 94-present, since he wasn’t considered a big name when he fought Ortiz (which is part of why Zuffa made the match, as it was known that Ortiz was going to test the waters as a free agent). Re-running the numbers using only UFCs 94, 98, 104, and 113 bumps him from 572,000 average buys to 644,000 buys. However, UFC 94 was headlined by GSP-Penn II and drew high above any other Machida event. If we take out UFC 94, we are left with three cards headlined by Machida with an average buyrate of 552,000. That drops him 20,000 buys lower than our previous estimate for 2006-present in part 5, so our original estimate was still pretty good. The 2008-present number in part 5 was 610,000, which is 34,000 lower than the 644,000 just mentioned, but the number in part 5 includes the fight against Ortiz, which was on a card with 475,000 buys.
IMPORTANT FACTORS THAT WERE IGNORED
Promotion. I have no doubt that the promotional push by Zuffa has a significant impact on PPV buyrates. The first season of The Ultimate Fighter is a great example of Zuffa’s promotion at its best. The season was used to reach new viewers on Spike TV and build a fight between coaches Randy Couture and Chuck Liddell. The finale was broadcast live on Spike about ten days before the pay-per-view with Couture-Liddell II. For a while, Zuffa seemed to use Ultimate Fight Night events live on Wednesdays to build interest in that weekend’s PPV. TUF is still used to build interest in a fight between the coaches at the end of the season. A recent strategy is to air two prelims live on Spike to increase last-minute buys of the PPV. Countdown shows are used before most shows, but Primetime shows are only used for the biggest fights. Primetime shows create a major problem – the shows likely increase the buyrate, thus making the featured fighters bigger draws… but only the biggest draws get a Primetime show in the first place.
All of these factors are hard to account for in any analysis, but it needs to be done if you want an accurate, precise number for a fighter’s draw.
Timing of Events. The timing of both Zuffa and non-Zuffa events matter here. Non-Zuffa events can clutter the schedule and potentially give “too much” free MMA, discouraging PPV buys. Occasionally, a UFC PPV goes up against a big boxing PPV, as MMA Payout reader Brain Smasher pointed out in the comments to part 5 about the Franklin-Belfort fight at UFC 103 going head-to-head with Mayweather-Marquez in Mayweather’s return bout from retirement.
While the non-Zuffa events can cause distortions in PPV buyrates, a bigger issue is the timing of other Zuffa events. We’ve seen situations where UFC PPVs are scheduled about three weeks apart. However, to really control for the timing of the event, we would have to go one step further and distinguish between events that happen on the same cable/satellite billing cycle and which ones are spread over two billing cycles. Having multiple events in the same billing cycle for viewers likely has a bigger impact than if the same situation (number of events and time between events) were spread over two billing cycles.
Strength of Card. Michael Rome raised this issue in the comments to part one. Here’s his comment:
How does this take strength of card into consideration, or does it at all?
Someone like Rashad Evans benefits enormously from being on a card like UFC 92 with 3 main events, whereas Randy Couture headlines against Mark Coleman with no help on the rest of the card.
Similarly, until this past weekend, the UFC made sure to help out Anderson Silva by pairing him with other big superstars to keep him happy with his pay.
Rome’s point is a huge problem with any analysis, and any method for dealing with strength of card is going to be somewhat arbitrary. A special case that is commonly brought up is including UFC 100 in any analysis, since that card was stacked and did an astounding 1,600,000 buys (making it a huge outlier). His example with Couture is an excellent illustration of how a potentially bigger draw looks worse in our analysis, since the bigger name may be expected to carry a card on his own, while some of the other fighters – such as Evans or Silva – often get paired on cards with other notable fighters, which should provide an extra bump in the fighter’s numbers that is not due to the fighter.
When trying to look at strength of card, a more subtle issue arises. Strength of card is not simply about the fighters who appear on the card; it’s also about what fights occur. It’s easy to imagine that a card with St. Pierre and Silva making title defenses will get fewer buys than a card with a St. Pierre – Silva superfight. So, even if you wanted to estimate strength of card with a regression model, you’ll need to find a way of distinguishing between fighters appearing on the same card against other opponents and against each other. I’ll address this issue a bit more in the Regression Models section below.
With the dominant champions in most classes (including BJ Penn until recently), it may be hard to get a number on the title bump. To get at the title bump for a fighter, you would hope to see X buys for a fighter before they’re champion, an increase up to Y buys while champion, and then a drop back down between X and Y buys after the fighter loses the title. LHW seems to be the only weight we could hope to see that, but it’s been plagued by very short title reigns recently which might not provide a good estimate.
The title bump is a complicated issue if you really want to get at a causal effect. Title-holders are regarded as the best fighters in their divisions, so it’s not clear if a title bump is because viewers want to see the best fighters or if it’s because they simply care more about title fights than non-title fights. Also, if you lose a title and your PPV buyrate drops, it’s not clear if it’s because of the loss (which resulted in you no longer holding the title) or because your subsequent fights will often not be title fights. You can try to tease this apart a bit by looking at the next fight and whether it was a title shot (immediate rematch) or not. But, again, there’s a problem here – an immediate rematch is more likely to happen after a close fight or a controversial ending. In those cases, the former champion still has a claim to being “the best” in the division.
My suspicion is that there are (at least) two big elements to the title bump. First, it’s easier to convince casual fans and non-fans to sit down and watch a title fight, since it’s perceived as watching “the best” (which is generally true). Second, the title reign leads credibility to the fighter’s abilities, and I think this has played a big role with Rashad Evans and Forrest Griffin, as the title reign negated any remaining sentiments that they were nothing more than good-not-great fighters who got unwarranted attention for being reality show winners.
Using Mean PPV Buys. This is one of the more interesting criticisms I’ve seen raised, generally with the implication that “people who know what they’re doing” use median instead of mean, since the mean can be skewed by outliers. In practice, you typically want some measure of average (whether it’s mean or median) and a measure of spread. We probably think differently about a fighter who consistently draws 600,000 buys and a fighter who draws 300,000 buys half the time and 900,000 buys the other half of the time. Assuming both fighters have been on an even number of cards, the mean and median for both fighters is 600,000 buys. Typical measures of spread are variance or standard deviation (the latter being the square root of the former) and confidence intervals (which are computed using standard deviation). In our hypothetical example here, the first fighter has zero variance in his buyrate, but the second fighter’s variance is positive.
Also, it’s worth pointing out that median can be just as misleading as mean. Suppose our second fighter has been on nine cards. Four of his first eight fights drew 300,000 buys, and the other four drew 900,000 buys. So, we see a pattern that he either gets 300,000 or 900,000 buys. If his 9th fight gets 900,000 buys, his median draw is 900,000 buys. If his 9th fight instead gets 300,000 buys, his median draw is 300,000 buys. If we use mean, his draw is either 633,000 or 567,000 buys, respectively. In this case, mean is much more informative than median.
In all honesty, I used mean PPV buys as the measure because that’s what I see reported in the MMA media in discussions about fighter draws. I could have included standard deviation or 95% confidence intervals, but they’re not very helpful here. The standard deviations here are pretty damn big. Using the Jason Brilz example from above, here’s what you’d see if I reported the output from statistical software (and decided not to round my numbers off to the nearest thousand):
Average PPV buys for cards with Jason Brilz – 591,666.7
Standard error – 229,280.3
95% confidence interval – [-394,846.7 , 1,578,180]
Number of observations – 3 (UFC 96, 103, and 114)
For practical purposes, all most people care about from these numbers is that (i) his average is about 592,000 buys and (ii) his buyrate fluctuates a lot (which you can see with the huge standard error), meaning he’s not consistently drawing 592K+ buys.
Regression Models. This part will be a bit more technical than the other sections, but I’ll try to keep it fairly accessible. If you have any questions, feel free to leave a comment, and I’ll try to get a response to you fairly quickly.
Regression models are a great tool, and many have brought them up as a way of controlling for a lot of the issues we’re dealing with. One reason I would tend to avoid using regression is that I suspect many readers have no experience using them, though I think Kelsey has provided a very intuitive description of regression analysis. Once we make the decision to run some regressions, we start running into problems. I described some of them in the comments to part two in response to reader Eric Nitsch:
The statistics nerd in me says you should toss these things into a regression with a bunch of dummy variables, but that may not even be possible with all the interaction terms you would want to include (since they end up being the numbers you’re really interested in) because of the relatively small number of UFC PPVs (which gives our sample size). You lose a degree of freedom for each additional thing you estimate. For instance, say you want to see the draw for Lesnar, the draw for St. Pierre, and then you want to see what they pull together (since they’ve been on two cards together). OK, run a regression with a dummy for Lesnar, a dummy for GSP, and a dummy for having both on the card. When you try to do that for all the possible combinations that we’ve seen (Penn & Silva, Couture & Liddell, etc), you start losing degrees of freedom rapidly. We can easily come up with a list of important factors driving PPV buys that’s longer than the number of UFC PPVs we have… but tossing all of those into a regression isn’t possible because you need more data points (i.e. UFC PPVs) than parameters that you are trying to estimate (eg. Lesnar, GSP, and Lesnar-GSP draws).
The main thing is that you need to be very careful in interpreting the output from any regression, as it may not be telling you what you think it’s telling you. When you run a regression using only dummy variables, you may be tempted to interpret the results as providing means. Sometimes, this is true. Here’s a simple example. Using the data from 2006-present, I can run,
PPV Buys = B0 + B1*Lesnar
Where Lesnar = 1 if Lesnar fought on the PPV and Lesnar = 0 if he did not fight on the card. You get
B0 = 498,300 (33,087.99)
B1 = 508,700 (109,740.5)
The numbers in parentheses are the standard errors, in case you’re interested. For a card without Lesnar, the regression tells us we should expect B0 + B1*0 = B0 = 498,300 buys. For a card with Lesnar, we should expect B0 + B1*1 = B0 + B1 = 498,300 + 508,700 = 1,007,000 buys. If you go back to part three and look in the first table, these are exactly the numbers you see after rounding to the nearest thousand. Recall that in part three, we were comparing average of cards without a fighter to cards with the fighter. In that analysis, we just looked at means. So the regression here is just giving us the means we had before.
The issue with interpreting the regression coefficients is that the interpretation is based on “holding other things fixed.” Seeing how things worked out nicely above, you might be tempted to run a similar regression with a dummy for each of the ten fighters. You would run,
PPV Buys = B0 + B1*Lesnar + B2*GSP + B3*Liddell + B4*Evans + B5*Jackson + B6*Griffin
+ B7*Machida + B8*Penn + B9*Silva + B10*Couture
and expect it to give you the same results we saw in part five. It won’t. I’ll paste the regression output for 2008-present at the bottom for those who are interested. It looks nasty since I don’t feel the need to reformat it, but the results should be clear if you have any experience with regressions.
We’ve finally hit the end of the series. We started in parts 1 & 2 with the numbers that you typically see in the MMA media. In part 5, we finally got to a reasonable baseline of 200-250,000 buys for a UFC PPV without a big draw and were able to get an estimate of how many PPV buys each of the ten fighters on our list bring beyond this baseline. In our final piece, I’ve tried to go over some of the many problems encountered if you want more precise estimates.
As for me, a new semester is beginning, which will decrease the time I can devote to these pieces. I’ll still be around to respond to comments and have a few ideas for future pieces that I’ll be working on in whatever spare time I can find. Thanks for reading!
Results from regression on the following model:
PPV Buys = B0 + B1*Lesnar + B2*GSP + B3*Liddell + B4*Evans + B5*Jackson + B6*Griffin
+ B7*Machida + B8*Penn + B9*Silva + B10*Couture
Source | SS df MS Number of obs = 34
————-+—————————— F( 10, 23) = 4.33
Model | 2.1121e+12 10 2.1121e+11 Prob > F = 0.0018
Residual | 1.1221e+12 23 4.8789e+10 R-squared = 0.6530
————-+—————————— Adj R-squared = 0.5022
Total | 3.2343e+12 33 9.8008e+10 Root MSE = 2.2e+05
ppv_buys | Coef. Std. Err. t P>|t| [95% Conf. Interval]
liddell | 167049.1 147391 1.13 0.269 -137852.3 471950.6
evans | 213727.2 123626.7 1.73 0.097 -42014.22 469468.5
rampage | 283804.5 148840.9 1.91 0.069 -24096.38 591705.4
forrest | 157770 136196.1 1.16 0.259 -123973 439513.1
machida | 176701.5 121627.1 1.45 0.160 -74903.21 428306.3
penn | 152011.3 113508.3 1.34 0.194 -82798.43 386821.1
a_silva | 138784.7 123387.2 1.12 0.272 -116461.1 394030.6
couture | 110967.6 145147.2 0.76 0.452 -189292.2 411227.5
brock | 590756.4 122131.5 4.84 0.000 338108.1 843404.7
gsp | 321508 119255.3 2.70 0.013 74809.52 568206.4
_cons | 265446.9 73573.9 3.61 0.001 113247.7 417646.1