Dabe Alan

Why it’s time to grow up and start ignoring the monthly NPD reports

Why it’s time to grow up and start ignoring the monthly NPD reports

The game and mainstream press has a love affair with the monthly NPD reports. It’s easy to see why: the data is lined up in simple lists, the company’s analysts provide pre-packaged quotes, and the information gives the illusion of authority and allows for strong headlines about how the gaming industry is on the rise or, as it’s being reported now, in decline. The problem is that the NPD reports bear only a passing resemblance to what’s actually going on with the gaming industry, and it’s time to stop pretending that we can make blanket claims about the state of the industry based on NPD information.

Anyone who says they know the state of the industry is a big, dumb liar. This is why.

There is way too much going on

It’s important to remember two things about the monthly NPD reports: they only look at the United States and the focus is squarely on retail sales. “NPD’s monthly point-of-sale data reports on US Games Industry sales occurring from new physical purchases at retail which is the largest channel for games sales… it does not account for consumer purchases made via digital distribution, used game sales, subscriptions, mobiles, rentals, or social network games,” the disclaimer included with the data discloses. There is a quarterly report that offers some estimates at the size of those other markets, but its methodology is just as vague.

“The estimates are published in NPD’s report, Games Market Dynamics: U.S. (formerly known as the Games Industry:  Total Consumer Spend) –released by The NPD Group in June 2012 - and are derived leveraging NPD’s portfolio of physical POS tracking (Retail Tracking Service) and consumer research including the Games Acquisition Monitor, Video Game and PC Game Subscriptions Report and Consumer Tracking Service. It also incorporates consumer spend estimation provided by NPD retail and publisher partners, and calibration with third-party sources,” the press release states.

“Each month, we issue one total spend number to the media (always provided via an analyst quote with the monthly retail sales data deliverable),” NPD analyst David Riley told me when I asked about online sales. “This sales figure represents an estimate of spend on all content across all channels and on all platforms.” When I asked about whether they had any way of looking at Steam sales numbers, he directed me to the statement above.

The monthly NPD reports neglect how an increasing number of people buy and play games. The NPD Group tracks a tiny part of an increasingly digital world, and that problem is only going to get worse; releasing sales charts without looking at digital distribution or mobile games is like trying to chart the success of McDonald’s without being able to count sales of fries. EA’s Frank Gibeau has stated the company doesn’t even look at the NPD numbers anymore.

“It’s totally irrelevant,” he told Gamesindustry. “We don’t even really look at it internally anymore. We’re more focused on our services and how we’re connected with consumers. The number of Nucleus accounts we’re growing, the amount of engagement time that we have, the amount of services that we’re running - those are more important metrics for us than unit sales according to NPD and North America.”

David McQuillan, President of NPD Games, stands behind the company’s focus on retail sales. “While digital is a growing part of the industry and something that needs to be addressed for the future, the current games industry is still largely rooted in retail and any industry player involved with AAA content simply can’t take their eye away from the retail environment. Successful companies are looking at how their products are performing within all channels, particularly retail,” he stated.

You don’t have to dig far to find how flawed this statement is, and why anyone placing faith in NPD data as a measure of the industry is misguided. The NPD October data stated that XCOM: Enemy Unknown sold 114,000 units, and there have already been articles calling the game a disappointment. That claim is based on retail sales in one market, mind you.

The reality is that XCOM had over 70,000 players concurrently playing the game on Steam near launch. There’s no hard rule, but it’s estimated that concurrent players can range from 5 to 25 percent of total players, which means that XCOM may have sold 280,000 copies on Steam at the low end.  2K Games isn’t likely to share specific information with the press; sales numbers are almost always considered proprietary. Have you ever noticed how many companies describe sales as “shipped” versus “sold” when milestones are released?

The power of online sales

There is no way to know the performance of games like XCOM after you factor in online sales without 2K Games telling us directly, which makes any guesses about how well the game did or not did not perform so many darts thrown in the dark. We like to write about how XCOM didn’t show up in the charts and what that means, but the reality is the game didn’t show up on a chart one specific company put together based on incredibly limited data. It says nothing about real-world sales or performances of the title.

It’s time we stopped pretending it did.

Players buying digital games, add-ons, and paying into free-to-pay games are all but invisible to the NPD Group, and the existence of this market is rarely factored in when armchair analysts talk about the performance of a game based only on re-writing NPD’s press releases. The video game industry has become both huge and fragmented and trying to estimate how much money players are spending on games, even in just one territory, has become an impossible task.

Steam is NPD’s largest blind spot, especially considering the size of the virtual store front and the number of people who buy games exclusively through Valve’s service. Estimates say that Steam is around 70 percent of the digital distribution market in total, and detailed sales information on Steam games is rarely shared with the press, much less analysts. In fact, Valve controls the flow of information by dictating what developers can and can’t say about the performance of their own games.

One developer I spoke with stated they couldn’t share sales information about their game, due to Valve’s sensitivity about sales numbers being released. That’s why, they told me, the NPD Group doesn’t include digital sales. He described the monthly NPD report as being increasingly useless.

“Generally speaking, there’s an NDA clause in Steam contracts that says that confidential information like sales data can not be shared unless both parties agree to share it. So if the developer or publishers ask Valve and they say yes, they’re free to share sales data,” another developer who spoke under the condition of anonymity told the Penny Arcade Report.

Valve’s secrecy about the size of Steam and the sales of the games within the service hamper anyone’s ability to make accurate claims about the size of the video game industry, and specifically the PC games market. “Valve is happy to let developers share sales milestones (Shooter 1378 has sold over half a million copies on Steam so far!) and relative data (revenue for Platformer 438 increase 1000% during its 50% off sale compared to the previous sales period) but they don’t want developers to share data that would reveal too much about the service itself like concrete sales figures during a limited time Steam promoted sale,” the developer continued. Without taking into account sales of games via Steam the monthly top ten list from the NPD Group is hard to take seriously as an accurate picture of sales performance of any title.

Valve has no reason to share data with for-profit third parties like the NPD Group. Valve hoards concrete sales information jealously, which is why the company is so fond of speaking in general percentages. It’s easy to say sales are up 200 percent, for instance, but the actual numbers behind that sort of percentage are rarely shared. It’s rare to hear developers critical of the situation: who cares about Valve being private with its business when the company drives sales for so many games? The fact remains that Valve does not share specific sales information, and Steam counts for an incredible amount of consumer spend.

(Just a quick note: The relative percentage quote or vague milestone is a common trick when it comes to hiding sales data. Sony may love to say that Journey is the “fastest selling” PSN release in history, but notice how the company refused to explain what that statement means? Giving specific information would reveal too much about Sony’s digital business, so real numbers will likely never be shared. Next time someone brags about how much sales have gone up, keep in mind that doubling your sales isn’t that impressive if you go from two to four copies.)

What about smaller digital storefronts? Does GoG.com share any sales information with NPD? “GOG.com does not share its sales data with anybody but the 100 publishers & developers we work with, and we do not intend to change this approach,” a company representative told the Penny Arcade Report. “We prefer spending our time focusing on selling and promoting our PC & MAC games and making gamers happy, rather than being obsessed with analyzing the sales performance of our competitors or spending time dropping sales reports and explanations to 3rd parties such as NPD, GFK and the like. Our time is precious and we basically prefer to dedicate our efforts to please our gamers and our partners.”

That means that the company is going down the tubes, right? The NPD reports say that sales are down in general, and things are looking dire. “Our year-over-year sales are continuing to grow healthily,” we were told. “Many reports published in the last few years claim that computer games sales are going down, but we actually see the very opposite from our end, and we expect that we will continue to succeed at making gamers and publishers / developers happy as we continue to grow.”

These success stories can’t be accurately counted in the monthly NPD reports, and the world of PC gaming is filled with similar games and companies. Riot Games has a massive hit with League of Legends, and that’s a game with a rabid worldwide fanbase that pays for content through the game client itself.

Tencent Holdings, a Chinese company with a variety of investments in gaming including a minority ownership of Epic Games, owns a majority share of Riot Games. Unless Tencent breaks out gaming revenue by market in its earnings reports, the NPD Group has no idea how much money American players are spending on League of Legends. How does League of Legends revenue compare to the games on the top ten sales charts released by the NPD Group? We have no idea.

Trying to paint any kind of picture about the health of video games using NPD data is a fool’s errand. Does anyone think that Angry Birds Star Wars wasn’t a huge hit, with a large number of in-app purchases of extra content? That money is either not tracked, or guessed at, by the NPD Group. Some groups claim that iOS may be the most popular gaming platform of all time, and Apple isn’t sharing much data with analysts when it comes to how much money people are spending on the service. How much money are players dumping into iOS games, and how would an accurate picture of that money affect the outlook on the US gaming industry? We have no idea.

The monthly reports also only deal with consumer spend, and ignore revenue like the ad sales on Xbox Live or successful marketing services like TapJoy. You don’t have to take money directly from gamers to draw a profit, and these aspects of the gaming business are also not considered when we make sweeping statements about how bad things are. The reality is we just don’t know.

What the NPD does

The NPD reports give us data on one thing: retail sales of video games in the United States. In the modern video game business that’s much like claiming you understand the sales of candy bars because you spent time at a convenience store down the street for an hour or two. The sales numbers are an incomplete picture of a huge business, and the coverage given to this data gives the numbers a sense of weight and authority they don’t deserve.

The NPD Group exists to sell detailed analysis and reports to business clients, and the monthly reports shared to the press free of charge, complete with quotes and easy to replicate lists, are marketing. They exist to make it seem as if the NPD Group is the authority on video game sales, an illusion the press is willing to help create, when the reality is that the true size of the industry in any market is damn near unknowable.

There are too many storefronts, too many self-contained free to play games, too many walled gardens for anyone to collect enough detailed information to make an educated call about whether video games are doing well or poorly. Anyone who tries to tell you differently is trying to sell you something. The information we gain from the NPD reports makes for interesting conversation, but that’s where the importance ends. There is no authority on the general state of the game industry in 2012.

It’s time to stop breathlessly writing up the monthly NPD reports as if we’re learning anything of worth about how much people spend on gaming. It’s time to stop pretending this data allows us to forecast the health of gaming. It’s time for the gaming press to grow up and realize we can’t understand the intricacies of an industry this large by reading two pages of marketing materials from a single company.