Boy those Feedburner stats move! A few weeks back I was on the Podcast Brothers and the topic of Feedburner stats came up. We discussed the fact that I had seen an overnight reduction of 80% in the “supposed” subscriber count on my blog. Apparently a few listeners were curious, so here is the screenshot of the “Feed Stats” for MWGblog. As I posted back in October, Feedburner was reporting a subscriber count north of 5000 for this humble little blog.
That number continued to increase to 6200 and then in early January, bam… overnight it went to 1100. Now frankly, I never trusted the 6000+ number. It looked great, but something in my gut just said that was way too high. Conversely, I now wonder if I should trust the new number. It has continued to increase to just over 2200 as I write this. This brings up a few interesting points. First I’d be ecstatic if the number was 20, but I would like to have a number that was at least in the realm of “accurate”. (Lets say at least in the ballpark.) I realize that feed subscriber count is not a science, but when you see swings this dramatic it does nothing but undermine all credibility. When I do see those Feedburner buttons on other sites with subscriber counts I now tend to think “yeah, right…”
That is the problem – loss of credibility. Now don’t get me wrong, I’m not down on Feedburner, heck it is free. Since using their service, other than the stats hiccup mentioned here, it has been fantastic. Also I did email them, and while no specific cause was given they did point to this post abut Rojo. But still, 80%?
The reason this is important to me is that we are starting to pay attention to statistics for podcasts at GigaVox. In that context, my experience offers some lessons I’d like to apply moving forward. Accurate stats are a requirement. Podcasters and advertisers need them: clear, accurate and reliable stats. I have never been a fan of the CPM approach, but “I don’t know” will never cut it as an answer about your audience size. Also, podcasters will need to be careful. Could you imagine relying on a third party stats provider for a CPM based ad campaign and you get back a graph like the one above? Hope you didn’t already cash the check, someone will want some money back. (I also would not expect any return business.) Everyone ends up loosing in that scenario. So no answers here today, but I must confess, in a round about way I am glad this happened. It has given me some good food for thought moving forward on stats for podcasting.
Mike,
This is always a sore spot for the podcast for profit community. Adam Curry/Ron Bloom at Podshow.com eschewed the CPM model for ‘brand’ advertising. I’m not a business owner, never taken marketing, so ‘brand’ advertising is not a term I’m familiar with at all. But for now they seemed to be keeping the Splenda and Folger’s ads running. And the obligatory Go Daddy promo appears on a lot of Podshow properties. So maybe ‘brand’ advertising is one possible route around the CPM convention used for most advertising. Who knows?
One follow-up to the readership of your blog. I know that TechCrunch follows the uptake of the web-based RSS readers pretty closely. Bloglines was tops, now Google RSS reader seems to have overtaken that spot. Right now in Bloglines using this path to your feed:
http://mwgblog.com/feed
Bloglines reports 9 subscriptions. Like you say, there’s no sure bet on how many readers there are. And Bloglines may not be the best ‘scaling’ factor to apply to FeedBurner’s stats. So the best estimate (turning on the irony filter here) is you have somehwere between 9 and 2206 subscribers to MWBlog (turn off irony filter). It’s unfortunate, especially as you pointed out when you’re talking to advertisers. Still no easy answers after almost 3 years of podcasting.
Michael –
I’m at a conference today so apologies for a quick response. But after a quick look at your stats, it appears that the change is a direct result of the number being reported to us by one of the web-based aggregators. I don’t want to reveal your private stats info in a comment, but would be happy to (if you want me to) or we can chat by e-mail. If you’re interested in seeing what I’m seeing, compare the numbers on your subscriber number for one day vs. ‘all time’ and look at the gaps in the user agents. Now – this may be a mistake by that aggregator – in which case we can certainly contact them to find out why their reported number changed dramatically. For more on how we work with web-based services to determine subscriber numbers, see this post:
http://blogs.feedburner.com/feedburner/archives/2007/01/whats_up_with_that_vol_1.php
On the advertising side: I don’t really think subscriber numbers matter much in advertising. Since the subscriber number is an indication of how many people have subscribed – but not how many have actually *consumed* content – it’s not a good proxy when talking to advertisers. Downloads, item views, clicks – all would be much better barometers for how large your audience is.
I’d love to continue this conversation – I think it’s an important one and it’s certainly something we think a lot about at FeedBurner. Thanks for raising the point.
–Rick Klau
VP, Publisher Services
rickk@feedburner.com
AIM/Y!/Skype: RickKlau
312.756.0022 x2012
Michael…
I have always looked @ RSS stats to be an estimation of usage for podcasts, but not a good measure of audience to report on. The actual usage will be affected by syndication, plays through the web, etc. Even saying the number was accurate, the actual plays/downloads can be skewed from something as simple as the default iTunes settings only downloading the newest show; if their iTunes has not been open for x days, it may skip shows.
I still believe that analytics based on, at a minimum, downloads or, better yet, actual usage are a better measure for reporting.
Several folks left comments above that indicate subscriber counts are not something advertisers will be interested in.
If that were true, there’d be little chance that the Audit Bureau of Circulation would be in business. ABC measures magazine distribution by auditing subscriber lists and measuring rack sales. On this basis alone, ad agencies buy millions of dollars worth of magazine ads each month. There is no stat offered to prove each magazine reader saw or read each ad, so why would podcasters want to allow that higher standard to be applied to them?
In my opinion, that’s a big mistake, and if that mistake is made often enough, it will seriously devalue our audience.
I realize the discussion is branching out a bit here, but I believe any discussion that involves podcasters allowing themselves to be held to higher standards than broadcast or print is a mistake.
Scott –
The ABC is measuring something fundamentally different, and is constrained by its medium (print) versus what we as an industry can look at online. I can’t speak to ABC’s methodology in any detail, but from what you lay out above, you’re describing a sample poll, coupled with numbers provided by the publishers themsleves, and some educated guesses made about pass-along readership.
Those inferences are required because there’s no way to actually measure how many times a magazine was opened.
Compare to podcasts (and, to broaden the discussion a bit, feeds). I wrote a lengthy post over at our blog a couple weeks ago about audience engagement:
http://blogs.feedburner.com/feedburner/archives/2007/02/feedburners_view_of_the_feed_m.php
Bottom line, we *can* measure resyndication, downloads, views, clicks, etc. Focusing on subscriber numbers alone – especially when those numbers may reflect inactive audience members who are not in fact reading/listening – without taking into account other metrics (not guesses, but metrics) seems to me to miss the larger picture. I think advertisers are acutely interested in this when looking at podcasts, and we’d like to think we’re helping podcasters out by getting a complete picture of their audience.
–Rick
I look at podcast stats daily and I can tell you one thing most podcasters have no idea what their real numbers are. We have been working hard on stats for a long time and are beta testing and verfying data in a new stats engine we will be releasing to those that are part of our family of podcast communities.
When I process 600 plus shows for advertising deals I have to know within a couple of percentage points what the numbers are for those 600 shows. Luckily I have been doing this long enough now and with the auxiliary survey and repeat listener data that we track we can tell where a podcasters audience is headed.
Thus when I present sponsorships and I tell the advertisers that we are going to be able to get them 2.5 million listner exposures that it really is 2.5 million listener exposures. The total downloads may be in excess of that but by combining data from multiple sources you are able to build a picture that is very accurate of what the active audience size is for a show.
This is one of the reasons out advertisers continue to expand and renew contracts month after month is because we are delivering exceptional ROI.
I know Michael hates the CPM model but most media buyers are still living in that world and this why we have to convert them one at a time. Michaels model of flat sponsorships is great but it is one that is not easily replicated across hundreeds of shows.
Podcasters just need to realize that the RSS Feedburner number is not what your going to get paid on in any advertising campaign nor should it be used as a indication of what ones audience size is.
Audience participation is another metric that factors into the advertising marketability, and while we have been incorporating that data we have a pretty good picture before a single ad is run on what performance a specific show is going to have and are able to assign a performance value to the podcast.