Web Business by Ken Burbary - Latest Comments in The Dirty Little Secret of Social Media Monitoring

Re: The Dirty Little Secret of Social Media Monitoring

Eric Swain — Thu, 04 Mar 2010 06:45:59 -0000

Ken,

Did you ever resolve the issues with the vendors?

Re: The Dirty Little Secret of Social Media Monitoring

Bill Rice — Fri, 12 Feb 2010 13:50:25 -0000

I think the fundamental answer here is that hooking a vacuum cleaner up to the Internet is very difficult (even for Google). Any social media monitoring software is going to have to make choices, which impact categories of coverage. You're likely seeing some of those programmatic choices in your data.

Re: The Dirty Little Secret of Social Media Monitoring

Randall Garcia — Thu, 13 Aug 2009 13:37:13 -0000

Ken,

Interesting article. Looking forward to hearing the updates, since my agency went through the same experience of inconsistent results. We had been using Radian6, but after seeing several presentations, including TNS Cymfony and WoolLabs Web Dig, we settled with the latter. I won't go into why we didn't choose the others (frankly, the differences made my head spin), but we did go with Wool's product because of the sheer depth of data. Real data. It isn't the prettiest GUI, but they said they are working on it. If you haven't heard of them, it is worth checking it out. We started with them for a 3 month trial, and have now signed on for an on-going engagement.

Good luck!

Randy

Re: The Dirty Little Secret of Social Media Monitoring

Ken Burbary — Thu, 23 Jul 2009 19:07:28 -0000

You're right Paul, the best platform right now would be a mix of the best features & coverage of many different platforms. While it's possible to do this now, the barrier for entry is higher. Not all organizations or individuals have the knowledge, resources and time to implement such a solution. However, for DIYers, there is light at the end of the tunnel. Thanks for the comment/visit.

Re: The Dirty Little Secret of Social Media Monitoring

NewMediAroused — Wed, 22 Jul 2009 21:47:20 -0000

Glad to hear R6 andSM2 have contacted you to work through things. I too use both of these tools, in tandem, for the same monitoring on a regular basis. You will find that many of the differences are a necessary by-product of how each service filters for spam and identifies "valid" posts.

More importantly, both services allow you to import RSS feeds, so it is possible to combine the two into a de-duped, aggregate report (with some fussing with it). Personally, I like to have the final product come through Techrigy because they allow me to download the full content of all posts into a database that I can analyze on my own machine.

Plug R6 into SM2, then add in a well developed Yahoo! Pipes feed as well, set your Filters wisely, and you've got the industry's best listening platform....

Paul Dyer
http://www.dyersituations.com

Re: The Dirty Little Secret of Social Media Monitoring

zakig — Mon, 20 Jul 2009 07:11:29 -0000

Great post Ken, equally so the discussion you've sparked here!

Finding the most relevant and interesting "chatter" whilst weeding out the spam will always be different one provider to the next, which solution(s) can provide the information you need in a timely and clean fashion?

There is so much information available to us right now that filtering out the noise and finding the juicy bits is as big a challenge as getting complete coverage.

Zak
www.moreover.com

Re: The Dirty Little Secret of Social Media Monitoring

Ken Burbary — Fri, 17 Jul 2009 16:55:36 -0000

Steve - In principle I agree with the argument that "many people rely on the output of these systems without truly understanding their customer requirements". However, how does a customer get that level of understanding about the system capabilities/shortcomings before selecting one? That information isn't exactly the type you'll find in marketing material.

Trial and error? Research on this is limited at best right now, given how young the space is. We're all learning what works/what doesn't together.

Re: The Dirty Little Secret of Social Media Monitoring

Ken Burbary — Fri, 17 Jul 2009 16:50:45 -0000

Alexandros,

You make great points, that I concur with. As much as I would like to see less complexity in choosing/operationalizing a monitoring tool, the reality is that we need to rely on many sources. I'd enjoy a conversation with you to learn a bit more about the differences you've observed.

Re: The Dirty Little Secret of Social Media Monitoring

Ken Burbary — Fri, 17 Jul 2009 16:43:22 -0000

UPDATE: I've spent the last several days since this post was published working closely with the helpful and friendly folks at both Radian6 and Techrigy. We investigated the results, made some changes to configurations and re-run the searches. While this closed the gap between tools, there are still differences that remain, and are worth discussing.

I'll be publishing the results, observations and thoughts about both tools in a new post soon.

Re: The Dirty Little Secret of Social Media Monitoring

Alexandros Poulos — Wed, 15 Jul 2009 16:40:24 -0000

Ken, you started a terrific discussion here that is all so timely with the growing urgency of social media monitoring for any type of online marketing nowadays. In our agency's experience with these tools (esp. Radian6 vis-a-vis Alterian/Techrigy SM2), we have come across both quantitative and qualitative differences, especially as these tools attempt to gauge more advanced metrics like conversation sentiments, engagement levels, influencer ratings, and so on.

Frankly, I doubt you will be able to get a bottom line answer as to what these differences are attributed to. As noted by a number of participants in this discussion, the same issue applies to web analytics tools and search engines. So, what we usually do is rely on not just one, but a number of (paid and free) tools for social media monitoring and each time we go over the results we are getting with respect to the specific goal at hand. The social media monitoring tools are of great help (and will be of even more help as these technologies mature), but their use still requires significant oversight to ensure useful analysis and effective action taking.

Re: The Dirty Little Secret of Social Media Monitoring

Steve_Dodd — Tue, 14 Jul 2009 21:10:02 -0000

Well, I've been watching all of these comments and found a couple of key points very interesting.

1. The focus is on two of the many, many monitoring / analytic products and services out there. They all do things differently just like every responder to this post does things differently from their competitors. If each of you were given the same question by a client, you'd all likely have a different answer because you all do things differently to remain competitive.

2. No search engine (Social Media Monitoring or otherwise) returns the same responses nor holds the same data. Why didn't you also ask Microsoft, Yahoo and Google to respond?

The real issue is to know enough about your chosen tool provider(s) to ask the right questions to ensure you get the informationyou needed for your specific project. This is one key reason why the KDPaine's, Nielsen's, Cymfony's etc of the world attack the monitoring packages on a regular basis. In my opinion, too many people rely on the output of these systems without truly understanding their customer requirements, the system capabilties and to a certain extent, its shortcomings.

Re: The Dirty Little Secret of Social Media Monitoring

MartinEdic — Tue, 14 Jul 2009 11:52:54 -0000

At first glance I'd guess these variations are in how the media are classified- as the total number of results is not that far apart, 688 for R6 vs. 822 for SM2. The two system work very differently with SM2 having a longer historical database (about 2.5 years for some sources) that it pulls from. I would compare the new results for each daily as they come in or set a date range- SM2 allows this, not sure about R6 (I am former marketing director for Techrigy but otherwise not affiliated with them).
The other comparison that is far more important is the total amount of meta-data collected for each result. This meta data enables the wide range of analysis that the tools offer. Sheer results are meaningless without the ability to discover demographic, sentiment, emotive, popularity/influence, etc. of those results.

Re: The Dirty Little Secret of Social Media Monitoring

Adam Needles — Tue, 14 Jul 2009 09:55:33 -0000

Ken, I like the premise of this post.

I think it's important for any marketer looking at purchasing any type of marketing systems -- whether monitoring, marketing automation, content management, customer relationship management, etc. -- to keep in mind.

I know as marketers we tend to want to focus on the bottom line, but the details matter. And this means we need to understand 1.) how different platforms approach the task they are focused on managing/automating/etc. and 2.) what the underlying data is that they are using.

We need to be smarter buyers of marketing technology ... and we need to think more strategically about leveraging and purchasing marketing technology.

This -- in my mind -- is as much about a 'call to arms' for marketers as it is about vendor capabilities. We need to balance our strategic and operating mindsets as marketers with a system mindset as we build out our systems for complex, integrated marketing management.

I know this is a topic that Scott Brinker has touched on via his Chief Marketing Technologist blog (http://www.chiefmartec.com/), and I think it's a great call-out here.

Great dialogue -- and great start at raising the tech IQ among us marketers.

Re: The Dirty Little Secret of Social Media Monitoring

jimschwab — Tue, 14 Jul 2009 09:13:30 -0000

Hi Ken, great discussion here, as former SVP at Harris Interactive a market research firm and current VP at Techrigy a couple comments...

- Your application should determine how important content integrity, accuracy or representativeness are.... our clients use SM2 for everything from sales lead gen to qualitative market research, and many others
- All data/content have caveats, you need to understand your business intelligence sources caveats so you can make the most informed decision
- I have seen the best and most scientific data gathering conducted only to be twisted into whatever conclusion the analyst wants, it happens....
- If your business decision is a big one, use multiple sources of intell mixed in with your gut and experience

Techrigy has spent 3+ years understanding how to collect, store and help users understand SM content, it is a very complex technology challenge and it's still an early market. We have documentation, training vidoes and other tools to help our users best understand the content they gather. Happy to share these with you all.
Jim Schwab
VP Sales & Marketing

Re: The Dirty Little Secret of Social Media Monitoring

terryfoster — Tue, 14 Jul 2009 08:45:01 -0000

Ken

This has long been a problem in the media monitoring industry in general as no two companies approach it in the same way and it is nearly impossible for them to replicate each others data sets exactly. For the social media monitoring companies, they often struggle to identify where the influential and relevant "conversations" are taking place. There is also the issue with how much of any given site is spidered and as well as how often. These factors, amongst other things, can affect the end results in a profound way.

Re: The Dirty Little Secret of Social Media Monitoring

leonchaddock — Tue, 14 Jul 2009 08:32:32 -0000

This is very interesting. As others have said the technology varies so much from provider to provider. Taking this away from social media monitoring try comparing the results in Google to those in Yahoo, or google blog search to technorati. Completely different results in terms of volume.

As others have pointed out spam filters can play a major role in changing these numbers aswell.

What is key if you are looking at using a vendor for social media monitoring that you compare the results across several at a time to see what works best for you.

We would recommend you add us to the list of vendors to compare!

Thanks for this great subject matter!

Leon
www.sentimentmetrics.com

Re: The Dirty Little Secret of Social Media Monitoring

lance — Tue, 14 Jul 2009 07:53:07 -0000

While I do think it is prudent to understand the data sources that each of these companies use to compile their solutions this is not just an issue in social media measurement. If you look at the broader more mature Web analytics market the same issue exists. Google Analytics, Sitemeter, Alexa, and Compete to name a few off the top of my head all produce different results. Measuring activity on the social web is much more challenging and getting matching data sets across a number of different providers is not going to happen in the near term.

Re: The Dirty Little Secret of Social Media Monitoring

Chuck Hemann — Tue, 14 Jul 2009 07:41:35 -0000

Hey Ken - To your second paragraph here, the real danger is that the chances of us knowing who has the best coverage for blogs, forums or twitter beforehand is probably pretty small. That's what I'd be most concerned with.

Anyway, I look forward to the update from R6 and Techrigy.

Re: The Dirty Little Secret of Social Media Monitoring

M. Edward (Ed) Borasky — Tue, 14 Jul 2009 01:10:10 -0000

There's also the issue of how Twitter "sentiment analysis" is done. It looks like many of the Twitter sentiment analysis tools simply count the number of happy-face emoticons as "positive", frowny-face emoticons as "negative", and emoticon-free tweets as "neutral". This is what Twitter search apparently does, and these tools simply copy the Twitter search "algorithm".

While there's no "perfect" sentiment analysis algorithm yet, there are some that are *much* better than counting emoticons! If you're going to market a tool with a claim of "Twitter sentiment analysis", I think you should disclose the algorithm.

Re: The Dirty Little Secret of Social Media Monitoring

Jamie Favreau — Mon, 13 Jul 2009 23:46:28 -0000

I am definitely interested in what both sides have to say about this. What happens if you are a small business and can not afford to have both but both are valid in your business? How do you decide which is best? I think this is where they brains of both should explain why they monitor their stuff one way and why the other does it another. I realize there aren't standards but how do you choose the best thing for your strategy?

Re: The Dirty Little Secret of Social Media Monitoring

Ken Burbary — Mon, 13 Jul 2009 23:43:35 -0000

Jennifer - Thanks for tossing another hat into the ring here. I will take a look at Scott Labs for comparison.

Re: The Dirty Little Secret of Social Media Monitoring

Ken Burbary — Mon, 13 Jul 2009 23:42:56 -0000

You're right, no one tool is enough/ fool proof. Thus, it is increasingly important for marketers to understand what each tools limitations are, and what combination of tools is needed to effectively provide coverage.

Re: The Dirty Little Secret of Social Media Monitoring

Ken Burbary — Mon, 13 Jul 2009 23:40:31 -0000

Connie, Thank you for taking the time to respond.

This helps better understand some of the specifics of how SM2 is handling the searches. That, coupled with the support I'm getting from @jimmyrey are painting a better picture that I will elaborate on in an update to this post after the dust is settled. Oh, and for the purposes of our search, we were not filtering any sources.

Re: The Dirty Little Secret of Social Media Monitoring

Ken Burbary — Mon, 13 Jul 2009 23:35:21 -0000

Thanks for all your help/support on this issue today Jim, greatly appreciated.

Re: The Dirty Little Secret of Social Media Monitoring

Ken Burbary — Mon, 13 Jul 2009 23:31:04 -0000

Good points Josh. Exactly what I'm trying to bring to the surface with this post. As the social marketing activities mature, I think we'll have more established toolsets, a combination of search tools, that get deployed as a standard practice to give brands the best view of the landscape as possible. I don't see any single tool being able to either.