Online Card Sort software

(Yes I am back from the summer break...!)

As always I am very interested in the new (and sometimes free) tools that are made available online that may make our lives easier as researchers. See other posts here, here or here. While some of you researchers may still have a hard time of getting a more basic "drag-'n-drop" feature programmed in your online questionnaire, there now online programs like Optimalsort and WebSort that allow card-sorting online. Create a study, send a link to participants, and analyze the results- all through a web-based interface.

The two companies do not really offer the tool for free -- the guys need to make a living -- yet they do have a "pay-for-use" scheme. Check them out and play a bit if you have a moment...

The Problem with Surveys

Paul Neto tries to keep things simple on his blog. Last Monday, he posted his vision on why today's market research surveys are broken. Visit the original here.

He takes us through a short history of surveys and research methods ever since central location became en vogue in the late 1950s and ends with the introduction of a couple of MR firms who are getting some recognition for doing innovative things today.

He ends his post by saying that ... online research is a sector that has been growing incredibly fast... ...Though, due to lack of innovation there are many concerns in it's direction. Some major firms are even starting to pull away from using online research due to the rise of professional survey takers, poor methodologies, weak panels and questionable sampling techniques.

Nearly 10 years ago, we were all trying to justify online research to clients. Today just about everyone is online because it's the place to be. The next generation of research is not far around the corner. It really only takes a little innovation to lead to big changes...

I do applaud his post and agree with almost everything he says. I recommend you visit the original here. But I wonder: how should we be preparing ourselves as an industry? The answer I believe must be found somewhere in the shift which will undoubtedly happen over the next 12 years: we will move away from measuring "claimed behaviour" (the whats, whens, wheres and how much) toward analysing "real behaviour". The only thing left in the near future that we may still need to ask for is the why. All the other Ws can be measured simply because (research) data will become a commodity now that all new technologies today are based upon the concept of at least having huge databases.

So on one hand we should have more "web 2.0" type of communication with respondents: really engage them by using survey methods which facilitate creativity, collaboration and sharing of experiences and information on one hand. On the other hand we should facilitate those companies owning the databases to make data accessible and transform digits into data into information into knowledge into insights...

Drop me a line, I am interested in hearing your thoughts!

Please secure my concept!

All of you who are dealing with online concept testing heard the question: "how do you secure my stimulus material in your online survey?". I am sure you've all come up with the appropriate answers to your client. Either you use specific software tools to secure graphic material in online surveys, or you actually designed the tool yourself within your company.

Using JavaScript, it is common practice to ensure that respondents are discourages from copying the images within a survey. The advantages of using such techniques are:

• Disabling the use of the right click button to prevent the “copy,” “save,” and “print” functions.



• Restrict the ability to print from the User’s toolbar.



• Attempting to print a page results in a blank page printing.



• Security is delivered without requiring any specific view program.
  
  

But there simply is no way to completely prevent a respondent from saving images, there are limitations. A savvy internet user may still save the images: The respondent may use “Print Screen” to get a picture of the image. However, when using Print Screen the entire screen is saved, including template headers and footers, forcing the respondent to edit the saved image. The image may also be saved using the “Save As” from the browser toolbar. The tools do not prevent the respondent from viewing the source code which contains the link to the image. However, this function is disabled if the respondent is using an Internet Explorer browser to take the survey due to the use of a secure survey link.

So what to do?

Other non software related measures to secure graphics and video include:

• Excluding certain cities / zip codes / regions from your sample (perhaps those where competitor employees are living / working). E.g. should you be doing an online Kraft concept test in the US, why not consider to exclude the home-town of P&G



• Obviously you should always have the "security screening": excluding certain professions (e.g. marketing, press, certain industries related to the product)



• Excluding certain e-mail accounts (e.g. no panellists that enrolled in the panel using an e-mail address of the a certain company will be invited)



• Showing graphs and images for only a couple of seconds (to limit the time of actually taking a picture of the screen with a mobile device or photo camera)

I am not in favour of sending out Non-disclosure agreements to panellists prior to any certain research: this will only trigger more attention!

In the end, should you have found the perfect software, it is still possible for malicious respondents to simply take a picture of the screen with their camera or mobile phone!

Fact remains that no 100% guarantee can be given, but remember: this is valid for other field work modes too, right? Any ideas?

To outsource or not to outsource scripting?

In a recent post on GMI's blog the five main reasons to off-shore of data-collection and processing and even project servicing are mentioned. According to ESOMAR’s 2007 Global Market Research Report, outsourcing has become wide spread among the world’s 25 largest market research firms in the search to stay competitive by lowering costs. The Top 5 reasons mentioned in the article are: 1) more time to spend on insights, 2) Economies of scale, 3) alleviate skill shortage, 4) Save on technology costs and 5) quicker response to customers. I guess the real underlying reason for the industry to outsource is the assumption that the cost savings gained by this practice will simply flow back into the company coffers. And indeed: many companies now use outsource services, especially for questionnaire programming (scripting). But I can think of plenty of reasons why outsourcing of scripting is not the best strategy for our market research industry. Here the most important ones that come to ming: When does outsourcing of questionnaire programming not make sense?

1. When it involves a lot of non-standardized research projects If your research firm is in the business of ad-hoc customized market research, every survey will be more or less different, which makes outsourcing them risky. Communication with local programmers is difficult enough, won’t it be impossible with a non-native English speaker living half-way around the world in another time-zone? These scripts are best handeled in house by researchers who have first hand knowledge of the research objectives.
2. When the questionnaire is the core-source of success for your research project If it is critical to the success of your project, outsourcing is probably not the best solution. A lot of back-and-forth between the scripter and researcher will take place and again time zone differences impede the relationship when frequent discussion is needed to communicate requirements and last-minute changes.
3. When experience is required (know-how) If a manual can replace the briefing to the programming or DP department than send the job abroad. If expertise is needed from the scripter, if scripting is another quality check in the process of getting the best suited online survey programmed, think twice before you decide to do it abroad. How can they perform a task the way I want it done without asking a lot of questions…the questions will drive me nuts? I remember hard it was to convince an Indian programmer that chocolate sprinklers in Holland are used for sandwich filling (nice: with lot's of butter!) and it really did not belong to the Cake / Pastry category!

An additional concer has always been retention of employees at the vendor. And now it's a widespread one: Staff churn is absolutely increasing. At first it looked as if it was just the programmers which college kids. These kids get a few months of experience and move on to greener pastures in their country. But we're seeing it across all staff levels now. It's a big issue.

Once a MR firm starts to outsource, it will take months (if not a couple of years) to realize that outsourcing may not be the best approach for efficient and seamless scripting. By the time the company realized this, the people who had the knowledge of doing it in-home would already have been laid off. And it will cost double the effort to obtain this business critical knowledge back in the company.

Add the WOW to online shelf tests: what's possible today?

On December 5th, Merrill Dubrow had a post in which he gave reasons why today's market research business is broken. I described four causes:

1. Length of questionnaire
2. Boring, repetitive, poorly written surveys
3. Declining cooperation rates
4. Great variation in quality across online sample providers

Many people left a response so Merill's comments are right on the button! Oliver replied that the issues are all intertwined, and quite rightly so! #4 is a consequence of #3, which is a consequence of #1 and #2. So, we need to fix #1 and #2 to fix #3, which will allow us to fix #4. I am currently checking what's out there that will help make the surveys more engaging, and I am specifically looking for today's interactive survey solutions. Here is a first great example from Fuse: Fuse Pack Test Fuse specialises in interactive market research survey solutions globally and they claim that all Fuse stimulus can be embedded into 3rd party surveys. So what other companies are offering great examples of engaging respondents? I am particularly interested in seeing 3D product design tests, package test and Shelf tests. Here is just another example, but if you now any others, just leave a comment! CIAO's (Greenfield Online) Fridge Shelf Test: So anyone who would like to add other examples of great shelf tests? Just leave the name of the company, your reason why you like it the URL to their demo in a comment. I will add to my list below (as long as the demo is openly accessible over the Web) Looking forward to receiving your other impressive examples!

• Fuse Shelf Tests
• GMI's Virtual Shopping
• Harris Interactive's Shelf Test
• Synovate's 3D product display