Does your survey platform do everything you want?

There seems to be an ever-increasing number of survey platforms that purport to do everything. By “everything”, I mean they collect and analyse data, produce crosstabs or tabulations, generate charts and allow you to make a dashboard. That’s everything you need. Right? Well, maybe. Or, maybe not. This article explores the strengths and weaknesses of these platforms and will try to help you establish how to deal with any shortcomings. I’ll wrap up by showing how MRDC’s software may help.

There are good platforms and bad platforms

Like any other product field, there will be good and bad platforms. There will be expensive and cheap platforms. There will be easy-to-use platforms and ones that need regular use and training to use effectively. However, these more apparent aspects of product positioning can lead to overlooking weaknesses and stumbling blocks. Here are my 12 most common stumbling blocks.

12 common problems with online survey platforms

• The platform is primarily for (short) customer feedback surveys
• Sharing data with other applications is difficult/impossible
• Tracking studies where questionnaires change even slightly are problematic
• Multiple response questions are impossible/cumbersome to analyse and report
• Complex or custom crosstabs are not possible
• Repetitive parts of a questionnaire are difficult/impossible to analyse (e.g. meal occasions)
• Complex or repetitive variable definitions are difficult (or impossible)
• Commonly-used market research techniques like rim or target weighting are not available
• There is no significance testing or only one variant of significance testing
• Charting is limited to a small number of possible outputs
• Sharing projects with clients is difficult or impossible
• You need to buy additional (expensive) modules to get a full range of tools

The perils of inefficiency platforms

I saw an online demonstration of a competitor product a year or two ago where they showed how to handle rim weighting in their software; it was a new feature. It was a cumbersome process where the demonstrator showed how to upload targets, “sync” the data and produce a weighted table. The “sync” part failed in the live demonstration, which was embarrassing, but the real embarrassment, in my opinion, was the time it took to perform what I consider a simple task. Something that should take no more than five or ten minutes became a failure after thirty minutes of effort and “syncing”.

Why does efficiency matter?

The point I am making here is not that our software is excellent for handling rim weighting but that most platforms will come with inefficiencies. I have witnessed someone using another platform spend three hours adding the top two and bottom two boxes to about 100 rating statements. Again, you can achieve this in one minute with our software (and several others). Inefficiency doesn’t matter if it’s something you do once in a blue moon, but it does matter if it is something you want to do with any regularity or where errors are easy to make.

Exploring the 10 common problems

Of course, the 10 common problems I cited above are examples of the issues that platform users encounter. I’ve grouped these problems into three categories – data, analysis and reporting for further discussion in the rest of this article. I would make these three key points:

• Most end-to-end platforms have a weakness (most have a strength too).
• If a platform’s weaknesses are not a problem for you, then fine
• If a platform’s weaknesses are a problem, you need to fix it

If a platform has a weakness that affects your efficiency and what you can offer clients, it’s time to fix it. It’s like a dripping tap that will never get better and probably worsen as the industry moves forward and your clients’ expectations increase.

Data problems

1. Data traps

One of my pet hates is what I call ‘data traps’. What I mean by this is that once your data is stored within a survey platform, it is, at worst, impossible or difficult to take it anywhere else. This is almost unforgivable in an era when data should be easily portable. At a minimum, it should be possible to output data by pressing a couple of buttons to Excel, ASCII, SPSS and Triple-S. I get particularly annoyed when platforms make it easy to import data but difficult to export. That’s a real data trap.

2. Data complexities

Many platforms can only handle flat files, meaning there is one record for each respondent in the survey. Many surveys need to have repetitive chunks of data. For example, you may have questions about eating occasions, daily TV viewing, purchases, or demographics of each household member. Further, you may have data loops. This is where you might ask the same questions for several brands – for example, each make/model of a car that is considered when making a purchase. Many platforms will try to ‘flatten’ this data, but this is not practical if there are 1000 (or even 50) possible models.

3. Data basics

Some platforms struggle with things that I consider basic data structures. Multiple-response questions, particularly with many responses, can be problematic. Numeric values may be limited – for example, large numbers of numeric values with decimal places.

4. Question basics

Questions, where the responses are rotated or randomised can be cumbersome to handle efficiently as well as filters on questions, particularly if the filters are complex.

These issues may be roadblocks or require considerable work or effort to handle. Frequently, it is not just the time you have to spend to handle a complication; it is the heightened risk of making an error that is concerning. I have seen people working with data with a grim determination to fix a problem, but this does not make it right to use the software in this way.

Analysis problems

Analysis problems are harder to classify as there are so many detailed options that you may need.

1. Showing data how you want it

I think grouping responses together or building new variables that pull data together from two or more questions is a prerequisite. Similarly, it should be possible to group answers from numeric questions easily. Adding top-two and bottom-two boxes should be minimal work – or, indeed, a top-three category if that is what you want. Similarly, building banners for cross-analysis in tabulations should be easy and not limited to a small number of columns.

2. Significance testing and other statistical tests

Significance testing is one of many statistical tests you may want to apply to your analyses. It’s a test that is commonplace and demanded by many customers. However, there are many variants of significance tests. Most platforms are only likely to have the most common variant. In addition, a whole host of statistical tests are commonly needed. Amongst these are mean score, standard deviation, standard error, error variance, median, mode, maximum, minimum, chi-square and more.

3. Summary tables and more complex tables

Most platforms will allow you to tabulate x by y; most will allow you to apply a simple filter to your tables. However, you will often want to cut the data in more complex ways or summarise more concisely for easy reading or, perhaps, easy charting. Summary tables can be impossible or cumbersome to generate in some platforms. However, they are at the heart of market research reporting, enabling data users to quickly gain an overall picture. Of course, a summary table is only an example of how you want to view or report survey data, but it illustrates an analysis type you might regularly need.

Reporting and delivery problems

When it comes to reporting and delivering survey data, there are many routes that you want to follow. There may be a need for a PowerPoint report, a dashboard so that clients can explore data or an interactive tool that allows end clients to delve more deeply into the data. Further, there may be a need to deliver case data or aggregated data in a convenient form to be loaded into a client’s reporting systems. This may need to be an automated process carried out in real-time or periodically. I would consider two things as necessary, though:

1. Flexibility

Flexibility sits at the core of reporting and delivery. When it comes to reporting and delivery, generalisations are difficult to make. Being able to output respondent data, tables, charts etc., easily and with flexibility increases the chances of efficiently delivering what your clients want. I accept that this is a vague statement, but as I have stated earlier, some software wants to trap you inside or is limited in what you can deliver.

2. Simple or complex dashboards

When it comes to dashboards, there are two types – simple ones and complex, customised ones. Yes, that’s another over-simplification, but it is a significant differentiation. Most platforms have a dashboard component. For some platforms, this is their strength. I would expect a platform to have the capability of producing a basic dashboard quickly and easily. I wouldn’t expect a dashboard component capable of high customisation and complexity. Why? The answer is that basic dashboards should be easy and quick to prepare, whereas building a complex dashboard is a skilled task using a complex product like Tableau. Except for larger or more complex tracking studies, most surveys only need (or have the budget) for a basic dashboard. ‘Basic’ doesn’t mean unattractive or very limited, but it does mean easy to provide and easy to navigate for the recipient or user.

Where does this leave MRDC’s software?

Well, you might have guessed that I would return to our products at some point. In fact, I want to indicate how MRDCL can either work with your existing platform or become your data processing hub. Let’s explore these two possibilities separately.

1. Using your platform with MRDCL

First of all, MRDCL is not a ‘data trap’. You can get data in and out of MRDCL quickly and flexibly. Secondly, MRDCL offers solutions to all data and analysis problems cited above. You will not be restricted, and there are efficient tools to perform the tasks discussed. If the data platform you use is, for example, convenient for collecting data and delivering a dashboard, linking it to MRDCL will be easy. That, of course, assumes you are not using a platform that is a data trap. This will broaden your capabilities at a stroke. MRDCL has a free tool (Resolve), which allows you to share projects for in-depth analysis. MRDCL will soon have a new module so that you can automate PowerPoint decks too.

2. Using MRDCL as a hub

In principle, this is the same as using your platform with MRDCL. Conceptually, it is different as you are using the strengths of your platform and removing its weaknesses. MRDCL will allow you to generate any variables, tabulations, and analysis you want and interact with most applications. This gives you the power to have the best tools for whatever you want to deliver. If you need to interact with Tableau, for example, to generate complex dashboards, MRDCL will allow you to do this.

Let’s start a conversation

As I stated at the beginning, if your platform does everything you want without inflexibility and too much effort, there is no reason to look further. However, suppose you see bottlenecks, inflexibility, limitations or feel that something is taking longer than it should. In that case, MRDCL is likely to be a good solution. We like to connect to as many other products as possible and want to be known as a data processing hub. Talk to us, as we may solve problems inexpensively for you.