In my experience, crosstabs have not conceptually become more complex in recent years, but the need merge research data with other data sources has increased and become more complex. It is the reason that I see MRDCL becoming more of a survey data management system that excels at tabulations rather than simply a program that excels at tabulations. In 2022, it will excel at tabulations and reporting, but that’s another story, so let’s focus on the data management aspect.
Bringing data management to the fore
I am the first to admit that data management doesn’t sound interesting. It doesn’t sound particularly important, more administrative than skilled. The reality is that staff in data processing and data analysis teams often waste far more time than is noticed moving data from one place to another or getting the data in a convenient form to process what is needed. So, let’s look at three examples.
1. Surveys with respondent data in more than one file
This requirement is not new. There are many examples of this:
- Coded data from open-ended questions that you need to merge with the main survey data
- Data from platforms with limitations, such that when you export the data, you have to make two or more files (more common than you might expect!)
- Profile data about survey respondents, coming from a database or CRM system
2. Analysis needing data from more than one survey
This requirement is a growing area. In the past, pre-test and post-test surveys have been the mainstay of this type of need. However, analysis that uses data from more than one survey, analysis that tracks respondent behaviour over a period and transactional data all demand similar survey analysis tools.
3. External data, such as sales figures that you need alongside survey analysis
Another rapidly growing requirement is reporting non-research data, such as sales, targets, KPIs etc., alongside survey data. I deliberately called this ‘reporting’ rather than ‘analysis’, as, generally, those preparing PowerPoint reports and dashboards add this data, sometimes manually, but usually as a separate task from tabulations. However, as MRDCL matures into a data management system that excels at tabulations, we have increased MRDCL’s scope to merge external data as part of the tabulation process, making reporting, such as in PowerPoint, easier. More of this later.
How does MRDCL merge data?
MRDCL has three tools that make these types of analysis practical to handle. The three tools are:
1. READ function
The READ function has existed for many years, but it enables you to read data from more than one file or stream with different rules as required. For example, you can read data from more than one file, matching it with a record identifier (ID). The READ could be very simple, such as reading from a matching respondent record from each file one at a time. Alternatively, it could be more complex, such as where one file contains a database of customers of which a small number complete a survey. In the latter case, you may wish to use a lookup to merge data from the customer database with your survey data in the analysis. The READ function has all the tools you to control different scenarios, including where records are only present in some files, for example,
2. Excel data merge facility
Sales data, targets, KPIs and other metrics are often readily available in Excel files or CSV files. The Excel data merge facilities in MRDCL allow you to retrieve data from external files to either match that data to records or groups of records. To take a simple example, if sales data for ten sales regions is available in Excel, it is easy to automate the use of that data in MRDCL. What’s more, if the external file has updates, the analysis will automatically update. Automating these tasks and keeping all analysis and reporting controls in one place reduces both costs and errors. Again, MRDCL has all the tools to compare figures, such as comparing survey data with targets or similar data.
3. Table output templates
Putting external data into tables is something with which almost all other tabulation software programs struggle. Many can process large volumes of records; some can handle sophisticated analysis. However, dropping external data into tables is something at which MRDCL excels. You can achieve this by using simple templates that put external data where you want it to appear in the tables. This facility makes charting, reporting, and delivering data in dashboards easier, faster, and, more importantly, automated.
Using these techniques in other ways for automation
MRDCL also uses these techniques in other applications. Here are two examples:
Validating data
MRDCL can be used to validate data. For example, if a respondent claims to buy a product for a particular price, MRDCL can use a lookup to check whether the data is valid – for example, to check the price is within expected ranges. This facility means that an MRDCL script file remains unchanged; all that changes is the external that anyone can amend. Unskilled staff can, for example, add products or amend prices if there are new products. Additionally, it may be possible to automate the creation of the lookup file to automate the whole process.
Grouping data
MRDCL allows you to use external files to control how your data analysis. Consider, for example, where you want to group 1000 products under luxury, expensive, mid-range, budget. Again, if you can control this from an external file, you can automate the process like the previous example.
Looking at the big picture
Crosstabulations may have become conceptually more straightforward in recent years, but the data in those crosstabs has brought new demands. These demands are beyond the capabilities of most tabulation software systems. However, MRDCL’s data management and automation tools mean that you can meet these new challenges. I do not see these needs diminishing; indeed, you should prepare for increasing demand.
Ease of reporting
As an industry, market research has been slow to make ease of reporting a priority. The tools we have in MRDCL make a significant impact, plus the new reporting module available in MRDCL in 2022 will take this a step further. Buyers of research quite rightly expect data research agencies to provide information exactly how they want it. It’s not an unreasonable expectation when other sectors have managed this area so well. Talk to us about how we can help you improve your data management (the boring bit!), analysis and reporting.