What should I look for when automating survey building?

Tips for automating repetitive tasks in survey research 

Editor’s note: Brad Patton is chief software architect at Researchscape, and Jeffrey Henning is chief research officer at Researchscape International.

If you do a lot of survey research, you should investigate automating repetitive tasks. Before automating, you need to implement time-usage tracking to understand what needs to be automated. 

The first step is to implement time sheets for your staff, tracking those items you may wish to automate: in our case, we have tracking codes for questionnaire design, survey programming, fielding, data cleansing and analysis, among other common tasks. 

The second step is to create a feature database, which can be as simple as a spreadsheet or as complex as a case management system. As subtasks come up, create a case for a new issue or upvote a case that isn’t. For instance, in survey programming, we had a list of manual steps the programmer would take for “Choose all that apply” questions, such as making sure there was an exclusive choice like “None of the above” and setting its properties appropriately.

By tracking staff’s time and tasks, you can prioritize what you want the IT department to automate. For instance, early on in our survey programming automation, we automated many details related to “all that apply” questions and “choose one” nominal questions. 

Six questions to ask before you automate your survey building

You’ll also need to review if your survey platform supports the automation that you will need. Based on our own experiences automating all aspects of survey research, across five major platforms, our team pulled together a list of features and capabilities a research firm should look for in choosing a survey solution.

• Does the platform have a rich application programming interface (API)? An API is the interface between the survey system and other software applications. The more functionality the API offers, the greater the flexibility you have in programming automations.
• Is the API well documented? If the API does not come with clear and comprehensive documentation, your IT staff could spend hours or days trying to find or reverse engineer what you need.
• Is the platform affordable? Are there hidden fees or limitations to how you can use the solution? Is use of the API within your budget? For one platform, given the cost of its API, we had to forego the API altogether and limit our automation to reading and writing XML files that the platform could import and export. That significantly limited what could be automated.
• Are you able to push and pull data through the API? If all you can do is pull response data out, then you won't be able to save time by using the API to automate survey programming, for instance.
• Is the API consistent and well designed? You'd be surprised how many are inconsistent, often due to solutions cobbled together through mergers and acquisitions or because the API was an afterthought. An inconsistent design yields inconsistent formats for results and more time spent developing. For instance, one platform sometimes uses “true” or “false” in parameters, but for other parameters uses “0” or “1” instead; some names of API calls use abbreviations while others spell out each word. Such inconsistencies make it harder to program without constantly consulting a reference guide.
• Is the API abstracted in any way? Some platforms simply return internal data structures (for instance, for question types). Unfortunately, this means when the vendor makes changes to its application it breaks your solution because those internal structures have changed. Then you can count on being offline until you can work together with the vendor to figure out what changed and how that affected your data. 

Because our clients use a range of survey platforms, we’ve learned the ins and outs of these different APIs. This led us to be deliberate about the platform we choose for our own needs.

Thanks to our careful study of our team’s time usage, and our subsequent investment in automating the most time-consuming tasks, we were able to grow 55% over the past 12 months without hiring. Almost every organization is short staffed. Reducing time-consuming grunt work should be a priority. Make sure your survey platform’s API will support your automation needs.