Additional input needed...

As part of our 2023 Q Report survey of Quirk’s readers earlier this summer we asked an open-end on their current or planned use of generative AI in their work. (“Are you using or planning to use generative AI in the research process?”) The yesses and nos (and all the variations thereof) were pretty equally split, with a slight edge to those saying they are using or planning to use it. 

With this being early days (and, in the case of this survey, which was fielded in July, even earlier days), it’s understandable that there are so many in wait-and-see mode. And, unlike any new method or tool in recent memory, major and unavoidable issues around privacy and data security loom over its use by companies and organizations, as many of our readers indicated in their responses.

In the spirit of things, I used ChatGPT to generate a report on the responses to the open-end. Here is an edited version of what it came up with (a bit bland but not bad!) along with some handpicked survey responses sprinkled in to add color.

Survey response: Yes, using generative AI

A significant portion of respondents indicated that they are already using generative AI in their research processes. Some of the common applications include:

Data analysis: Generative AI is being used to analyze data, which can include tasks like summarizing findings, coding open-ended responses and identifying trends.

It is built into our dashboard product and we always use it for on-the-fly analysis.

Content generation: Respondents are leveraging generative AI to help create content for reports, blogs, newsletters and questionnaires.

Idea generation: Generative AI serves as a valuable tool for brainstorming and idea generation, helping researchers come up with innovative concepts.

Yes, we explore broad topics/genres/audience interests this way. More in a brainstorming approach as well as analyzing open-ends.

Survey development: It is used to assist in the development of surveys, including generating question options and response lists.

I use it as a "thought partner" when writing questionnaires. For example, I might use it to help me define a technical term I want to use in more consumer-friendly language or give me an exhaustive list of answer options to closed-ended questions.

Text analytics: Generative AI aids in text analytics, making it easier to extract insights from large volumes of unstructured text data.

ChatGPT helped me come up with a calculation to prove the impact a survey I implemented was having on a corporate goal. We have experimented with it doing some basic text analytics, focus group write-up summaries, questionnaire-writing assistance, etc., as well.

Report writing: Some respondents use generative AI to automate or assist in report writing, helping to create summaries and narratives.

We will initially be using it to 1) generate or improve questionnaires and 2) identify fraud. Then we hope to use it to 3) create reports from survey data.

Survey response: Yes, planning to use generative AI

Another substantial group of respondents expressed their intention to incorporate generative AI into their research processes in the future. While the exact plans varied, common themes included:

Efficiency improvement: Many organizations see the potential for generative AI to improve research efficiency by automating time-consuming tasks.

Idea generation: Similar to current users, those planning to use generative AI aim to harness its idea-generation capabilities for research and content creation.

Secondary research: It is considered a valuable tool for secondary research tasks, including literature reviews and background research.

We are testing ways of using it. For now it is being used most effectively in improving desk research and in helping to build response option lists for surveys. We will test it for open-end and interview transcript theme identification and summarization as well.

Coding and analysis: Respondents foresee the use of generative AI in coding and analyzing qualitative data.

Questionnaire development: It is expected to play a role in questionnaire development, making the process faster and more efficient.

Have looked into platforms that use AI to speed up and simplify the analysis of open-ended survey responses and feedback from qualitative interviews. Following this year's Quirk's event in London, I am also planning to try out ChatGPT more to develop questionnaires and discussion guides.

Content summarization: Planned usage includes using generative AI to summarize research findings and insights.

Survey response: Mixed intentions

Some respondents provided mixed or uncertain responses, suggesting that they are in the early stages of exploring generative AI's potential but have not yet formulated concrete plans for its integration into their research processes.

Testing for summarizing of open text. Early tests to generate concept stimulus had very poor results so unlikely to use for that any time soon.

Currently have an innovations task force researching this.

We're having fun – but it's not integrated into our workstream.

We’ve tried it a bit and will continue but we don’t produce crap so it has to be stellar.

I think we’re going to be late adopters on generative AI. For literature reviews, it’s still useless. (Too many hallucinated articles.) Within the restrictions of a single dataset, I can see it being useful to flush out any additional themes from a series of transcripts or help create more sophisticated visualizations of relationships between codes. But for right now, it’s a glorified auto-complete.

Possibly – we have a huge AI governance process that none of us really feel like dealing with, so it would have to be an amazing benefit to go through the pain. We are going through this now in order to use (legacy) Clarabridge. 

Survey response: Not currently using generative AI

A smaller group of respondents indicated that they are not currently using generative AI in their research processes. Some cited concerns about privacy, data security and the need for more information before adopting these technologies.

Where possible and legal but haven't done so yet. Waiting on guidance from the company on how to leverage AI.

Would love to but have to wait for our IT security team to give us the go-ahead. Don't think it will happen any time soon.

Beginning to use. Experimenting with a ring-fenced version of ChatGPT to maintain proprietary information.

I am the only research person and have product/brand management responsibilities too, so no time to carve out to learn about this.

No – company won't allow it on work computers.

Two extremes. 

A sampling of a few of the more definitive takes on the question about AI usage:

PRO

Here is how we are using it. Research design: find and summarize background information, define target audiences, identify market gaps, explain complex concepts and topics, connect different ideas, journal review, tutorials and knowledge test, challenge conventional wisdom, brainstorm new approaches, empathy (“describe how a patient would feel about …”). Questionnaire design: generate response list, bias check, question authoring. Data process: link to our data file (Google Drive – had issues linking it to our OneDrive), code simple OE data (color), paraphrase and classify text, generate SPSS syntax or Excel VBA, create data analysis outline/suggestions. Project communication/reporting: fix grammar, lengthen or shorten, critique writing and give feedback on how to improve, continue writing on an idea, write in the style of, create reporting outline to “tell a story,” chart/visual suggestions.

CON

I don't see this as valuable for us at all. If anything, it's a hazard for better bot responses. I don't understand what the purpose of Chat GPT would be in research, as we are crafting questions around very specific goals. In my experience, it's just a more user-friendly form of search. The results I get are general common knowledge. Nothing mind-blowing.

Absolutely not if I can help it.

Not yet but it will be forced upon us. 

Yes. We are researching how to leverage AI tools so we can fire people and boost profits.

Making inroads

Generative AI, exemplified by ChatGPT, is making inroads into the research process across various industries. Current users find value in its ability to automate tasks, generate ideas and assist in data analysis. Those planning to adopt it in the future see opportunities to enhance research efficiency, particularly in areas like coding, content generation and secondary research.

However, there remain concerns about privacy, security and the need for careful implementation. As generative AI continues to evolve, organizations will need to weigh the potential benefits against these considerations and tailor their adoption strategies to suit their specific research needs and objectives. The landscape of generative AI in research is changing rapidly and it will be intriguing to observe how these technologies shape the future of research methodologies and practices. 

Methodology

The Q Report work life and salary and compensation study of end-client/corporate researchers is based on data gathered from an invite-only online survey sent to pre-qualified marketing research subscribers of Quirk’s. The survey was fielded from May 24 to July 10, 2023. In total we received 1,969 usable qualified responses of which 707 were from end-client researchers and used for this end-client report. An interval (margin of error) of 2.17 at the 95% confidence level was achieved for the entire study. (Not all respondents answered all questions.)