AI has many benefits for market researchers and insights analysts. It reduces time-consuming tasks, delivers deeper insights faster, and unlocks time for strategic work. However, AI also comes with risks that can be challenging to assess and mitigate, especially in the crowded AI marketplace. Knowing what's AI and what's AI washing is difficult and there are limited guidelines for evaluating AI solutions.
To assist buyers in navigating these challenges, ESOMAR, an independent global hub for research insights and analytics, created a list of 20 questions. These questions help evaluate an AI provider’s reputation, capabilities, human oversight, and data governance protocols. In this post, we will discuss how we solve for these important guidelines at Thematic.
Before reviewing our answers, make sure to download the full ESOMAR guide for AI buyers that provides context for each question and what to look out for.
This section will help you understand our credentials.
Thematic was founded by Alyona Medelyan, PhD. She started her journey in AI in 1998. Her Masters and PhD (sponsored by Google) were specifically in the area of making sense of text using algorithms. Her academic research resulted in algorithms used by hundreds of organizations. Since graduating, Dr. Medelyan worked in commercializing AI technology to help startups build their first products and established companies add AI to their offering. She founded Thematic, when several companies asked her help them understand what's driving their NPS.
By speaking with market research experts, Dr. Medelyan quickly learned the importance of transparency and trust over a black-box AI. Thematic's core solution was built around the idea of human and AI collaboration.
Our R&D team at Thematic is dedicated to continuously evolving our proprietary Hybrid AI, integrating both in-house and third-party algorithms and models. We believe that Large Language Models have solved nearly all tricky language understanding problems. But there are trade-offs that come from using them. With over 20 years of experience in AI commercialization, we specialize in expertly managing these trade-offs to deliver optimal solutions.
At Thematic, our mission is to get insights from every voice into the hands of every decision maker. This task typically falls into the hands of researchers working in Voice of Customer, Customer Experience and Customer Insights teams. These teams tend to be small, under-resourced and overwhelmed with requests. They often struggle with analyzing qualitative feedback, which is siloed, scattered across tools, and noisy.
There are 3 ways in which AI helps researchers solve these problems and delivers benefits:
Some people say that a Large Language Model is an oddly shaped tool. There has never been a tool that lets you ask any question and return a reasonable answer. The answer quality typically depends on the quality of the question you ask, and the quality of the model.
With that, there are several key problems that we had to solve when integrating LLMs in Thematic:
As you can see, there is no quick solution. What worked well for us is:
A) Narrow down individual tasks, for example, find broad themes, find sentiment of a sentence, auto-score customer conversation.
B) Make sure we provide high-quality specific context for that task, that reduces hallucinations and increases consistency.
What doesn't work well is keeping the task broad. For example, if you were to ask a Large Language Model "here's my dataset, what are the insights?", you will get a different set of insights every time.
This section will help you evaluate our services from a practical standpoint and whether our capabilities align with your business purpose and are likely to provide clear benefits.
Thematic is a best-of-breed AI solution for analysis of unstructured customer feedback. So, AI plays a major part of our offering. There wouldn't be Thematic without AI.
Here's a list of our key AI functionalities:
Thematic develops all AI solutions internally. We have a dedicated and well-oiled R&D team that runs rigorous process to build, evaluate and launch AI features in production. We use a variety of AI models, some of which are developed internally, and others delivered by third parties. We most commonly use with AWS Bedrock and Microsoft Azure Cognitive Services, which lets us use Large Language Models in a secure way.
Please note that we are are constantly updating our underlying AI models, as new models are released. We use an evaluation framework that is constantly evaluating model performance on individual tasks and picks the model that provides the best trade-off of speed, accuracy and cost.
Our proprietary non-generative AI models have been trained on a large volume of customer feedback that we gathered over the past 10 years with explicit permission of our customers. Before training, all data was de-identified to protect its origins. All customers had the option to opt-out from this process.
With the arrival of Large Language Models, we have stopped training custom models. We never train LLMs in-house or share our data with LLM providers for training purposes. Instead, we deliver accuracy through prompt optimization and a complex evaluation process involving small amounts of manually labeled data.
This section will clarify our ethical principles and our view on important topics such as biases, data security and resilience.
Accuracy of the output is top of mind for us at Thematic. We offer many ways to verify and validate the accuracy of its themes, sentiment, answer and summary quality. We also offer automated quality assurance with recommended actions to take to improve unreliable results. The main idea is that AI makes suggestions and user accepts or rejects them, while seeing the impact these changes have on quality.
While our AI models are powerful tools for feedback analysis, like all technology, they have their limitations. They may not fully capture nuanced human emotions or context-specific language depending on the dataset. To mitigate this, we continuously update our models and prompts. We also leverage human oversight to interpret complex insights, ensuring the highest accuracy and relevance in our analyses. This blend of AI precision and human expertise helps us deliver balanced and insightful feedback solutions
At Thematic, we take our duty of care seriously when designing our AI solution. We have outlined and are following the specific guidelines we deploy for the use of Generative AI at Thematic. We see ourselves as the guardians of your data, making sure it's anonymized and handled with the utmost security. We're also committed to fairness, using smart techniques to spot and tackle bias, so you get insights you can truly trust. Our goal is to keep everything transparent and empower you with the knowledge to make confident decisions, making sure our technology lifts everyone up in the process.
This section will explain human involvement and oversight, both in development and use of Thematic. You'll learn what role the human plays when working with Thematic's AI.
The vast majority of Thematic is AI powered, so by default all analytics are AI generated. The model created is completely transparent and the user can introspect it and the results at any time.
We believe it is necessary to understand and evaluate how AI has analyzed data, to ensure the system delivers a high quality of information and supports trust in the insights.
Where generative AI is used we make it obvious and we keep track of user review cycles, so that it's clear which analysis was validated by a human.
Thematic is guided by clear ethical principles, focusing on fairness, privacy, and transparency. These principles shape how our AI behaves, ensuring it treats all data equally, protects your privacy, and operates in a clear and understandable way. We ensure analysts can see exactly why decisions are being made by the model and provide tools for them to determine if bias has been introduced. Our products are developed by a diverse teams who spot and address biases at all stages of development. We also maintain strict data privacy standards. By keeping these values at the core, we ensure our AI serves and respects everyone involved.
At Thematic, we integrate human oversight by having dedicated teams review and audit our AI models, ensuring they align with ethical standards. This includes both staff within Thematic, and at our customers. This involves continual verification of data accuracy, and assessing the impact of AI-driven insights. We involve cross-functional teams in decision-making processes to maintain diverse perspectives.
In this section, we'll help you understand our approach to data protection, information security, compliance policies, procedures and practices.
At Thematic, we rely solely on Microsoft and AWS for our Generative AI models. We trust their top-tier data practices to ensure our training data is accurate, complete, and relevant. Their strong commitment to data integrity and privacy aligns perfectly with our goals and legal requirements. By partnering with them, we ensure our AI provides trustworthy insights while upholding data security and confidentiality.
Our customer's data is only ever used to train models wholly within that customer's workspace and control. Data can not be used for any other purpose.
At Thematic, we meticulously document the origin and processing of training and input data in our internal knowledge base. This is critical as transparency and ownership are key value of our company. This information is available to authorized stakeholders to ensure data integrity and trust.
Models controlled by customers also have full documentation of how and when they were created along with any modifications by the customer.
Here's the link to our privacy policy.
To comply with data protection laws, Thematic implements strict data anonymization and encryption protocols, ensures transparent consent processes. Every year we undergo a strict audit under our SOC2 TypeII compliance, to make sure that we comply with all legal data handling practices, and keep abreast of the latest legal requirements to maintain robust privacy measures.
We are compliant with the GDPR, CCPA and other legislative requirements.
Please note, that the full ESOMAR question talks about protection of research participants. In Thematic, we do not work with research participants. We only analyze public data or data that the company already gathered from their customers, prospects, or employees.
To enhance Thematic's resilience against potential threats, we conduct regular penetration testing to identify and address vulnerabilities in our systems proactively. Additionally, we comply with the SOC 2 Type II standard, which involves comprehensive audits of our security controls to ensure the confidentiality, integrity, and availability of our data. These measures collectively reinforce our commitment to maintaining a secure and reliable platform for our clients.
Security, availability and confidentiality are considered at all stages of the software development lifecycle. Where possible we use the NIST frameworks to match industry best practices.
At Thematic, we clearly define and communicate data ownership, intellectual property rights, and usage permissions to all stakeholders. We ensure that our clients understand how their data is used and stored.
In the video below, see how we prioritize data and insights governance within the Thematic platform.
Our system prioritizes reliability, privacy, and data sovereignty. We guarantee that AI learnings remain the property of the customer. We uphold stringent data sovereignty and privacy standards, ensuring that data remains within the specified region or data center. By leveraging AWS's robust security and privacy measures, we maintain data protection and compliance, allowing us to concentrate on delivering our core services responsibly.
Each customer is assigned a particular geographic region and no services outside that region are used in the delivery of Thematic's services.
At Thematic, we clearly establish that the ownership of the output generated by our AI models resides with the customer. We ensure this understanding is part of our agreements, providing full transparency and control over the insights and results produced from their data.
In this post, Thematic responds to important questions from ESOMAR to clearly and transparently share how we use AI. We explains that AI is used throughout Thematic, to integrate, analyze and report on feedback data. We pride ourselves on holding ourselves to the highest standards of integrity, ethical principles and transparency with data and AI.
Book a demo today to explore how our AI data analytics will make it faster and easier to get your Voice of Customer, CX and product insights.
Join the newsletter to receive the latest updates in your inbox.
