Written by: Yi-Chieh Chen, Project Manager at Lee Muyueh


The following AI tools related to the content below are available for your use:
Transcription Assistant
Design Tool Assistant


In product design and user interviews, transcript analysis is a heavy and critical phase. As a designer or product manager, extracting valuable insights from large amounts of transcripts often requires a significant investment of time and effort. From a conversational perspective, our team hopes to use GPT technology to develop a transcript analysis tool that assists designers, aiming to solve the major challenges in the tedious process of handling transcripts.


The starting point of this project originates from our understanding of designers' daily work: the data volume of transcripts is massive and often unstructured, making it an extremely challenging process to dig out meaningful insights from them and apply them to design frameworks such as Customer Journey Maps, Empathy Maps, and Business Model Canvases. Our goal is to develop a set of tools that can not only automate these tedious tasks, but also guide users to provide supplementary information in different steps and generate formatted reports. It is not just about completing reports, but understanding the characteristics and goals of the product or service better during the generation process.

 

Development Process: How to Make the Tool More Efficient

During the development process, we conducted numerous experiments. Below we share the development experience and limitations regarding three core requirements:
 

1. How to make responses based on facts

   We know that a common issue with generative AI is "hallucination," where the AI generates fictional or incorrect content. Therefore, when designing the GPT tools, we particularly emphasized that responses must be based on the transcript content uploaded by the user. To this end, we designed several mechanisms to ensure the authenticity of responses:
   
   -Based on transcript content: Every response is generated based on specific content in the transcript, preventing the AI from improvising freely. For example, GPT will first check whether the transcript contains the user's question, and will respond only if the answer is yes, thereby improving the accuracy of the answer.
   
   -Prompting missing data: If certain critical data is missing from the transcript (such as speaker names or specific background information), GPT will actively ask the user whether they can supplement this data and include it in the analysis process. This not only enhances the accuracy of the answer but also avoids misleading conclusions caused by incomplete data.
   
   Limitation: Prompts can reduce the probability of fabrication, but speculative responses may still occur due to the AI's reasoning capability.
   
    

2. Experiments with Google Drive integration and image formatting

   To make the analysis results more practical, we utilized the GPT Action feature to integrate the GPT tool with Google Drive, designing an automatically generated report formatting system. This feature is activated when a user calls the Action function (for example: help me make this into Google Slides). Key aspects of this feature include:
   
   -Automated report generation: After the user completes the transcript analysis, the generated results are automatically applied to a pre-designed Google Slides template. In this way, designers not only obtain a complete transcript analysis report but can also view the formatted visualized results immediately. However, these results are currently stored in the cloud folder of the GPT developer, which presents data security issues that need consideration if launched publicly.
   
   -Templated presentation of images and text: The system can automatically transform textual results into visualized design frameworks, such as a Business Model Canvas or an Empathy Map, and embed them into the report as images. This not only reduces manual organization time but also makes the final presentation more professional and aesthetically pleasing.
   
   Limitations:
   -It is necessary to consider data security regulations for transcripts, because the generated results are currently stored in the GPT developer's cloud folder.
   -When the number of generated characters is too large, it may exceed the layout and affect visual appeal.
   
    

3. Guiding users to supplement information

   When analyzing transcripts, AI frequently encounters situations with insufficient information. Therefore, we added multiple confirmation points and data supplementation inquiries into the instructions to help users provide more complete data during the analysis process:
   -Interactive confirmation: After completing the analysis of each step, GPT will confirm with the user to ensure the generated content meets expectations, and ask whether supplements or modifications are needed. This design not only enhances the user's sense of participation but also ensures that every detail in the transcript is fully utilized.
   
   
    

AI is Not Omnipotent: Randomness and the Necessity of Human Assistance

Although our GPT tools can greatly improve the efficiency and accuracy of transcript analysis, AI is not omnipotent. Generative AI will still produce less-than-ideal results due to the randomness of its internal mechanisms. This means that human assistance and review remain indispensable.

-Randomness of generated content: Even though GPT can provide high-quality responses in most cases, some unexpected answers may still appear. These answers might be over-simplified phrasing adjustments, or skipping important steps under certain contexts. This requires users to check and adjust after each generated result to ensure the final outcome meets actual needs.

-Human review and supplementation: The advantage of AI lies in automation and efficiency, but for certain more nuanced or high-level analysis tasks, human assistance is still required. Designers need to make judgments based on the generated results and propose corrections when necessary, so that the entire analysis process can truly achieve the ideal effect.


 

Conclusion

During this development process, we deeply realized the critical value of "clear definitions" and "correct questioning." Whenever we conversed with GPT but failed to obtain the expected analysis results, it became an opportunity for us to reflect on whether there was room for improvement in our own question design. Such a process not only allows for a re-examination of the problem itself but also prompts the team to develop a deeper understanding of the content structure within the transcript when exploring "problem definitions."

This is precisely the chemical reaction we expect "human-AI collaboration" to bring — AI is not just a tool; it also becomes a partner that guides us to rethink and deliberate repeatedly. It allows us to continuously refine our insight and analysis capabilities through interaction with technology. This kind of collaboration not only improves work efficiency but also leads us toward more accurate and profound discoveries during each iteration.


The following AI tools related to the content above are available for your use:
Transcription Assistant
Design Tool Assistant