Teaching use of AI with meta-reflections

The use of generative AI tools like ChatGPT, Claude, and others is rapidly increasing in business and management education. As an educator who recently had students use these tools for research assignments, I wanted to share some thoughts and takeaways.

My goal was to have students use AI tools for certain tasks, like literature reviews and critiques, while also requiring them to do “meta-reflections” or their thoughts after they performed certain tasks. In other words, these are reflections on the content, process, and premise of their activities.

Content reflections are about the students’ thoughts on the actual content they co-created with the AI tools. What prompts were effective or not? What were some sample outputs generated? This grounds them in examining the substantive work enabled by AI.

Process reflections are where students detailed their specific process of utilizing different AI tools. How did they employ each tool and combine them into an effective workflow? This reveals how AI impacts their actual approach to academic tasks like researching and writing papers.

Premise reflections are where students surfaced and critiqued their values and assumptions about what AI makes possible. They compared how they would have completed assignments with vs. without AI tools. This urges them to challenge preconceived notions of how AI should and should not be used.

In assessing how students felt about and used AI tools, I encountered the following takeaways.

(1) Generative AI tools as "autocomplete engines". I find it fascinating that students themselves identified how we cannot expect flawless factual accuracy from today’s AI chatbots. My students and I have found out that for academic and research purposes, the strengths of the AI tools  lean more towards brainstorming, exploring ideas, summarizing documents, and translating thoughts into text. Teachers should provide guidance on the appropriate use cases and limitations of different AI tools to set proper expectations.

(2) The art of prompting is an important skill. In business and research, the ability to clearly articulate prompts and instructions is extremely valuable - be it directed to managers, subordinates, or AI chatbots. Students who took time to craft thoughtful, detailed prompts also produced more concrete, meaningful outputs with AI tools. Guiding students to provide clear goals and constraints to AI teaches a transferable management skill – giving articulate and effective instructions.

(3) Leveraging on AI-assisted workflows unlock unprecedented efficiency of learning. Students experimented with tools like ChatGPT, Claude, and niche research apps to assemble customized AI workflows. They critically examined the strengths of each tool and learned to combine them to maximize productivity. There were specialized tools like Elicit, Consensus, and Perplexity that helped in searching for relevant literature. ChatPDF and Humata helped in engaging with learning materials and research articles. We should teach students this iterative design process and how to take advantage of different AI capabilities.

Surprisingly, not all students were initially eager to use AI tools for academic and research purposes - perhaps because my students are in their third and fourth year of college already. Some have developed their own unique voices in writing, and thus, they were skeptical about relying on AI for what they felt should be original work in the traditional sense. However, most experimented with different tools and developed customized workflows to use them effectively. The meta-reflection assignment provided insight into how critically they actually leveraged AI.

Indeed, generative AI holds tremendous promise, but also some pitfalls, especially for student work. With guidance and meta-reflective assignments, educators can teach students to leverage these tools effectively while developing transferable skills around prompting, critical thinking, and ethical assumptions. The meta-reflection process ensures students stay grounded in human-centered values as they integrate AI capabilities. Our role as educators is to provide structure and perspective so that the students may utilize AI consciously and for social good.