Earlier this month I had the chance to speak at the Spring Teaching Forum hosted by the Poorvu Center for Teaching and Learning at Yale University. My visit to New Haven was brief but very enjoyable, and I'm grateful for the hospitality of my teaching center colleagues there. I had a lively conversation with the Poorvu Center graduate teaching fellows, who asked very good questions about teaching with technology, and I picked up Annotation by Remi Kalir and Antero Garcia from the bookstore, which tells you what kind of impulse buys I like to make. And I got a tour of Sterling Memorial Library, which houses the Poorvu Center, and is designed to look just like a Gothic cathedral. See my pictures below!

The forum was the centerpiece of my visit, however, and the topic of the forum was, no surprise here, teaching in an age of generative artificial intelligence. I provided the keynote, and two Yale faculty members served as respondents, sharing their experiences and perspectives on teaching an AI. All three of us then served as a panel for a lengthy Q&A session. There's still a lot that higher education still has to figure out about generative AI and what it means for teaching and learning, but I feel like we have collectively made some good progress in this area since ChatGPT was released last November.

As a case in point, one of those faculty respondents, Tisa Wenger, professor of American religious history, shared an AI-based assignment from one of her spring courses that I found particularly clever. She co-taught a graduate course titled "History of Modern Christianity: American Encounters, Postmodern Transformations" with Erika Helgen, associate professor of Latin American and Latinx Christianity. Their students were in Yale's Master of Divinity program, a professional program for Christian ministry.

Tisa and Erika knew they wanted to explore the use of ChatGPT this spring, and so they put a placeholder assignment on their syllabus in January. "Artificial intelligence and history: Students will engage with the ChatGPT bot on a topic that we have covered in class, analyzing the AI response. Due March 1st. 15% of final grade." They were intentionally vague, knowing they would have to determine what they meant by "engage with" and "analyze" at a later date.

The instructors spent some time interacting with ChatGPT to see how they might use it constructively with their students. They tried having ChatGPT interpret quotes from course readings, but it was spectacularly bad at that. They tried asking ChatGPT faculty questions related to the course material, but the chatbot made stuff up, as it is prone to do. After a consultation with Poorvu Center staff, Tisa and Erika focused the assignment on the kinds of skills they wanted their students to develop, while taking advantage of ChatGPT's affordances. (Strengths is too strong of a word here.)

The final assignment asked students to enter one of eight different prompts into ChatGPT, each of which asked the chatbot to write something from a particular perspective. For example, here are a few of the prompts students had to choose from:

• "Write a treatise arguing that churches should no longer receive state funding from the perspective of an eighteenth-century Virginian."
• "Write a sermon from the perspective of a revolutionary Catholic priest arguing that God supports independence from Spain."
• "Write a sermon by Jarena Lee explaining why she as a woman would be permitted to preach."

Students were asked to analyze the ChatGPT response in a two-to-three-page paper using the following questions as a guide: "What did ChatGPT do well, and what did it do poorly? What's missing that you might expect to be there in light of the assigned reading? What types of historical narratives are being promoted or reified in this output?" Each of the prompts came along with a different secondary source for students to draw upon when writing their analysis.

As Tisa reported at the Yale event, ChatGPT's output was pretty bad but the student papers were very good. She shared a few examples of what ChatGPT wrote, and, while all the sermons and treatises were generally in the ballpark of what was requested, it was clear that large language models aren't always good at capturing historical context or the nuances of complex arguments. For instance, in its sermon supporting Mexican independence from Spain, ChatGPT quotes Jesus more than once but never makes an appeal to La Virgen de Guadalupe (Mary, mother of Jesus), who was a much larger figure in colonial Mexico. As one of Tisa's students wrote, "If ChatGPT had really done its homework, it would have known to put Guadalupe front and center."

Or take the ChatGPT-generated sermon in the style of Jarena Lee, the first woman preacher in the African Methodist Episcopal Church. While the sermon made the kinds of points that Lee might have made in favor of women preaching, it did so without any of Lee's signature preaching style. ChatGPT equivocated ("...the possibility that God may be calling...") where Lee would have spoken with assurance. Tisa and Erika's student wrote, "It provides a sermon that is too generic and could just as easily be offered by a white Southern Baptist woman in 2023."

I love this assignment because it leverages what ChatGPT is good for and what it's not. The chatbot can put together a coherent argument, it can emulate certain genres of writing (like a sermon), and it can provide each student with a different output for the same input. On the other hand, it can be woefully generic in its writing, and, of course, it doesn't actually know anything, it's just stringing words together. It's to be expected that those words won't account for actual details of historical records and contexts.

It also occurred to me as Tisa was presenting this assignment that her students wrote very insightful critiques of ChatGPT's output. They didn't pull any punches, and they wrote with a kind of authority. Had they been critiquing sermons and treatises written by their peers, I don't think they would have been as critical. And giving the students reliable secondary sources to draw on as they responded to ChatGPT's output put the students in a position of authority, knowing more about their topics than their "audience," the chatbot. I'm reminded of Paul Hanstedt's rhetorical triangles, and the advantages of empowering students to write with authority on a topic.

There are lots of ways that educators are figuring out how to use ChatGPT and similar generative AI tools in teaching and learning. I really like the idea of treating ChatGPT as a kind of audience for student thinking and writing, and I was glad to hear about Tisa Wenger and Erika Helgen's assignment this spring that did just that! Thanks to Tisa and Erika for allowing me to share their story here in the newsletter.

Thanks for reading!

If you found this newsletter useful, please forward it to a colleague who might like it! If you have thoughts on any of the topics above, please reach out via email, Twitter, Mastodon, or LinkedIn.
​

​Consider supporting Intentional Teaching through Patreon. For just $3 US per month, you get access to bonus podcast content, Patreon-only teaching resources (like a guide to facilitating class sessions on Zoom), an archive of past newsletters, and a community of intentional educators. Patreon supporters also get 20% off my book, Intentional Tech: Principles to Guide the Use of Educational Technology in College Teaching, when they order from West Virginia University Press.