Pros and cons of custom AI chatbots in teaching and learning


Teaching with AI Agents: A Conversation about Cogniti

I think the first custom AI chatbot I tried was one called “Are You a Witch?” designed by past podcast guest Marc Watkins. This chatbot would answer your questions like ChatGPT, but unlike ChatGPT it would only do so after accusing you of witchcraft (in the most caricatured way possible) and making you solve a riddle. That chatbot was kind of silly, but I soon heard about faculty and other instructors building chatbots to do all kinds of more useful things, like answering students questions about the syllabus or giving students feedback on drafts of their assignments. Just recently, for example, I blogged about a custom chatbot that graphic design professor Nikhil Ghodke built to help his students reflect on their work in advance of writing a designer’s statement.

If you have a paid ChatGPT account, you, too, can create a custom chatbot for your students. Dan Levy and Angela Pérez Albertos walk you through this process in Chapter 10 of their bookTeaching Effectively with ChatGPT, available free online. The problem with using ChatGPT like this in an educational setting is that, one, your students need to have ChatGPT accounts. Free ones will do, but that’s still an ask. A second problem is that you won’t get to see how your students interact with the chatbot you create, unless you ask them share screenshots or chat logs. Enter Cogniti, a tool for creating custom chatbots designed by faculty and staff at the University of Sydney. Cogniti makes it easy for instructors to design special-purpose AI chatbots, which the Cogniti team calls “agents,” then share those agents with students to use through a learning management system and—importantly—have some visibility into how students are actually using the agents.

I have a theory that in a few years, this kind of teaching-focused custom AI chatbot is going to be a standard tool available to instructors in higher education. I may be wrong about that, but if it turns out to be the case, it makes sense to start figuring out the affordances and limitations of these tools now. To that end, I reached out to Cogniti lead developer Danny Liu, professor of educational technologies at Sydney, to come on the podcast and give us an introduction to Cogniti. Danny recruited a couple of his Sydney colleagues--Isabelle Hesse, senior lecturer in English, and Matthew Clemson, senior lecturer of biochemistry--who have been experimenting with Cogniti to join us, and we had a great conversation about the current and potential roles of custom chatbots in teaching and learning.

You can listen to our conversation about Cogniti-powered AI agents here, or search for "Intentional Teaching" in your favorite podcast app.

Around the Web

This is the part of the newsletter where I link to things that I find interesting in the hopes that you do, too.

  • "The Reading Struggle Meets AI" - Beth McMurtrie's latest deep dive into teaching for the Chronicle of Higher Education acts as a sequel of sorts to a piece she wrote last year on the reading challenge in higher ed. She updates that conversation and weaves in some generative AI discussion as she does. I was particularly struck by reported comments from my Ole Miss colleague Liz Norell, who has been interviewing students about college reading. Liz reports that the biggest reasons students don't "do the reading" is that (1) the reading is rarely talked about in class and (2) they don't understand what they are supposed to be reading for. Those seem like challenges we can address through our teaching!
  • "Five Steps to Defend Higher Ed" - On a recent Take It or Leave It panel episode of the podcast, my guests and I discussed a Chronicle piece by Kevin McClure that called out higher education leadership for not doing more to respond to the threats posed to higher ed by the current U.S. presidential administration. In our panel discussion, we talked a lot about what faculty and staff can do in the current moment individually and collectively. That's the focus of McClure's follow-up piece that I'm recommending here. If you'd like to operationalize a little hope into your professional work, McClure has concrete suggestions from a variety of thoughtful colleagues.
  • "The Expertise Paradox" - This piece by Katie Harbath came across my Bluesky feed last week. She starts by noting that Elon Musk has said that cutting U.S. government spending was "more of an uphill climb than I anticipated," underselling the work of his "DOGE" unit that can be reasonably be seen as a debacle for the federal government. Harbath goes on to write about the "expertise paradox," the usually incorrect assumption that deep expertise in one area translates into competence in another area. The essay is a fascinating exploration of expertise and the role it plays in the U.S. today, and it has me wanting to write more about teaching and learning through the lens of expertise.

Thanks for reading!

If you found this newsletter useful, please forward it to a colleague who might like it! That's one of the best ways you can support the work I'm doing here at Intentional Teaching.

Or consider subscribing to the Intentional Teaching podcast. For just $3 US per month, you can help defray production costs for the podcast and newsletter and you get access to subscriber-only podcast bonus episodes.

Intentional Teaching with Derek Bruff

Welcome to the Intentional Teaching newsletter! I'm Derek Bruff, educator and author. The name of this newsletter is a reminder that we should be intentional in how we teach, but also in how we develop as teachers over time. I hope this newsletter will be a valuable part of your professional development as an educator.

Read more from Intentional Teaching with Derek Bruff

High Structure Course Design Justin Shaffer has a new book out on high structure course design! I met Justin a few years ago through a Macmillan Learning webinar on teaching with classroom response systems. I learned that not only did he use that particular technology very effectively in his teaching, but he also had a wealth of experience in active learning and course design more generally. When I wanted to put together a podcast episode on "studio" approaches to biology (in which lab and...

Defending Higher Education with Kevin McClure If you've listened to the recent "Take It or Leave It" episodes of the Intentional Teaching podcast, you'll be familiar with Kevin McClure's essay "Higher Ed Is Adrift." In the essay, Kevin outlines some of the many attacks the current U.S. presidential administration is leveraging against higher ed, and he notes that many faculty and staff are finding their institutional leaders' responses lacking. We talked about the essay on the "Take It or...

HigherEd PodCon banner featuring logos of dozens of higher ed podcasts

Podcasting in Higher Ed Last weekend I had the opportunity to attend HigherEd PodCon, the first conference devoted to podcasting across higher ed. Thanks to UPCEA, which sponsors my Intentional Teaching podcast, for sending me to Chicago for this very engaging conference! On my blog this week I shared some highlights from the conference in a post I called "Higher Ed Podcasting Is Having a Moment." If you're interested in higher ed podcasting, either as an avid listener or as a (potential)...