Is it worth paying for GenAI?

With nine out of 10 topics already planned in a law module I was updating, I needed to select a topic for the last session as well as edit the content for another session to ensure it was detailed enough but not too complicated. I also needed to formulate hypothetical scenarios for workshops.

So I wondered if artificial intelligence (AI) could help with these tasks and speed up the process of updating my constitutional and administrative law module, as well as check my work afterwards. I wasn’t sure which generative AI platform to use, though, and whether it was worth paying for a subscription. I asked some experts at a workshop on Gen AI in the legal classroom, and when they didn’t have an answer for me I decided to find one.

The results were…surprising. 

How does generative AI work?

The first question, is how does generative AI (GenAI) work? At a basic level, it is just an advanced version of predictive text. If you’re interested in the science behind it, this Financial Times explainer is less boring than it sounds, I promise.

What GenAI models did I test with which prompts?

I tested three free and three paid-for Gen AI models:

For my prompts, I used the “syllabus co-creator” and “lesson crafter” templates from More Useful Things’ instructor prompts. Each platform received the same basic information about my module, students and planned sessions. 

This is how many prompts it took each platform to arrive at a useable first draft of the syllabus:

Image

Test 1: ChatGPT4 and ChatGPT4o

The prompting process for ChatGPT4 was straightforward and intuitive and the outputs accurate. Its activity suggestions were sensible, appropriate and varied, and it was more detailed overall than Claude or Gemini. However, the topic suggestions didn’t fit with the rest of the module and weren’t practical enough in my view. I also found them too niche, and less creative and detailed than some of the other models I tested.

Should you pay for ChatGPT 4o?

With the free version’s cap on prompts meaning you’ll temporarily be shut out of the platform, I did get more done in less time with the 4o version. I also got access to the image-creating GenAI DALL-E, which enabled me to make pictures for my lecture slides. However, otherwise, the output on 4o was almost indistinguishable from 4.

So, while the longer conversations and image creation were useful, I don’t think they’re worth the price tag.

Test 2: Claude 3.5 Sonnet and Sonnet Pro

Claude seems more conversational than ChatGPT, making for a more natural-feeling prompt process, which makes it a good platform for honing your prompting skills. This model also gave me better topic suggestions, one of which will end up in the final version of my module.

• Resource collection: AI transformers like ChatGPT are here, so what next?
• Deepfakes are coming for higher education. What can we do to prepare?
• Mastering generative AI: crafting reusable prompts for effective learning design

However, despite more data input, the results were less detailed on Claude than on other models I tested. The workshop activity suggestions were not practical enough as well as being repetitive proposals such as Q&As, presentations and quizzes. Additionally, the output formatting was my least favourite – I found it difficult to read from an accessibility standpoint.

Should you pay for Claude Pro?

My short answer is: no. 

Again, you’re limited on prompts with the free version, but there wasn’t a marked difference between the free and subscription versions. In fact, it took longer to prompt the pro version.

Test 3: Gemini and Gemini Advanced

The free model was – to put it bluntly – terrible. It struggled to deal with more than one set of instructions at a time, stalled and repeatedly misinterpreted my prompts. I had to spend a long time refining and restating myself to get to something even remotely useful.

In the end, I didn’t get as far with this model as the others because I got so frustrated that I gave up.

Should you pay for Gemini Advanced?

Given how bad the free version was, I nearly didn’t test the advanced version. However – in the name of science – I gave it a go, and I was glad I did because, in a shocking turn of events, it ended up being my favourite.

Gemini Advanced was the most perceptive and intuitive platform, making for first drafts that needed less refinement and were more practical. The outputs were also more detailed overall and the lesson plans were creative and included a good range of suggested activities. The topic suggestion was the same one as Claude’s, but I arrived at it more quickly, and it gave me plenty of detail without prompting.

This model clearly had the biggest difference between its free and paid versions and is the only one I think is really worth paying for.

So should you take out a GenAI subscription?

GenAI significantly sped up the process of updating my module. My unscientific estimate is that – over the course of the whole process – it saved me several weeks’ worth of work. This is mostly because it gave me starting points for subject content rather than me having to do extensive research of the possibilities. Instead, I could zero straight in on specific topics. So, for time saving alone, I would recommend using it. I also think that my syllabus and lesson plans are better for having made use of this technology.

As for subscriptions, Gemini Advanced was the clear winner for me, and it is tempting to pay for it. I will be taking advantage of my free trial, that’s for sure. 

So have I convinced you to use GenAI for module updates? Have I sold you a subscription? If so, do you think I can get a commission…serious question.

Sophia De Arez Cintra is programme director for postgraduate diploma and first year MSc in law and professional practice and a senior lecturer in professional legal education in the Dickson Poon School of Law at King’s College London.

If you would like advice and insight from academics and university staff delivered direct to your inbox each week, sign up for the Campus newsletter.