Use of AI tools in Academics

Sean van der Merwe

Topics

How I talk to students about AI in 2024
How I use AI for myself

AI Chatbots are a great starting point for a lot of tasks, but not a good endpoint

My thoughts

AI tools are good at expanding and contracting text.

They summarise complicated topics in plain language
They explain simple things clearly
So they are like Wikipedia page generators
- They make something like a Wikipedia page on the fly based on what you ask
BUT without any accuracy checks!

Fun example

Two weeks ago I asked Google Bard for memorable moments from the last 80 years, both international and local, and among the many good moments provided was this one:

2023: Appointment of the first female Chief Justice of South Africa: In July 2023, Justice Raymond Zondo was sworn in as the first female Chief Justice of South Africa. This historic appointment marked a significant milestone in the country’s journey towards gender equality and legal reform.

How I talk to students about AI in 2024

My students

My students will use AI in their work

I know this because they already did so last year
- A portion of my class is already employed
At first companies used ChatGPT
- Asking it to explain concepts to them
- To generate code and to check/correct code
But they realised that ChatGPT doesn’t respect privacy
Private AI is being implemented in companies
- Used to turn meeting transcripts into meeting minutes
- The meeting transcripts themselves coming from voice-to-text AI systems

My assessments

My students used AI in ALL assessments in 2023
- Tests, assignments, discussions, essays, presentations, etc.
Usage was not equal
- Some students refused to try it even when it would have clearly helped them
- Most students gave it a try but didn’t benefit much
- Some students jumped on the opportunity to abuse it
My course builds on most of what they learned in the 3 previous years
- If a student had a 70% average, and retain half, then they have a big gap
- Students try to use internet and AI to fill the gap

Adapting and preparing

Last year I was not ready, this year I will be
- (probably not, but I’m trying to be positive)
When students can use AI to answer, the rubrics/memos/mark allocations must adapt
The first thing that must change though is the course information + study guide

Course information snippets

Most assignments will require you to search the internet for tips or resources. ❗Cite or link those resources explicitly. Integrate the citations and links neatly into your writing so that it flows and connects clearly. Heavy assessment weight will be given to this process.

❗AI tools may be used in this course, but their use must be delineated and acknowledged clearly. For example, consider placing prompts used in an appendix and referring to that appendix in the sections that are inspired by AI responses. Under no circumstances may you copy-paste AI responses into your submissions – summarise the responses in your own words and retype code in your own style.

Citing ALL sources

I’m going to push my students to acknowledge every source:

Obviously published papers and textbooks are ideal sources, but that’s often the last place students go,
They go to wikis, blogs, forums like Stack-Overflow and Reddit,
They go to AI chatbots,
They go to each other,
They refer to lecturer notes and lecturer, but far too seldom

How do I get students to take proper notes in class and then cite that in their assignments?

Solutions I’m going to push this year

Standard ChatGPT language does not normally include real references
- So increase the mark allocation for providing real references
- Give serious credit for integrating real references into the flow of the text

The above is a challenge in big classes and some types of assessments, but

ChatGPT responses tend to be superficial
- So raise the cognitive level of required responses
- Or change the focus or style of questions to require depth

How I use AI for myself

Uses for lecturers

These tools are fantastic for creating assessments
- Use them to generate word problems, they are great at that!
  - These tools are bad at math, statistics, and probability, as those require logic and precision
They are good at translating though
- That includes translating into programming code 😀
  - But note that code it generates almost always has errors
Can draft guides and exercises

Guide query

I asked Bing AI the following:

I am teaching statistics to a group of students in the first year of university. I would like to introduce them to Microsoft Excel. Specifically, I want to guide them through performing some simple statistical tasks using Excel functions. Help me draw up a guide for the students describing how to do some statistical tasks. Ideally, the first task would be to generate a tall block of random numbers with short headings upon which they can then apply other statistical functions.

It gave me the first 5 steps in detail.
I fixed one formula error and the changed the information after Step 5.
I then added additional tasks myself in order to expand the exercise.
- I did ask Bing AI for assistance with the additions but was not impressed, so mostly did those steps myself.

Guide detail

More lecturer uses

They are great at answering basic student questions about the work
- Copy the question into Bing AI, copy the answer back to the student
- Example: student asks what a line of code means, you paste the line of code into the chat bot and ask it to explain it
Use it to draft a difficult email
- Or a motivating Blackboard post to get students excited about a topic

How to adapt

Step 1: Use it yourself

Only by using AI yourself will you learn:
- What it can do
- What it can’t do
- How it presents things
- How to identify it

In my planning for this year I asked Bard:

I’m teaching a course on Bayesian statistics and want to promote deeper learning of key concepts such types of prior distributions. Please give me a specific example of how I can adapt an assessment or task to push my students towards deep engagement with the material and longer term acquisition of the outcomes.

Step 2: Spotting issues

The most obvious is when a student just does a copy-paste:
- Questions in their answers,
- or words that are a clear response to a prompt,
  - Like “Sure! Here is an explanation of…”, or “Certainly. The …”
- or end with a leading statement
Turn-it-in has an AI detection tool that tries to pick up AI use.
- If it says 100% then you know there’s a problem
- If it says 0% then it’s probably all good
- Anything in between means the student probably used a little bit of AI text but didn’t rely on it entirely
  - This is sometimes fine, and sometimes not