Introduction to ChatGPT

1. ChatGPT: What is it, and
how to detect it!
Damian Gordon

3. What is ChatGPT?

4. What is ChatGPT?
ChatGPT is a ChatBot that uses a GPT
(Generative Pre-trained Transformer),
one of many LLMs, and is more
commonly called a
Transformer Algorithm

5. What is a transformer algorithm?

6. What is a transformer algorithm?
… So, a transformer algorithm:
- Takes in millions of chats (off Facebook, etc.)
- Looks for patterns in the text
- Looks for words the commonly co-occur
- Tries to extract a set of rules about grammar,
turn-taking, and answering

7. What is ChatGPT?
What are some of
the issues?

8. It might not be able to log in
when you want to:

9. And it still doesn’t always work
even if you can log in:

10. It’s only good up to
September 2021:

11. It doesn’t seem to understand the
plotpoint of a famous 1980 movie:

12. Maths is
hard:
It can’t do
maths …
The first
time you ask
it …
But it
learns!

13. It’s not as smart as it seems to
be:

14. In this case ChatGPT doesn’t
understand the words “last name”:

15. John Searle
• American philosopher
widely noted for
contributions to the
philosophy of language,
philosophy of mind, and
social philosophy
• His notable concepts include
the "Chinese room"
argument against "strong"
artificial intelligence.

17. Rules and
associations that
ChatGPT has
extracted from the
millions of chats.

18. Rules and
associations that
ChatGPT has
extracted from the
millions of chats.
But it really doesn’t
understand
anything, it’s just
pattern matching.

19. So what do we do?
DETECTION TOOLS!

20. There are a number of ChatGPT
detection tools:
1. GPT-ZERO
2. OpenAI GPT2 Output Detector
3. Hello-SimpleAI ChatGPT Detector
4. Contentatscale AI Content Detector
5. Writers AI Content Detector

21. There are a number of ChatGPT
detection tools:
GPT-ZERO
https://gptzero.me/

22. There are a number of ChatGPT
detection tools:

23. There are a number of ChatGPT
detection tools:

24. There are a number of ChatGPT
detection tools:

25. And they are really highly
accurate:
GPT-ZERO
10 Texts by ChatGPT – 100% accurate
10 Texts by Human – 100% accurate
OpenAI GPT2 Output Detector
25 Texts by ChatGPT – 100% accurate
25 Texts by Human – 95% accurate

26. However there are a number of
very simple ways to fool it:
1. Ask ChatGPT to rewrite its own
answers in a less detectable way.
2. Input the answer it into a “Writing
Assistant” tool and change it up.
3. Replace “the” to “teh” and replace all
the “.” with “,”.

27. And they become a lot less
accurate:
GPT-ZERO
4 Texts by ChatGPT – 25% accurate
4 Texts by Human – 75% accurate
OpenAI GPT2 Output Detector
10 Texts by ChatGPT – 30% accurate
7 Texts by Human – 72% accurate

28. However there are a number of
very simple ways to fool it:
1. Ask ChatGPT to rewrite its own
answers in a less detectable way.
2. Input the answer it into a “Writing
Assistant” tool and change it up.
3. Replace “the” to “teh” and replace all
the “.” with “,”.

29. However there are a number of
very simple ways to fool it:
1. Ask ChatGPT to rewrite its own
answers in a less detectable way.
2. Input the answer it into a “Writing
Assistant” tool and change it up.
3. Replace “the” to “the” and replace all
the “.” with “,”.

30. However there are a number of
very simple ways to fool it:
This is using AI tools
to solve AI problems

31. However there are a number of
very simple ways to fool it:
So What Do We Do?

32. So I guess the better way is to avoid
essay-type assignments (1/5)
• Include questions that require students to
demonstrate their judgement.
• Ask about current events, anything after
2021 for ChatGPT.
• Ask them to answer questions "in the
context of the discussions we have had in
class" or "in the context of my lecture
notes"

33. So I guess the better way is to avoid
essay-type assignments (2/5)
• Ask them to create an Infographic instead
of text.
• Ask them to work in groups, and track
their discussions on a tool like
FeedbackFruits
• Ask them to create an initial draft "an early
deposit" before their final submission
• Ask them something creative.

34. So I guess the better way is to avoid
essay-type assignments (3/5)
• Ask students to submit an assignment,
and following that ask them to add at least
10 reflective comments on their text, on
why they presented the work in the way
that they did.
• Ask the students to go out into the field,
and do some observations; and write
reflections about their experience.

35. So I guess the better way is to avoid
essay-type assignments (4/5)
• Personalise the assignment in some way, it
could be by asking the students to bring
their own experience into the assignments,
or it could be to ask them to use their
student number as a basecode for
calculation or computation.
• Ask them to create a video answer, it'll be
obvious if they know what they are talking
about.

36. So I guess the better way is to avoid
essay-type assignments (5/5)
• Which one is most normalised?
• [Is it Figure 1, Figure 2, Figure 3, or Figure
4?]

37. Chat-HUMANS:
• What do people think?

38. Contact:
• If there is any way I can help,
please let me know:
• Damian.X.Gordon@TUDublin.ie