The best way to teach students to think about AI? Make them argue with it

Universities have largely responded to generative artificial intelligence in one of two ways: ban it or permit it with a disclaimer about responsible use. Neither is sufficient because both treat AI as an integrity problem rather than a pedagogical one.

The more consequential question is whether students can evaluate what AI gives them. Workplaces need graduates who can interrogate a model’s reasoning, identify where it is confidently wrong and take responsibility for the decision that follows.

These capabilities do not develop by accident or through AI disclosure policies. In my business information systems (BIS) course, I designed two assessments built around this principle. The first is a live debate in which students are assigned positions on how AI is reshaping work, with motions covering individual, organisational and future-of-work impact.

• Five tips for using AI in university assessment
• Banning AI won’t make students care about learning
• Campus Talks: What does ‘AI across the curriculum’ look like in practice?

The second task follows directly from the first. Students take one position they defended in the debate, ask an AI tool to produce a counterargument, and then evaluate it in writing, identifying which aspects are well reasoned, where it overreaches and where it fails to engage with the disciplinary evidence. They must explain why they agree or disagree, using course concepts. Learning emerges through the interaction itself, as students test their assumptions, reconsider their reasoning and decide which claims withstand scrutiny. The task does not ask whether the AI was right; it asks whether the student can tell.

Design principles for developing judgement of AI outputs

Here are three key design principles to enhance learning that happens through interaction with AI.

1. Create a stake in the question

Students need a position before they engage AI as a critical tool. A common failure in AI literacy pedagogy is asking students to evaluate an AI output before they have developed any stake in the question. Without a defended position, students have nothing to test the AI against.

The way to create that stake is through tasks that require public intellectual commitment: debates and oral defence. In a postgraduate BIS course, students are assigned to take positions on how AI is reshaping organisations: whether it will eliminate technical data skills, whether it should be central to business decision-making, how it is redefining individual roles. They argue live, without notes, while fielding questions from peers.

When they sit down with an AI tool afterward, they have a position already defended.

2. Be adversarial

The encounter with AI should be adversarial, not collaborative. The standard use case – ask AI to help you write, refine or summarise – positions the model as assistant and the student as passive recipient. Instead, encourage students to ask the AI to produce the strongest possible counterargument to their defended position, then evaluate its strength and weaknesses, where it is overconfident or poorly evidenced, where it fails to engage with the disciplinary reasoning they brought to the debate. They must explain why they agree or disagree and connect that judgement to course concepts and evidence.

3. Emphasise reflection as metacognitive act

Reflection must be structured around the quality of judgement, not just the fact that reflection occurred. A student who documents every AI interaction but cannot explain why one response was better reasoned than another does not yet have the capability we are trying to build. The highest marks should not go to students who simply note that they disagreed with AI but to those who can reconstruct their reasoning, acknowledge when the AI made a fair point, and explain why that was sufficient/insufficient to move them.

This is not just a reflective exercise. It is also a professional skill. The bottleneck in AI-supported work is not generation but evaluation: identifying where a model has been confidently wrong and taking responsibility for the decision that follows.

AI capabilities that employers need from graduates

Student reflections and assessment performance suggest the development of three capabilities:

1. Evidence-based argumentation: the ability to construct a position, anticipate where it is vulnerable and defend it against challenge. Students described moving beyond surface familiarity with AI concepts to stress-testing their claims against contrary evidence, a demand created by the public accountability of live debate.
2. Metacognitive self-assessment: the capacity to examine not just what you think but how you arrived at it, and where your reasoning holds or breaks down. Students identified this as the harder skill to develop because it required them to recognise gaps in their reasoning rather than rationalising them away.
3. Calibrated judgement about AI itself: understanding not just what AI can do but where it falls short, overreaches or requires disciplinary correction. Students who entered the assessment with relatively uncritical assumptions about AI capability left with a more grounded and specific sense of where human judgement remains indispensable.

Together, these capabilities – argumentation, metacognition and AI discernment – align closely with the skills employers increasingly need from graduates working with AI. This is what structured adversarial engagement produces when assessment is designed around judgement rather than output. Universities that get this right will graduate students with a scarce capacity: the ability to know when to trust AI, when to override it and why their own reasoning remains the thing that cannot be delegated.

We shouldn’t keep treating AI as an integrity problem when the real challenge is teaching judgement. Until assessment asks students to challenge AI, weigh its claims and take responsibility for deciding when not to trust it, we will keep graduating fluent users rather than accountable thinkers.

Reihaneh Bidar is a lecturer in business information systems in the UQ Business School at the University of Queensland.

If you would like advice and insight from academics and university staff delivered direct to your inbox each week, sign up for the Campus newsletter.