A framework for critical thinking about GenAI
In a previous post, I mentioned that boosting critical thinking about GenAI output is complicated by the facts that critical thinking is a broad concept and the use of GenAI is quite varied. However, we can bring some method to the madness by adopting two heuristics: the five-step process model of thinking and Brachman's ontology for GenAI use.
Process model of thinking
There are many definitions of critical thinking, but all of them at least agree that it is a mode of thinking which meets some specific norms (usually to optimise for truth, but sometimes for beauty or justice, too). That means that to discuss critical thinking, it makes sense to have some scheme of thinking in general.
One scheme I find useful for this purpose is a process model showing five discrete steps.
The process starts with the realisation that there's something to resolve, whether that's some theoretical issue in academic literature, customer dissatisfaction about some product, or something else entirely that could be considered a problem. Then, after seeing the problem, one needs to gather relevant information. Not just that, but the information needs to be interpreted, evaluated, integrated with prior beliefs and understood: it has to make sense to the thinker. After this, convictions can be inferred, conclusions can be drawn, leading to new beliefs. These beliefs can then inform subsequent decisions or action selection.
Critical thinking is about going through these steps and doing them well. Broadly speaking, it's the problem from the first step that defines what it is exactly that should be done well. If you're working character design, you will be employing different standards than if you're screening drugs for therapeutic potential. But in both cases you can work rationally, by setting standards and trying to meet them. And in both cases you can work critically, evaluating whether your thought process itself is geared towards optimising the outcome.
This blog is mostly concerned with optimising epistemics. That means I presume thinkers are motivated to developing accurate beliefs about the world and wish to align their actions with these beliefs. Critical thinking is then the set of mental habits they should have to optimise the odds of obtaining such beliefs and actions. The five-step process model adds to this by positing steps that require their own questions, considerations and carefulness. In any context, one can break down the thinking process into these discrete steps and identify how to do them well.
Brachman's ontology
Since GenAI appears to be a multi-purpose problem-solving tool, it might seem as if even the five-step process model is not going to help create much order. However, despite its broad range of affordances, it is possible to categorise the actual use cases of GenAI, as Brachman et al. did on the basis of survey data (Brachman et al., 2024). They identified nine different use cases, grouped into three broad categories.
| Category | Subcategory | Description |
|---|---|---|
| Creation | Artefact Idea |
Generate a new artefact to be used directly or with some modification Generate an idea, to be used indirectly |
| Information | Search Learn Summarise Analyse |
Seek a fact or piece of information Learn about a new topic more broadly Generate a shorter version of a piece of content that describes the important elements Discover a new insight about information or data |
| Advice | Improve Guidance Validation |
Generate a better version Get guidance about how to make a decision Check whether an artefact satisfies a set of rules or constraints |
Using this as a basis, it becomes possible to look at what critical thinking would mean in the context of GenAI. For each subcategory in Brachman's ontology, we can look at what the right questions and considerations are for each step in the five-step process model, to arrive at a generalised view of critical thinking about GenAI. This generalised view can then in turn be used to look at specific learning activities or assessment forms in education.
In the next three posts, I will explore what happens if we apply the five-step process model to each category in Brachman's ontology.
References
Brachman, M., El-Ashry, A., Dugan, C., & Geyer, W. (2024, May). How knowledge workers use and want to use LLMs in an enterprise context. In _Extended Abstracts of the CHI Conference on Human Factors in Computing Systems_ (pp. 1-8).
Member discussion