PISA 2015 Results. Collaborative Problem Solving. Volume V.


Report link

Information on Volume 5.

Collaborative problem solving was an additional optional assessment introduced for PISA 2015. A large number of countries adopted it as an additional assessment.

PISA 2015 defines collaborative problem solving as the capacity of an individual to effectively engage in a process whereby two of more agents attempt to solve a problem by sharing the understanding and effort required to solve a problem by sharing the understanding and effort required to come to a solution and pooling their knowledge, skills and efforts to reach that solution. In the PISA assessment, one agent is the student whose performance is being evaluated; all other agents are computerised simulations.(P32)

Fifty seven countries took part in this additional assessment. Data from 51 countries was analysed for PISA 2015 V.

Box V.2.1 (P48) describes the University of Luxembourg’s evaluation of whether a computerised assessment can accurately measure students’ ability to collaborate with other humans and whether computer agents can faithfully replicate humans. The University compared human/computer collaborators in the assessments with human/human collaborators in the same situation. It found that this computer based assessment was a moderately good predictor of (students’) performance in the face to face collaboration units with another human.

Figure V.1.1 (P41) contains a table of countries’ performance in collaborative problem solving and attitudes towards collaboration.

Figure V.3.3. (P 70) groups countries together whose score in the assessment was not statistically different from each other.

Initial commentary

Teaching and encouraging students to collaborate with each other is at the heart of the lives of very many schools. An assessment which contradicts the punitive top down tests on individual students imposed by some governments on schools should be welcome. Every teacher know that students thrive in their schools, not in isolation but in the company of their peers. Students learn from each other both inside and outside their classrooms. Such an assessment should also help rebut the use of individual student test results to evaluate schools and teachers.

However, the decision by the OECD to substitute a human student in a collaborative situation with a computer agent is a cause for concern. It may be that, as the OECD says, in an increasingly digitalised world, people are increasingly having to collaborate with computer agents but the very basis of collaboration involves human to human relationships. Human collaborative relationships are the glue of civilized societies.
There is probably only limited value to be gathered from the Computer Problem Solving assessment in PISA. The assessments are confined to a small number of human/agent modules in which the computer agent responses are entirely predictable; something which the University of Luxembourg recognises itself as problematic when it says that; human agents are unpredictable.

Any claims, therefore, which PISA might make about the levels of collaborative problem solving in PISA have to be qualified by the extremely limited nature of the assessments and the fact that they did not involve human to human interaction.
A number of results are interesting for policy purposes.

PISA finds that girls greatly out-perform boys in collaborative problem solving. Boys prefer team work whereas girls tend to value relationships more.

Attitudes towards collaboration are more positive when students attend more physical education classes per week. The use of video games undermines collaborative problem solving.
Students who reported not being threatened by other students scored much higher than those who reported being threatened.
There is no significant difference in collaboration between advantaged and disadvantaged students or between immigrant and non-immigrant students.

The next meeting of the Trade Union Advisory Committee Working Group on Education and Skills on the 15th and 16th January in Paris will have the report on its agenda.