Presentation of Antonella Poce, for EDEN's NAP webinar on 'Mentors and enablers' - 2 June 2021, 17:00 CEST More info: https://www.eden-online.org/eden_conference/facilitating-and-mentoring-students-work-to-reach-expected-competencies/

1. Linking Higher Education and the
workplace through Critical Thinking
Enhancement: a Pilot experience in
the Education context
Antonella Poce1, Maria Rosaria Re2, Carlo De Medio3, Mara
Valente4, Alessandra Norgini5
1 University of Modena and Reggio Emilia, Dept. of Education and Humanities, Reggio
Emilia, Italy
antonella.poce@unimore.it
2,3,4,5 University Roma Tre, Dept. of Education, Rome, Italy

2. Critical Thinking
CT skills are more and more
considered pivotal for human and
social progress in terms of
innovation, economic and
knowledge growth by educational
policy (World Economic Forum,
2016; Scott, 2015).
Also, CT provides students with
tools to be autonomous thinkers,
active citizens (Davies, 2015) and
critical users of digital
technologies (Poce, 2012).

3. The pilot experience carried out
by CDM research group
The Centre for Museum Studies (CDM) research group
developed in the academic year 2020-2021 a series of
HEI learning paths aimed at promoting CT through
specific activities carried out within workplace context.
In the pilot experience students are continuously
solicitated in analyzing, interpreting, evaluating, and
finding solutions through specific work-based activities.

4. Research questions
• How students CT indicators levels change in a university
course designed to support students’ CT levels?
• What is the level of CT perceived by the students at the
end of the activities?
• What other transversal competences are solicited by a
university course foreseeing work-based activities?
• Did the online activities, including assignments and
meetings with educational professionals, improve the
levels of critical thinking in the participating students?

5. Participants
125 students from two master's degree programs at
University Roma Tre participated in 2 different modules:
• 7 students from the Master’s degree course in Pedagogical
sciences attending the 2nd year course in “Experimentalism,
Museum and Reading” from October to December 2020 (12
ECTS). Students had the choice to work individually or in
groups of up to three participants.
• 118 students from the Master’s degree course in Primary
school education attending the 1st year course in “Research
Methodology in Education” from October to November
2020 (4 ECTS). Students worked collaboratively in 17 groups
of 6-10 members.

6. Learning activities
Specific online meetings with stakeholders were hold
and experiences based on the following teaching
methodologies were discussed: problem solving,
oral dissertation, digital storytelling, analysis and
critical reflection.

7. Meetings and activities organised for
Experimentalism, Museum and Reading module
MEETING RELATED ACTIVITY
Creative approaches to explore the museum
collection
Designing a museum exhibition for a specific
community
Cultural heritage and museum in the
Flanders: a grassroots approach
Community and public engagement
programme through cultural heritage + oral
dissertation
An inquiry approach to museum education Inquiry into restoration of Palmyra site and
analysis of scientific articles + oral
dissertation
Building relationship through heritage Case study analysis + oral dissertation

8. Meetings and activities organised for
Research Methodology in Education module
MEETING RELATED ACTIVITY
The Inclusive Memory Project: design,
implementation, administration and results.
Adaptation of IM questionnaire to a different
categories of respondents + oral dissertation
Music of oral tradition as intangible Cultural
Heritage of humanity: an opportunity for
inclusive teaching.
Reading grid for empirical research
production + oral dissertation
Narrating the museum: the E-trouria VR app
and digital storytelling for heritage
education.
Collaborative storytelling for heritage
education realization + oral dissertation
Empirical studies of museum education.
Promoting social inclusion of different
categories of museum users.
Recognising strategies and stages of
educational research + oral dissertation

9. Evaluation phases
During the two modules, the evaluation phases were divided as
follows:
• Evaluation of the activities carried out by the students or groups
of students participating in the pilot experience, aimed at
monitoring the solicitation of CT skills during the modules;
• Final summative evaluation through a questionnaire, aimed at
assessing the perception of the levels of transverse competences
reached by the students at the end of the pilot experience and
the quality of the activities and meetings organised.

10. Evaluation tools
Two different evaluation tools were used for the two phases of
monitoring and evaluation of the pilot experience:
• Critical thinking evaluation rubric, already validated in previous
publications (Poce, 2017), and composed by six macro-indicators
on a scale from 1 to 5: use of the language, argumentation,
relevance, importance, critical evaluation and novelty.
• Questionnaire for self-assessment of transverse and professional
skills and evaluation of the quality of the activities and online
meetings carried out in the pilot phase. The questionnaire was
created ad hoc for the experience, starting from indicators
already validated and used in Poce, Agrusti and Re (2015).

11. CT evaluation rubric (Poce, 2017)

12. Questionnaire for self-assessment of
transverse and professional skills
(Poce, Agrusti, Re, 2015)
SECTION TYPES OF QUESTION
Evaluation of the course Likert scale
Evaluation of the online meeting with
stakeholders
Likert scale
Professional skills self-assessment Likert scale
Transverse skills self-assessment Multiple choice, open-ended and Likert
scale

13. Evaluation data on the quality of
online meetings
56% of the participating students assign the
maximum score of 5 to the didactic tools used as
support during the online meetings, such as videos,
pdfs and slides.
The language used by the stakeholders/lecturers is
defined as extremely clear for 45.6% of the students;
22.4% of the participants assign a score of 5 and
46.4% a score of 4 to the quality of the course in
terms of “Exhaustiveness of content”.

14. Evaluation data on the quality of the
online meeting

15. Self assessment of the most
stimulated transversal skills
Collaboration, Creativity and CT were rated as the
most supported skills, having been selected
respectively by 91%, 67% and 63% of students.
The skills least selected by the students were
Problem solving (30%) and Innovation (21%), even if
some work-based activities were designed to foster
such skills.

16. Self assessment of the most
stimulated transversal skills

17. Evaluation of the CT skills
The average score obtained by the 7 students attending
the 2nd year course in “Experimentalism, Museum and
Reading” changes from 21 points in the first activity to
22 points in the last activity (first activity ds =2.98; last
activity ds =2.76). The average score in outgoing activity
is always higher than the average score in incoming one,
except for the indicators use of language and novelty.
The Wilcoxon signed rank test shows that the
improvement in participants’ importance levels from the
first to the last activity is statistically significant
(p=0.048).

18. CT score distribution in the first
and in the last activity
The data analysis shows that the average score
obtained by the 118 students attending the 1st year
course in “Research Methodology in Education”
increased from 18.7 points (ds=2.4) in the first
activity to 22.5 points (ds=2.7) in the last activity. The
average score increases in all the indicators:
language use, argumentation, relevance, importance,
critical evaluation, and novelty.

19. CT score distribution in the first
and in the last activity

20. Statistically significant results
The Wilcoxon signed rank test was used in order to analyse whether the differences
in the scores assigned to the indicators between the first and last activity were
statistically significant:
the results show a significance p<0.01 for all six indicators analysed.

21. Conclusion
The pilot experience has large room for
improvement, but it presents some important
indications for the implementation of online
learning paths aimed at enhancing CT skills with the
support of external stakeholders and through
continuous instructional learning design.