Ce diaporama a bien été signalé.
Le téléchargement de votre SlideShare est en cours. ×

Methodology for the evaluation/assessment of competences

Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
This project is funded with support from the European Commission. This document reflects the views only
of the author, and...
This project is funded with support from the European Commission. This document reflects the views only
of the author, and...
This project is funded with support from the European Commission. This document reflects the views only
of the author, and...
Publicité
Publicité
Chargement dans…3
×

Consultez-les par la suite

1 sur 11 Publicité
Publicité

Plus De Contenu Connexe

Similaire à Methodology for the evaluation/assessment of competences (20)

Publicité

Plus récents (20)

Methodology for the evaluation/assessment of competences

  1. 1. This project is funded with support from the European Commission. This document reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein. Project Nº 518536-LLP-1-2011-1-ES-GRUNDTVIG-GMP “KEYROMA: Develop Key Competences in Social Skills for Roma Women to increase their participation in the service sector” Project I.I.T. GUIDELINES FOR ASSESSMENT OF COMPETENCIES AND TRAINING NEEDS ANALYSIS This project is funded with support from the European Commission. This document reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein. Project Nº 518536-LLP-1-2011-1-ES-GRUNDTVIG-GMP
  2. 2. This project is funded with support from the European Commission. This document reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein. Project Nº 518536-LLP-1-2011-1-ES-GRUNDTVIG-GMP Guidelines for assessment of competencies and training needs analysis This document aims at providing guidance and suggestions for the local teams involved in the Key Roma project in order to perform the assessment of the competencies of the Roma women member of the Local Learning Groups and the analysis of the training needs of the group members. It is assumed that a Local Learning Group consisting of Roma women has been set-up based on voluntary participation in each project location. The present guidelines need to be adapted and implemented with each Local Learning Group considering the specific situation and the profile of group members, putting on the first place the respect for the group members and the focus on empowering them to engage in a positive change process, beneficial for themselves, their families, their community and the overall society. These guidelines are meant to support partners in producing a report on competencies of group members. As specified in the project, the Local Learning Group is expected to evaluate and assess the various competencies, with a focus on the transversal social competencies already managed by Roma women participating into the project, acquired through formal, non-formal and informal education. On this basis, the real learning needs of the groups concerning social and communication competencies will be identifies, concentrating on those transversal competencies required for labour integration into the service sector. This process will make women aware of the potential that they have, increasing self-esteem and making them feeling responsible of their personal education process. This will be the starting point for each group of Roma women to understand on what kind of social competence they really want to focus. Let us first clarify the meaning of the key concepts we are using: formal, non-formal and informal education and learning and social competence. Formal education refers to the structured education system that runs from pre-school and primary school to university, and includes specialised programmes for technical and professional training. Non-formal education refers to any planned programme of personal and social education designed to improve a range of skills and competencies, outside the formal educational curriculum. Informal education refers to the lifelong process, whereby every individual acquires attitudes, values, skills and knowledge from the educational influences and resources in his or her own environment and from daily experience (family, neighbours, marketplace, mass media, work, play, etc.). Social competence is a complex, multidimensional concept consisting of social, emotional (e.g., affect regulation), cognitive (e.g., fund of information, skills for processing/acquisition, perspective taking), and behavioural (e.g., conversation skills, pro-social behaviour) skills, as well as motivational and expectancy sets (e.g., moral development, self-efficacy) needed for successful social adaptation. Social competence also reflects having an ability to take another's
  3. 3. This project is funded with support from the European Commission. This document reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein. Project Nº 518536-LLP-1-2011-1-ES-GRUNDTVIG-GMP perspective concerning a situation, learn from past experiences, and apply that learning to the changes in social interactions. Social competence is the foundation upon which expectations for future interaction with others are built, and upon which individuals develop perceptions of their own behaviour. The concept of social competence usually includes social skills, social communication, and interpersonal communication. Having social competence means possessing the social, emotional, and intellectual skills and behaviours needed to succeed as a member of society and to adapt in appropriate ways to the demands of social interaction, including interactions related to work environment. The skills and behaviours required for healthy social development vary with the demands of particular situations. Measuring social competencies Social competencies have been identified by the European Commission as one of the key benchmark indicators to be targeted in order to improve prosperity and well-being in its Member States (EU 2005). Social competencies can be broadly defined as the capabilities enabling individuals ‘to live together in the world’ (Arendt 1958) comprising aspects of interpersonal, intercultural, social and civic competencies. Social competencies reflect adjustment in different contexts in society, requiring a focus on particular facets of social competence, such as empathy, self-control, trust, respect for other people, or civic engagement. Behaviours which are functional in one context might be dysfunctional in another, implying that the assessment of social competencies involves culturally based value judgments. These values are not fixed, but are subject to change and such changes become more and more visible in the current period, affecting all society and the work environment. The assessment of social competencies can be done with a variety of methods, ranging from self-ratings or self-reports of behaviour, values, and motivations; direct behavioural observations (in natural situations or under experimental conditions) to the use by a specialist of different assessment instruments. Such instruments can include behaviour rating scales (to be completed by parents, teacher, employer, subordinates, or by the person whose competencies are assessed); interviewing; role play; discussion based on hypothetical scenarios; interpretation of video clips; social network analysis and socio-metric approaches; as well as computer simulations. A widely used instrument to assess personality characteristics is the ‘Big Five’ inventory, focused on openness, conscientiousness, extraversion, agreeableness, and neuroticism (OCEAN) (Costa and McCrae 1992; McCrae and Costa 2004). Openness is reflected in a strong intellectual curiosity and a preference for novelty and variety. Conscientiousness is exemplified by being disciplined, organized, and achievement-oriented. Extraversion is displayed through a higher degree of sociability, assertiveness, and talkativeness. Agreeableness refers to being helpful, cooperative, and sympathetic towards others. Neuroticism refers to degree of emotional stability, impulse control, and anxiety. Other widely used self-reporting instruments include: self-esteem scales; measures of self- efficacy or emotional intelligence; tests measuring cognitive and affective aspects of dispositional empathy.
  4. 4. This project is funded with support from the European Commission. This document reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein. Project Nº 518536-LLP-1-2011-1-ES-GRUNDTVIG-GMP Concerns have been raised about the consistency and reliability of self-assessments as well as biases in reporting (Hagerty et al. 2007). One problem with single item measures (such as scales where participants tick a box corresponding to the level they evaluate they posess the respective competence), although it is attractive, is that such approaches are only suitable to assess constructs that are simple and unambiguous. Sometimes it is hard to capture the complexity of a competence and there is the risk to miss important differences at the individual level. Also, data collected in this way might be ‘contaminated’ by the context in which they are collected. Besides this, recent research (Kurger and Dunning, 2009) has pointed out that there is a general tendency for people who are more competent to underestimate their competence, while people who are actually less competent tend to overestimate their competence. Psychometric scales comprising multiple items to measure a specific dimension, such as social intelligence, social responsibility, assertiveness, or empathy, are considered by some specialists as more reliable, yet often take longer time to complete, and without abbreviation are not suitable for large scale surveys. The same applies to attempts to measure social competencies on the basis of assessments in experimental settings, make belief scenarios, or interpretations of video clips, which usually take more time to collect. An additional challenge appears with these instruments in our case due to the specific situation of target group members, with important cultural differences compared to the mainstream society in which the respective tests have been validated and with sometimes limited possibilities of understanding some items or of expressing properly the answers. Ideally the measurement of social competencies should involve different assessment modes, combining self-reports, rating scales completed by others, as well as observational data to obtain reliable and valid measures. Instead of direct assessments, multiple measures could be used as indicators of latent constructs, which would also facilitate comparative approaches of assessment. The High Level Advisory Group of Experts set-up by the European Commission identified in their report on the social integration of disadvantaged ethnic minorities, published in 2007, 14 barriers which prevent members of these groups from fully participating in the labour market: (1) Lack of education and training (2) Lack of language skills (3) Lack of recognition of skills and qualifications (4) Lack of access to professions (5) Lack of access to citizenship (6) Lack of integration policies (7) Stereotypes, prejudices and negative attitudes (8) Lack of mobility and concentration in certain areas (9) Industrial change (10) Disincentives through welfare systems (11) Discrimination (12) Lack of information (13) Labour market competition (14) Undeclared work These barriers are particularly relevant for many Roma and some of them might affect directly members of our local groups. Therefore, special attention should be paid to identifying for each participant which barriers have a higher impact in preventing access to the labour market. It might be that some barriers will not apply to some participants and this also needs to be considered. For instance, some members of the group might have had experiences of travelling to other countries over the past few years, thus acquiring significant life experiences due to mobility and sometimes valuable language skills which could represent an important asset for their future professional development.
  5. 5. This project is funded with support from the European Commission. This document reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein. Project Nº 518536-LLP-1-2011-1-ES-GRUNDTVIG-GMP One additional challenge is that, from our previous experience and based on the first interactions with the members of our group, we anticipate that at least in some cases participants will not consider social competencies as relevant or playing a decisive role in finding a job. They will also expect that employers will not be interested in such competencies but rather require formal qualifications. This is also related to possible low expectations of the women in the group, based on their own previous experience and on what they see around them in the community. Therefore, besides the transversal social competencies, it is useful to make an inventory of other specific competencies that women in the group might have, including competencies related to certain job profiles. It is also important to provide opportunities for the women in the groups to express their own projects for personal and professional development and learning, including further education, qualification, etc. They will also need opportunities for exchange and joint reflection with peers in order to overcome some of the limitations they face in their concrete situation and to get fresh ideas about what they could do in the future in their professional life. In this context, the assessment should also reinforce the self-esteem of participants and avoid risks for them to feel looked down to in this process. Considering all these challenges, our suggestion is to use a combination of qualitative assessment methods which can be connected with specific life experiences and situations of group members. Thus, the methods recommended for assessment of competencies of Roma women in the project group include: 1. In-depth interviews with focus on life story and description of daily life experiences 2. Appreciative inquiry 3. Focus group The results obtained with these methods will serve, on one hand, for the development of the training activities to be implemented in the next phase of the project and, on the other hand, for identifying employers potentially interested in hiring Roma women in the group. The results can be structured in two categories: 1. Competencies that participants have and that can represent an asset for them on the labour market 2. Social competencies that are important for a successful integration in the labour market but that they lack or have in an unsatisfactory level. The first category should be presented to each participant individually and this can be a way to: - Get useful feedback from them and possibly add useful element that are identified by participants after looking at list of their competencies; - Boost the self-esteem of participants and their confidence that they can envisage professional progress. The second category should not be communicated to group members and should be kept confidential by the team. By communicating negative assessments they could be demotivated or manifest defensive attitudes. But these elements can give useful hints for the development of the training activities.
  6. 6. This project is funded with support from the European Commission. This document reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein. Project Nº 518536-LLP-1-2011-1-ES-GRUNDTVIG-GMP In order to reach to the two lists of competencies for each participant, we suggest taking an inductive approach: collect qualitative information and process this information by identifying elements to be put in each category. This can be done by paying attention to competencies acquired by the Roma women in the group through (the list is not exhaustive): - Formal education - Professional qualification trainings - Non-formal education activities - Work experiences, both formal and informal - Interaction with others (e.g. learning how to do something from a family member) - Daily household activities (e.g. cooking, gardening) - Providing care to family members (children, elderly, ill people, people with disabilities) - Participation in community events (e.g. organising wedding party, traditional festivals) - Travelling and changes in place of living (locally, nationally, internationally) - Media (e.g. radio, TV) Thus, the output of this process can be for each participant two lists: - Competencies acquired - Competencies lacking They can be presented in a descriptive way, as they result from the statements of participants, and can be categorised and organised at a later stage. 1. In-depth interviews Describing life situations is a simple and useful way to get information about what a person knows, thinks, or knows to do. Asking specific question about their daily life will also put participants in a comfortable situation, providing more reliable information than what can be obtained by answering questions from a questionnaire. Before the interview You explain to participants that an open discussion of about one hour about their life is necessary for making an assessment of their competencies, going beyond those acquired through formal qualifications or through work experiences. You set up together a meeting, specifying time and place. Make sure that the place is easy to reach for the participant an that it provides good conditions for the discussion, including privacy and protection from disturbances. Prepare a device for audio recording and make sure it has enough space for the whole interview. It is recommended to test the recorder if you have not used it before. During the interview Ask questions slowly and clearly, giving the person time to answer. You can use some closed questions (which prompt a respondent to give only a "yes" or "no" answer), but most should be open questions like: "Tell me about..."; "Describe..."; "What was it like when...?"; "In what ways...?"; "Why...?"; and "How...?" Start with easy, friendly questions and work your way up to more difficult or sensitive questions. The precise set of questions will not be the same for all participants, depending on various factors, such as their age, education, specific life experiences, etc.
  7. 7. This project is funded with support from the European Commission. This document reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein. Project Nº 518536-LLP-1-2011-1-ES-GRUNDTVIG-GMP Here is a set of suggested questions (select only the ones which are relevant and adapt as appropriate, both the content of the questions and the language used): - Tell me about your childhood. Where were you born? Where did you spend your first years? What do you remember? - Who were the most important persons for you when you were a child? What did you learn from them? - As you grew up, were you helping with household activities? What where you doing? Who taught you how? What did you like most? - Who were your friends? What were you doing together? Did you have conflicts with them? How were they dealt with? - Where did you go to school? How was it? What teacher you liked most? Why? How did you get along with the colleagues? - Let’s talk now about your work experiences (not necessarily related to employment). What was your task/activity? With whom were you working? Describe a successful experience of interaction with co-workers, with your boss and with clients/beneficiaries - At home, what are your tasks? How did you learn how to do that? What is your reaction if there is a disagreement between family members or between family members and others? Did you take care of other family members (children, brothers and sisters, elderly)? How? Could you give an advice to others in this situation, based on your experience? - Tell me about when you travelled abroad/to another region/city. Where did you go? With whom? What did you do there? What did you find surprising or different there? How did you manage to communicate with the people there? What are the most important things that you learnt from this experience? Listen carefully to what the person says; don't interrupt or correct. Maintain eye contact and show interest by leaning forward and nodding. As you listen to answers, other questions will come to mind. Asking follow-up questions will help you get more information. If someone is talking about an unhappy or painful experience, show that you understand how they feel ("That's very sad"). If the person doesn't want to talk about something, that's okay – just go to the next question. It's okay for there to be moments of silence or emotion. A person's life is important, and emotion is natural. Accept emotions as part of the process. An interview shouldn't last more than about an hour. People do best when they're not tired. If necessary you can schedule another time to continue. This also allows you to think about answers, and come up with other questions based on the answers and things that interest you. Don't forget to thank the person you've interviewed. Let her know that you are going to meet again to provide feedback after the analysis of the interview and for a second short interview (the appreciative enquiry). After the interview
  8. 8. This project is funded with support from the European Commission. This document reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein. Project Nº 518536-LLP-1-2011-1-ES-GRUNDTVIG-GMP You do not need to transcribe the whole interview (this will save a lot of time and energy). You just need to transcribe the sequences which reveal competences (things that the participant knows or knows to do, as well as specific attitudes). It is preferable to do this shortly after the interview, as it will be easier to remember details which can help understand possible difficult sequences. Processing interviews as you do them will also help you in the following ones as you gain useful experiences with each interview. 2. Appreciative inquiry This is a brief activity which aims both to complete the information acquired through the in- depth interview and to contribute to increasing the self-esteem and the confidence of participants by pointing out various positive elements about them. This can be done during the second meeting with each participant, when the conclusions of the analysis of the in-depth interview are presented. You can start by showing the conclusions you drawn from analysing the interview, pointing out only the positive elements (what the person knows, what she knows to do and the positive attitudes that she manifested in various situations). You can ask for feedback, confirmation of the list of competencies identified and also ask if she wants to add something to the list and illustrate each additional competence with a specific life experience. After this, ask a brief series of questions focused on positive elements. The structure of the interview can be the following: - Give two examples of situations from your life when you felt happy or proud - Describe two positive qualities of yourself and illustrate how they were shown in various situations - Imagine that you get a job according to your wishes and that you have worked for a year. How would you like your boss to praise you at the end of your first year at work? This type of questions is likely to help participants identify their strengths and picture themselves in a positive and successful professional experience. 3. Focus group Introduction: Focus group is a qualitative research method which takes the form of a group interview, a facilitated discussion led by a series of questions around a specific topic. Below are some instructions for the facilitator of the focus group. Duration of the focus group: 90-120 minutes The recommended number of participants in the focus group is between 8-10, in order to ensure a good interaction and group dynamics and allow everybody enough time to express opinions. Therefore, for our 15-20 participants, there should be two focus groups organised.
  9. 9. This project is funded with support from the European Commission. This document reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein. Project Nº 518536-LLP-1-2011-1-ES-GRUNDTVIG-GMP The division of participants in groups can be done based on various criteria but it’s important to ensure some degree of diversity in each group. This way, participants will be more stimulated by each other. Opening The moderator has to ensure that all invited participants are present and sit in a circle. After having introduced himself/herself and reminded that the purpose of the discussion is to get ideas useful for preparing a training programme and for facilitating the employment of participants, the moderator introduces the rules for focus group conduction, emphasising the following elements: • Confidentiality of the information provided. The discussion is recorded (video and sound) but the recorded material will only be used for elaborating the report and will not be made public; • The discussion will be approximately 90 minutes; • All participants should listen to each other respectfully but they are encouraged to formulate comments regarding what the others are saying. Interventions and interruptions are possible; • Participants speak clearly and loudly enough, one at a time. • Because of the time limitation, the moderator will interrupt the participants in case of diversion and will keep the conversation close to the main line. • Participants express their personal opinions and wishes in a way that will not harm or affect in a negative way the others. When everyone has confirmed agreement with these principles and rules, the moderator starts the focus group session. The focus group discussion INTRODUCTION OF PARTICIPANTS: If participants did not meet before, the moderator opens the discussion, giving the floor to each participant for a short introduction. NOTE: The moderator might need to pay special attention to encouraging some participants to speak and will make sure that all get to intervene. It might be necessary to adapt the formulation of the questions in order to ensure that they are well understood also by participants with lower educational background. The moderator will then address the following questions: 1. First of all, I would like to learn your opinion about employment. Is it difficult to get an employment for you these days? If yes, what makes it difficult? 2. What kind of job would you like to have? Why? 3. Share experiences about when you tried to get a job. How did it go? 4. What do you think employers are looking for? What kind of employees they want to have?
  10. 10. This project is funded with support from the European Commission. This document reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein. Project Nº 518536-LLP-1-2011-1-ES-GRUNDTVIG-GMP 5. What do you think you need in order to improve your chance to get the job you want? 6. What can you do to obtain that? Who can help? 7. Do you have experiences of working on the black market / without registration or official documents? What are the disadvantages of doing this? 8. Have you thought about setting-up a small business or joining together with other people to set up a social cooperative or another structure that could give you the possibility to work and earn an income officially? CLOSING OF DISCUSSION: The moderator has to give the opportunity to participants to ask questions (if they have any). After that, the moderator has to thank for the participation and specify again that the work together in the project will continue. NOTES: The suggested sequence of the questions is not mandatory. Changes and flexibility in the order of asking are allowed. It is important to discuss all questions and each participant should have the opportunity to express her opinion for each of the questions. In the case that participants start the discussion on a question that is not asked yet, the moderator will not ask that question again. TAKING NOTES OF MAIN ELEMENTS IMMEDIATELY AFTER THE END OF THE DISCUSSION: The discussion will be recorded, but it is important that the moderator does a short memo with the main accents of the discussion; points out and writes key quotations that could be used in the analyses; sketches out briefly his/her impressions of the participants – their involvement in the discussion and their reactions to specific topics, as well as any other elements that are not grasped by the recording. Reporting For the report on the competencies of the women in each group, we suggest the following format: 1. Brief introduction describing the group and the process of assessing the competencies 2. Competencies inventory for each group member, using the following structure: a. Level of formal education / basic reading and writing competencies in all known languages b. Acquired social competencies c. Lacking social competencies d. Other acquired competencies (e.g. technical skills, like cooking, computer skills, specific types of knowledge) 3. Summary analysis with conclusions leading to the training needs. The second section can have the format of a table, like the one below:
  11. 11. This project is funded with support from the European Commission. This document reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein. Project Nº 518536-LLP-1-2011-1-ES-GRUNDTVIG-GMP Name: 1. Formal education and basic competencies a. Formal education 8 grades Comments (e.g. local school) b. Reading skills in ……. Basic / average / good Comments c. Wring skills in ….. Basic / average / good Comments Acquired social competencies a. competence Illustrated by b. c. d. Lacking social competencies a. competence Illustrated by b. Other competencies a. competence Illustrated by b. c. Considering the explanation above, we do not provide a list of competencies to pick from, as this can be a misleading process. We propose to each evaluator to go through a stage of identifying from the interviews - What participants know regarding social interactions (with members of their family, with people in the community, with other people) - What participants know to do in terms of effective interactions with other people - Significant attitudes participants have towards the interaction with other people. The analysis should focus on both correcting lacks of important social competencies and on expanding and building upon competencies already acquired. References Council of Europe (2002) COMPASS – A manual for Human Rights Education with Young People, www.coe.int/compass Ingrid Schoon (2009) Measuring social competencies, German Council for Social and Economic Data, Working Papers Series European Commission (2007) Report of the High Level Advisory Group of Experts on the Social Integration of Ethnic Minorities and their Full Participation in the Labour Market FamCompass: Assessing and Validating Family Competences, http://www.famcompass.eu Workalo - The creation of new occupational patterns for cultural minorities: the gypsy case. CREA, University of Barcelona, http://www.neskes.net/workalo/teoriaan.htm Kuger and Dunning (2009) Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments, Psychology, no 1, 2009 pp 30-46

×