Evaluation of LISTN First Peoples: Learning Materials for Newcomers

Maureen Spearman                                                          February 20, 2019

Evaluation of LISTN First Peoples: Learning Materials for Newcomers,  CLB 5 Aboriginal Art Lesson

1. Target Learning Context:

Learners are at a CLB 5 level 

2. Learning Objectivesof the unit are: (as stated directly in the Teacher’s Notes of First Peoples: Learning Materials for Newcomers. Aboriginal Art Unit (CLB 5)  

·      “learn about the importance of cedar to Aboriginal art and culture.


·      learn about different types of traditional and contemporary art.


·      learn about different designs used by Aboriginal artists and the purpose of these designs. 

·      learn about a few Aboriginal artists.


·      learn about the important role art plays in Aboriginal cultures.”

3. Evaluation Criteria: Material Analysis

Evaluation Criteria:   Criteria are give a score between 1 and 5, with 1 being the lowest score (least extent), and 5 being the highest (most extent).	Score ?/5
1. Universal Criteria:  To what extent….
Do the materials allow and encourage the learners to think critically?	4
Are the materials cognitively engaging to this level and group of students?	5
Are the materials affectively engaging to the students?	4
Are the instructions in the material clear and understandable?	5
Do the materials allow for building upon known skills?	4
Are the materials authentic and realistic?	4
Do the authentic materials reflect native English speakers?	4
Do the materials allow and encourage learners to extend their learning beyond the classroom into their real life?	3
Can students derive and adapt material from the lessons that would be specific for them (make it personal)?	4
Does the material reflect natural English speaking in the local environment?	4
Does the material help to develop cultural awareness of many cultures?	3
Do the materials reflect the needs and goals of the leaners?	3
Do the materials encourage learners to participate and incorporate their own experiences?	3
Is the material useable in classes where students may miss classes regularly?	3
Does the material have emotional impact upon the learner through very novel experiences?	3
Is the material inclusive (age, race, gender, etc)?	4
Would the material motivate the learners?	3
Would the materials help improve the language confidence of the learners?	5
Score for Universal Critera:	68/90
2. Content-specific Criteria:  To what extent…
Do the materials reflect the suggested content of the CLB 5?	5
Are the materials useable and adaptable to a PBLA environment?	4
Are the materials acceptable or adaptable to other CLB levels?	3
Does the material have material useful for literacy students?	4
Is the material adaptable to different types of language programs?	4
Does the material have games and suggested activities that link into the lesson plan?	5
Does the material contain separate teacher resources to sufficiently explain the material?	5
Does the material allow for a range of interactive activities in pairs, groups and whole class?	5
Does the material apply to a broad range of learning styles?	4
Is there a large range of material to choose from for each lesson?	5
Are the authentic materials appealing?	4
Are the authentic materials diverse to give a broad perspective of the topic?	4
Are the authentic materials easily understandable at this level?	5
Are the authentic materials relevant to the topic?	5
Are the authentic materials an appropriate length?	5
Are the authentic materials using appropriate grammar and language?	5
Are the authentic materials useful for different learning styles?	4
Does the material have resources to extend the lesson?	5
Do the materials reflect the suggested content of the CLB 5 reading skills?	5
Do the materials reflect the suggested content of the CLB 5 speaking skills?	5
Do the materials reflect the suggested content of the CLB 5 writing skills?	5
Do the materials reflect the suggested content of the CLB 5 listening skills?	5
Does the material help the students understand the importance of the cedar tree to the BC indigenous communities?	5
Does the material help the students learn about types of traditional and contemporary art?	5
Does the material help students learn about different designs used by Aboriginal artists and the purpose of these designs?	5
Does the material help students learn about a few Aboriginal artists?	4
Does the material help students learn about the important role art plays in Aboriginal cultures?	5
Score for Content-specific Criteria	125/135
Age-specific Criteria:  To what extent….
Are the materials relevant or adaptable to a large range of ages?	5
Is the material accessible for learners with age-related problems (eg. hearing loss, possible mobility issues)	4
Score for Age-specific Criteria	9/10
Media Criteria:  To what extent….
Is the material inclusive to a range of levels of comfort with digital technology?	4
Is the material inclusive for teachers with a wide range of comfort with digital technology?	4
Is digital technology a necessary part of teaching in the classroom?	4
Does the material have sufficient supporting media material for the classroom (overheads, pictures, website, online games).	5
Does the material encourage the use of outside digital media by suggestions?	4
Are required digital technology tools commonly found in ESL classrooms  (overhead projector, large screen, digital projector, computer)?	4
Score for Media Criteria	25/30
Local Criteria:  To what extent…
Does the material help to develop cultural awareness of Canada?	3
Does the material help to develop cultural awareness and understanding of British Columbia?	5
Does the material help and encourage the students to connect with the local community?	4
Does the material help and encourage the learners to integrate into local society?	4
Does the material help the learners prepare for their citizenship studies?	4
Is the material designed for a range of class sizes?	5
Score for Local Criteria	25/30
Other Criteria:      To what extent….
Do I as a teacher like the content of the material?	5
Do I as a teacher socially connect with the material?	5
Would I feel comfortable using the material in the class?	5
Could I modify the material for my class?	4
Is the material accessible or can be made accessible to all learners (eyesight problems, moveability, disability)?	4
Does the material encourage learners to communicate in English only in the classroom?	4
Are assessments built into the material?	2
Does the material require minimal amounts of homework for the student?	4
Score for Other Citeria	33/40
Total Score for Evaluation	285/335
Total Percentage for Evaluation	85%

To conduct the evaluation: 

1.    Recruit a set of teachers who work in the CLB 4/5 levels to use the evaluation questions. 

2.    Give a range of 1 to 5 for each question.  (5 is the highest score, 1 is the lowest) 

3.    Teachers independently evaluate the material, through scoring and general comments for each section. 

4.    Tabulate the scores from each evaluator and also the average total score. Look to see if scores are similar to the average for the evaluators, or is there is a wide variation.  If there is a wide variation, try to learn why.  Summarize comments and recommendations.

Evaluation Summary: 

The evaluation results in a score of 85% which is very high, and thus I would suggest that the material is very suitable for the target learning context and the stated objectives.  Note that this scoring is only mine, and may vary with other evaluators. 

1. Universal Criteria: 

The material generally scores well in the universal criteria with some exceptions.  The material is engaging, authentic, easily understood and adaptable. However, some the areas that might improve are 1) more encouragement for students to extend the learning beyond the class, 2) integrating the material into more general cultural awareness of other world cultures and their own experiences, 3) providing more emotional impact in the lessons, 4) better reflecting the goals and needs of the learners.  

The objectives stated in the material do not reflect the in-depth activities of the lesson, and could be stated in a more action-oriented manner.  Instead of ‘learn’ prefacing all of the objectives, have “Learners understand the role of art in indigenous communities”, “Learners discuss the historical and cultural aspects of indigenous art”, “Learners are able to research various forms of indigenous art and artists for further learning”, “Learners understand the value of art and artisits in indigenous art and their own cultures”. 

2. Content-specific: 

The materials score very high in the content-specific criteria. The materials incorporate many writing and listening challenges and exercises, including new vocabulary, writing an email, preparing a poster and presentation, etc.  These are all skills which are not stated in the objectives but may be if the teacher using the material appropriately.  The levels for reading, listening, speaking and writing for CLB 5 are all clearly stated in the materials and are linked to suggested activities in CLB 2012 guide.  The one area which may be strengthened is the ability to adapt the material to different levels.  Although most of the visuals may be useful for other levels, most of the activities would have to be modified to some degree. 

The material used in the lesson has many authentic resources.  The pictures of indigenous art forms are the most important authentic resource of the material because they highlight and focus the student on the art forms, which is the focus of the objectives of the lesson. Another authentic resources is the email, which shows the learners an authentic example of an informal email.  However, the email may not be considered truly authentic as it written for the materials.  The poster for the art show is also not truly authentic, as it seems prepared specifically for the materials.  The materials do suggest websites where the students can access more authentic information on art and artists.  Videos could have been used to showcase some of the artists, and magazines or newspapers could also probably be used for more authentic material. However, some of the more authentic materials may be difficult for this level with regards to vocabulary and reading, and may have to be modified for use. 

3. Age-related criteria:

The material is acceptable for all ages. 

4. Media Criteria:

The media suggested in the material is all commonly found in ESL classrooms, and does not require above average skill by either the teacher or the students. The media use heightens the appeal of the material and several options are given for alternate use of media. 

5. Local and Other Criteria:

Developing cultural awareness of First Nations though indigenous art is the focus of the lesson and it does this very well.  However, the material does very little to help integrate this into Canadian culture or discuss other First Nations.  As a teacher I would welcome use of this material and feel comfortable using it in a class, but some indication of assessments that could be used within the material would be useful. 

Summary of Recommendations for Changes: 

1.    A stronger connection could be made between art of indigenous cultures and other cultures.  One example could be to write about differences or similarities between the two cultures. Learning about another culture may be very interesting for some students, but others might not be as motivated by the materials.  Linking student’s own cultures more into the materials from the beginning may help with motivation for learning and sharing.  

2.    It would be useful for teachers to have some suggestions as to what could be used for assessment.

3.    The material could be more inclusive of other First Nations of Canada. Newcomers to Canada may not recognize that there are many other First Nations with Canada, each with their own cultural identity, some of which are very different from the BC West Coast First Nations. 

4.    The material does not really consider the needs and goals of the learners. It is well into the material before learners are able to choose activities of interest.

5.    The objectives should be changed to incorporate more action-orientated statements.

References:

Canadian Language Benchmarks (2012) English as a Second Language for Adults. Toronto, On.         Retrieved from https://www.language.ca/publications/english-publications-  downloadable/ 

Collins. A. (2016). Framework for Evaluating ELT Materials.  Retrieved from      http://blogs.brighton.ac.uk/te714ac792/2016/02/29/framework-for-evaluating-elt-materials/

LISTN (2014) First Peoples: Learning Materials for Newcomers.  Aboriginal Art Unit (CLB 5)             Retrieved from https://listn.tutela.ca/wp-content/uploads/CLB5-Art-SS-2014-07-  29.pdf

Lynch, L. (2009). How to adapt authentic language materials in English language teaching.       Retrieved from  https://www.eslbase.com/teaching/adapt-authentic-materials

Rustamova, Z. (2016). Pre-use evaluation.Retrieved from           http://blogs.brighton.ac.uk/te714zr37/2016/03/10/pre-use-evaluation/

Tomlinson, B. (Ed.). (2014). Developing materials for Language Teaching. United Kingdom:    Bloomsbury Academic.