Tag Archives: Exams

Assessment Part 2 – How I Conduct My Speaking Test

Picture

Yesterday I reflected on how I think assessment is very often failing our students. Given such a bold claim I guess it’s only right that I put my money (or reputation at least) where my mouth is! I’m currently conducting speaking tests that seem to have been, so far, quite successful. I’ll be posting my full reflections once all my students have completed the tests and got their results. This is more a template as to how I went about introducing the speaking tests this semester that anyone is welcome to copy, compliment, criticize, offer suggestions for improvement or anything else!

First of all, some context, I had two weeks to administer around 450 speaking tests. That gave me two hours with each class of 30-40 students (1 hour per week). That basically left me with 3 minutes per student. My objectives were:

–          That I should test the students’ ability to produce English spontaneously.

–          The students should feel challenged.

–          The students should feel a real sense of achievement on completing the test.

–          Students of different level should be able to answer the questions, while still being able to show off their high level skills.

–          Students should receive feedback on areas they need to improve.

–          It should be a useful experience for the students.

–          It should allow me to identify common problems.

After a lot of deliberation on how to do this, the ever awesome Michael Griffin (@michaelgriffin) suggested I copy the speaking element of the NEAT test. This is a test being introduced by the Korean government that students will take before entering university (more about this in my next blog). The government is also aiming for it to replace the many different forms of English testing currently used in Korea. The format of the test, which I am going to briefly explain below, seemed to tick the boxes and would certainly be a useful experience for the students. I’m now going to outline the process I decided upon.

Stage 1 -> Preparing the Students

For me, this is by far the most important stage, and one I definitely didn’t do adequately last year. The first thing I had to do was explain the new test format to the students (as they haven’t done anything like it before, for their responses please see my previous post) and adequately prepare them for the test. To do this I set aside two lessons to explain the types of questions they would get and give them practice questions. Please see the PowerPoint below for a quick guide to the test format.

The test has four sections:

1. Advice Giving. The students were read a short problem. Although in the actual NEAT they would only hear a recording, I also allowed them to read the problem, They had10 seconds to think, 1 minute to answer.

2. Conversation. The students were asked 4 commonly themed questions. They had 20 seconds for each answer.

3. Story Telling. The students were shown six pictures in order and they had to make a story. They were given 10 seconds to think and 1 minute to answer.

4. Graph Description. The students were shown a graph. They had 10 seconds to describe and compare the data in the graph.

I decided advice giving was much harder than the others, and there was only time for students to answer two categories, so every student would get one question from advice giving and one from another category. They wouldn’t know in advance so they would have to practice for all of them.

After explaining the new format, including explaining the scoring chart so they knew exactly how they could gain/lose points, we spent 25 minutes practicing each category. The guide I gave to the students including tips etc. is in the PowerPoint below. The example questions used are in the scribd document below (feel free to use them). To practice I used the following pattern:

–          We analyze an example as a class.

–          I give them tips as to what is expected from them and where I think they can get easy or lose easy marks.

–          Student works with a partner to write what they think would be the perfect answer.

–          The students do one more practice question spontaneously.

–          I give feedback to the class including tips for the real thing.

Stage 2 -> Conducting the Tests

For obvious reasons, the tests had to be done one at a time and as efficiently as possible. While the majority of the students were in the classroom, one student at a time came into my office. To save time they didn’t pick a question, instead I had a big pile of questions that I went through one at a time. They were given 10 seconds thinking time for both quesitons, and then 1 minute to answer (apart from conversation, which they were given 20 seconds X 4 questions). Straight after they finished I filled in their scoring chart (please see scribd document below) and the next student came in.

Stage 3 -> Feedback

The students will be given their scoring chart (which is out of 45). They won’t, however, be given their score out of ten until a week later. I hope that by doing this the students will look at the chart at areas they did well and areas they need to improve. Although I would like to give more personalized feedback, with 450 students this would just take too long. I have also made notes on mistakes I heard in their speaking tests (during the test) that I will give them. Once all the test are finished we will have a class where I can give feedback on the most common mistakes made and do some practice activities to improve for next time. Although I would like to do more, time is a real problem.

Please feel free to use any formats of this speaking test that you like. Tomorrow I’m going to be blogging about how I think introducing NEAT, or any other speaking exam, into our assessment could have a real positive impact on the way our roles as NETs are seen in the Korean education system.

Comments

04/09/2012 05:53

Yet another quality and useful post Alex!

I must say I am a bit intimidated by your thoroughness and thoughtfulness. I would like to say that I would do the same if I were provided enough time to successfully accomplish all the tasks you set out to do.

That being said, while my speaking tests are under even more of a time crunch, without the time to prepare students or give feedback, there are quite a few points that I can take from you and mold to my own uses.

I test my whole school (approx 1000 students) and have one 45 min class (35-45 students per class) so efficiency is a must. I usually despair at the uselessness of it all. I like how you have broken the questions into categories and allot specific time to each student for thinking and for answering. I think I can take that idea and hopefully give my students a more even appraisal.

Thanks again for all the great work you do (and most importantly sharing!!!!). It is a fantastic legacy for the many who will follow you.

Reply
05/09/2012 20:23

Hey mate,

This is a great post, and I’d imagine would be thoroughly useful to a lot of people. Did you pimp this post out on Waygook at all – I really think that you should.

How did you find doing the improvement grading? It seems like a hell of a lot of categories to get through, though I understand that you want to help students in specific areas. I wonder though, whether it wouldn’t be easier to focus on a narrower range of things, and just note problem areas for the student. Did you feel you got enough data in 3 minutes to make all of those judgements?

Reply
AlienTeachers
25/09/2012 20:01

Hey buddy,

I’m glad you enjoyed it 🙂 TBH it was quite difficult getting through all the categories, the reason I chose them is that they are the categories they use for the real NEAT exam, so I though it would be good practice, but yes, I think I might streamline that next time.

Thanks as always dude,

Alex

Advertisements

Assessment Part 1 – Being Cruel to be Kind!

Picture

Before I start I should point out that what I’m really talking about here is testing in Korea, well actually testing in the two schools I’ve worked at in Korea. However, from discussions with friends working in Korea, and other teachers throughout Asia, it seems like the problem is much more widespread than the school I work in. I should probably also point out that I’m not saying problems with testing are only apparent in Asia, it’s just most of the teachers I talk to on this issue are based in Asia.

O.k. disclaimers over!

In my experience, the problem with testing and assessment here is that it is done purely to receive a score. Students go into an exam, they take the exam (in my case a 2 minute speaking test), they leave the exam, they wait a week, they are given a score and finally they complain or are happy about the score. The students often don’t see the test paper once the final bell rings, no feedback is given, no areas for improvement suggested, they are just ranked and placed on a chart so they can see how well/badly they are doing. If this was just once in a student’s career, for say, entering university, then fine, I could understand, but that’s not the case. This monotonous process seems to be happening over and over and over and over and over. All the way through a student’s school life, they are tested, given a score, ranked and then move on. What the heck is the point? It takes 1 month out of a 7 month time table every year and I just can’t figure out what is being achieved other than forcing students into undue stress.

So the students know they’re failing? But they don’t know why or what parts they are failing.

So a student knows they are doing well? Are they really going to know they are doing well in conversation class by reproducing a script that has likely been written for them?

So a teacher can see where the class needs to improve? All the teachers get is a sheet with students’ scores on.

To improve the students knowledge? The students cram for one week, don’t sleep, regurgitate the knowledge, and then promptly forget it again 24 hours later.

Unfortunately, it seems this has happened so many times that all anyone now cares about is this pointless, meaningless, score! When I introduced the new format of speaking test to my students this semester the negative response from both co-teachers and students was overwhelming because they could not think beyond the test to get a score culture that is so ingrained.

To summarise the issue, the students had always been asked by their conversation teachers to go away, prepare to answer some pre-made questions, go into a room, reproduce the script they had memorized and then get a score. What mostly happened was that the students went away with the questions, the students then went to hakwons or, if their parents could speak English, to their parents, got a script written for them, which they memorized and regurgitated. They were then told they were great at speaking English and everyone was happy! Test done, high scores achieved! This is how it had always been for them and this is how they wanted it.

The speaking test format I suggested involved copying the new NEAT exams that are being introduced by the Korean government. There are 4 types of production questions, the students don’t know the questions before hand and they would only have around 10 seconds to answer. When I explained the format to my co-teachers and students I explained the following benefits:

–          It is good practice for an exam (or similar exam) students will likely have to take in the future.

–          I will provide the students with feedback letting them know exactly what area they need to improve on for future speaking exams.

–          It is actually testing their ability to converse, or at least produce, the English language.

–          To summarize, it is actually useful for them (I think).

I was met with the following objections to changing the format:

–          But asking them to produce language is not fair as it benefits students who have lived abroad.

–          Could you (the student speaking to me) do it in Korean?

–          It’s too hard for them (bear in mind my students are almost the highest level in Korea).

–          Not telling the students the questions before hand means their answers won’t be as good so the average score will be lower.

Both my students and co-teachers were upset by the changes. The thing is, my students are first grade high school. The score does not go on their official record that universities will see and the classes are mixed ability so it doesn’t affect the classes they will be put in but it’s all they can think about. Their score, at this stage, is really quite meaningless. But, no matter how many times I explained the benefits, they could not see the test as a positive experience as opposed to a score giving procedure. From speaking to other educators it seems I’m not alone in struggling against such a test giving culture.

I really think this is such a shame. There are so many positive benefits I believe conducting a speaking test can have for our students, to provide a few:

–          It can be a useful chance to practice exam skills.

–          It can provide feedback on what they need to improve for the ‘real thing’.

–          If provided with informative feedback it can improve confidence.

–          It can inform the teacher what need working on (after just one day of tests I know I need to do more work on prepositions!).

–          If we are really testing our students, and they are successful, they will feel a huge sense of accomplishment. If my students are successful in this test (which I genuinely believe they will be as it is designed for their level) they may really start believing they can converse in English (which they can). I don’t believe students would get the same sense of achievement by regurgitating a script their hakwon teacher wrote for them.

To conclude I think testing in Korea, and from what I understand many other educational systems around the world, really need to start thinking about what they are hoping to achieve with their assessment. We need to consider questions such as: Where are the benefits for the students? Where is the feedback going to come from? What are the students’ achieving? How are they going to achieve it?

If we do this perhaps we can start swinging the test giving culture into a useful experience for all concerned.

I’d really love to hear about other people’s experiences with administering tests, especially if you’ve managed to get students or co-teachers on board with the experience over results philosophy. Also, it would be great to hear about any other ideas for implementing meaningful assessment.

 
Tomorrow I’m going to post how I went about setting up the speaking tests that do, so far, seem to have been quite successful, in other words, my suggestion for implementing a meaningful speaking test (focused on Korea).

Also, you can follow me on twitter @AlexSWalsh

http://keltchat.wordpress.com/2012/07/07/all-about-assessment/

In the meantime I recommend checking out the following resources:

http://www.teachingenglish.org.uk/articles/testing-assessment

http://rhaylagar.hubpages.com/hub/The-Benefits-of-Giving-Exams

http://www.tefl.net/alexcase/tefl/young-learner-exams/

Comments

DK
03/09/2012 09:10

I completely agree with your assessment of any speaking tests that involve only memorizing and regurgitating. These tests do nothing to promote conversational skills, focusing only on pronunciation and clarity of speech (I’m assuming) rather than listening, comprehension, and responding in a timely manner. And to not receive any feedback whatsoever is a complete waste of time.

Teaching at university, I have all my students answer questions chosen at random, based on what they have learned over the course of the semester. They know all the possible question types before hand (approximately 30-40), but they will choose which ones they answer from a cup at the beginning of the test and I give additional bonus follow-up questions. Some find this challenging, but if they are given similar tests in middle school and high school, they would be much better prepared for it and be able to enter university with a far greater ability to express their ideas and opinions than they do now.

There is something broken with a system that thinks challenging students is a bad idea because it will effect their current “high scores”.
Many students in university expect A’s for mediocre work. In Canada I was happy to get above an 80, and to get a 90 or higher was awesome.

I say raise the bar and you will be pleasantly surprised at how many are able to meet and exceed your expectations.

Reply
AlienTeachers
03/09/2012 17:27

Hi DK!

First of all, thanks for your comment, it’s always appreciated. That’s really interesting to hear that you employ similar productive speaking tests at university. I wonder if you meet any kind of resistance from the students when you explain the procedure to them?

I completely agree that by raising the bar and challenging our students people might be pleasently surprised at what they’re really capable of.

Thanks again,

Alex

Reply
ioana
04/09/2012 09:27

hello, Alex!
I partially agree with what you’re saying about assessments.yes, right, for the test papers students just get a score, but it’s essential for them to write correctly. and this is the only possibility we have to check their spelling and partially their grammar. I had some surprises with my high school students. some of them have a good pronunciation, fluency, vocabulary but when it comes to writing they have serious problems. there was such a big difference that I couldn’t believe the test papers belonged to the same students that were able to speak in English for hours. each test paper has more parts and it’s easy both for you and your students to identify where the problems are. though it’s difficult to discuss with each of your students we usually tell them where the problems are. regarding the speaking tests, you’re right, we have to focus on encouraging them to react to some challenges and not just reproduce the phrases they have learn. I tell my students to try using English while speaking to their friends, their colleagues, their penfriends because this is the best possibility to get over their shyness and produce instant messages.

Reply
05/09/2012 03:39

Being cruel to be kind,

Is the same make me feel how we were born under edu-system, with no more applicable at all,
Meanwhile, this article discussed seems what is likely to happen in our country.
Naing