Say WHAT? Run that by me again…

Archive for the tag “Listening”

Insight on the Listening Process from John Field: article review

Article title: An insight into listeners’ problems: too much bottom-up or too much top-down?

JournalSystem 32 (2004)

Author: John Field, professor at University of Leeds, UK. Teaches psycholinguistics, child language, and English grammar. His widely-used textbook, Listening in the Language Classroom, won the Ben Warren International House Trust Prize. A teacher trainer, materials writer, and syllabus designer as well as practitioner, Field travels widely lecturing on L2 listening.

Doi: 10.1016/j.system.2004.05.002

Type of Study: Empirical study based on observations gleaned from an analysis of three related experiments.

Purpose: To clarify the relationship between L2 listeners’ use of top-down and bottom-up listening strategies.

Research questions:  1) If top-down and bottom-up information are in apparent conflict, which one prevails? and 2) How do learners deal with new items of vocabulary when they crop up in a listening passage?

Procedure: 47 NNS students from a leading British EFL school were given listening tests in a classroom with good acoustics. The tests were designed to reveal learners’ listening strategies by presenting them with a series of problematic items, forcing them to choose between semantics (representing top-down listening strategy) or phonology (representing bottom-up). Each test was slightly different, and designed to explore different aspects of the research questions.

Results: The experiments produced both expected and unexpected results. Fields found that L2 learners often do re-interpret or misinterpret words to fit their own schema (relying on top-down, rather than bottom up processing), but also that they accurately perceive other words by relying on the onset sound. Lastly, he found that learners rely on a third strategy as well, which he calls a “lexical strategy”; in this case, learners bypass the top-down strategy in favor of matching an unknown word with a similar-sounding familiar word, regardless of the word’s semantic appropriateness.

My thoughts: First, let me pat myself on the back for choosing such a brilliant and easily comprehensible article to review. Now, let me see if I can elaborate a bit on the design, and its simple elegance.

Field begins with a history of the opposing views of bottom-up-influence versus top-down-influence proponents; midway into the discussion, in a section labeled “The legacy of scripted materials“, he points out an interesting connection that I hadn’t realized before: many L2 learners have developed an expectation of understanding everything in the text, since listening materials have traditionally been heavily scripted (contrived for the learner’s benefit, rather than reflecting natural spoken language) and graded according to level. Think about that: understanding everything in the text. That means doing the kind of precise and accurate bottom-up processing that is almost impossible for L2 listeners to do in a real-world context. Is it any wonder, then, that some learners panic when they’re exposed to the unpredictability of real conversation occurring at normal speed? Since the EFL world has become more “communicative”, learners must now rely more heavily on their top-down processing skills to hypothesize and compensate for the bits of language that they cannot fully process in the speech stream in real time. So while many scholars insist that L2 learners cannot focus on semantics when they’re unable to catch sounds and segment them into words, other scholars–such as Long (1989) and Field himself (1997)– counter that learners’ top-down processing ability is exactly what enables them to make sense of what they hear. This ability (they say) is what provides the support necessary to compensate for L2 learners’ imperfect decoding skills.

Moving on to the experiments themselves, I’ll briefly explain the design:

The first experiment: This test consisted of groups of four to six high-frequency words likely to be familiar to the learners ( all high elementary or low intermediate level). Sometimes the words all belonged to the same lexical field (for example, desk, chair, lamp, computer), and sometimes only the two last words had a semantic connection (e.g. sunny, excited, bumpy, hot, cold). For the target items, Field changed the onset of the last word only, making a similar word which “didn’t fit” semantically (i.e. sunny, elegant, bumpy, hot, bold). Foils, or examples where the last words were not changed, were mixed in, giving learners examples of target patterns. Fields wanted to see whether what learners expected to hear would override what they actually heard. And what he found was…….the opposite! Out of 18 example sets, listeners only “re-interpreted” one answer to fit their expectations. Well, now, that’s discouraging if you thought you had a good case in favor of top-down processing skills. Field, however, re-assessed his test design and found an important flaw: he had chosen to change the onset–rather than middle or the offset of the target words. Native speakers attach great importance to the initial sound of words, and Field (cheerfully?) proclaims that this in itself is an interesting result, since it clearly shows that L2 speakers also pay great attention to the onset of words, to their obvious advantage.

John Field

John Field

As an aside, I do hypothesize that John Field is a cheerful person. I listened to a podcast of him speaking about his reception of a prestigious award; he describes himself as “gobsmacked” upon hearing the news, which I take to mean pleased and surprised. The surprise implies a very appealing modesty and genuineness. I wouldn’t mind having tea (or a beer?) with him.

So, on to the second experiment, which produced more expected, but this time positive results. This next test consisted of semantically constrained sentences marked by “acceptable but unpredictable” final words. For instance, “I couldn’t listen to the radio because of the XXX”. One might expect an appropriate answer to be noise, but the actual spoken answer is boys. Again, Field was curious to see whether learners would choose a more predictable answer over the what they actually heard, which was less appropriate or expected. This time, results showed that words in seven out of the twenty items were substitutions of “expected” words rather than the words listeners actually heard. But once again, there were unexpected results as well: an analysis of the words chosen showed that test takers tended to change to a word whose onset was similar to the target word (for instance, Field predicted that noise would be the preferred choice in the previous example about listening to the radio; learners, however, preferred to substitute voice, which shares a similar labial onset sound). Aha–further evidence of the importance of onset sounds, concluded Field.

Finally, on to the last and most interesting experiment. This time, subjects were presented with a sentence designed to provide a meaningful context for the very last word….which was a potentially unknown low-frequency lexical item. For instance….”They’re lazy in that office; they like to shirk“. Field hypothesized that learners might be tempted to substitute a more familiar and phonetically similar word, even if the context didn’t fit. In the case of the office question, learners might choose work, a word that didn’t necessarily make sense, but that sounded similar and was associated with offices.

So what happened? Well, as Field reports, “results were striking”. He found that 33.31% of the listeners did not accept the acoustic evidence of unfamiliar words, instead deciding on familiar and phonetically similar ones that often had very little connection with the sentence context. Eliminating those listeners who left questions blank, the number went up to 42.39%. Again, words that learners chose were often not those that Field had predicted. And not only did learners often choose semantically inappropriate words, but they sometimes chose words from different word classes! The trend, he found, did not vary according to individual either, as only one of the forty-seven subjects did not use this strategy at least once. And learners chose to match heard words with known words in each of the 20 test questions. There was no case of a question where a learner had not used this particular strategy. A significant result, right?

It all depends on how you look at it. Field admits that if the data for the third test (not adjusting for those who left questions blank) were to be calculated, it would show that learners’ choice of inappropriate words was not statistically significant (i.e. it was little better than chance) by quantitative standards. But, as he makes clear, such a pronouncement does not take into account “the most striking fact about the figure–that it was achieved despite the evidence of the listener’s ears and, in many cases, to the evidence of the contrary”. Learners, it seems, can sometimes ignore both top-down (the logical contextual choice) and bottom-up (the word as heard phonetically) evidence when faced with a difficult unknown lexical item. The tendency to substitute a known for an unknown word, then, is an entirely different phenomenon which he calls a lexical strategy.

In the end, Field provides insights rather than answers, and that, for me, is the beauty of this article. He hypothesizes, considers his results accurately and honestly, revises and expands his ideas, and finally sheds some real light on the process of listening for L2 learners. A must-read for all TESOL students and language EFL/ESL practitioners, Field’s article is also an interesting and accessible read for language lovers in general. Treat yourself to an enjoyable afternoon with John Field, and you’ll come away a little wiser.

 

Pragmatics and Identity: Problems in Assessment (Class six)

Dr. Jim’s speaking and listening class covered two full chapters of the Ear book this evening; discussion time was generally lively–sometimes bordering on boisterous but never actually crossing that delicate line–and marked by moments of enlightenment, when some concept was either finally understood or found to bear some personal meaning for one of the group. Tonight “my group” was Chunmei ( from Taiwan ), Alan ( from China ), Yae ( from England and Japan ) , Tokiko ( from Japan ) and me ( from the U.S. ).  Hiroko ( in the picture below ) was in the group at the next table, but I wanted to show her bright smile here anyway.

Chunmei and Hiroko keep discussion lively in Dr. Jim's class.

Chunmei and Hiroko keep discussion lively in Dr. Jim’s class.

We had been plowing through different listening-related concepts and negotiating for meaning with mixed success, when near the end of class we hit on an interesting phenomenon that was directly related to the topic of the hour: assessment.

I’d like to elaborate on the interesting conversation that ensued, but first, let me activate your schemata and talk a little bit about assessment. We encounter it on a daily basis: our health is assessed by doctors, our cars are assessed by mechanics, our financial situation is assessed by city officials ( who will tax us accordingly ), our skin type is assessed by flawlessly groomed saleswomen ( who will sell us the appropriate beauty cream ), situations are assessed, damages are assessed, our personalities are assessed,  and so on. Assessment can be objective ( hopefully your doctor will do a thorough and professional examination before he pronounces you either ill or  healthy ) or highly subjective ( say you poke your head into a restaurant, assess the atmosphere, and quickly decide it’s not for you, based on the feel of the place ), and either method of assessment can be valid, depending on the context.

So let’s talk about assessing second language learners, and why it’s such a tricky business. First of all, assessment involves judgement, and judgement implies consequences. The results of high stakes tests  ( such as SAT and GRE exams, or TOEFL and TOEIC exams )  can decide learners’ educational or career paths, and the fair and accurate assessment of the exams is a huge–I would say grave–responsibility. At least once a year in Japan, there is a testing scandal involving students who actually pass their high school or college entrance exams, but are denied admittance. After the fact, it is discovered that the tests results had been wrongly calculated ( and the average listener never learns whether this was an accident or a deliberate mistake ), apologies are extended, and appropriate punishment is meted out. Again, the average listener never learns what happened to the ill-fated student in the end.  I always wonder where they end up, and what kind of emotional baggage they’re saddled with as a result of the “mistake”.

Outright mistakes in assessment–whether deliberate or otherwise–are one thing, but the assessment tool itself ( in most cases, a test ) must be valid for it to be an accurate reflection of the learner’s proficiency. Some believe that no test can ever accurately measure a language learner’s ability. Lee Cronback, known as the “Father of Construct Validity” argued that this was the case. Still, in most formal learning situations, learners must be assessed, and tests must be both valid ( defined as accurately measuring what they purport to measure ) and reliable— ( defined by fair and consistent testing conditions )— or at least as valid and reliable as possible. The tests that Japanese learners are most familiar with are called criterion-referenced: this means that certain scores are equated with certain standards of proficiency or behavior in the subject matter being tested. For instance, a learner who scores in the 9th band ( top  level ) of the International English Language Testing System is one who “has fully operational command of the language: appropriate, accurate and fluent with complete understanding.”  

Think about those four components ( appropriateness, accuracy, fluency, and understanding ) and what they mean for a second language learner. Accuracy, understanding, and fluency are, to me, are more straightforward than appropriateness, since the latter involves pragmatics, which is never straightforward. Some who define fluency as not just speed and facility of production but as “natural use of language” also include an element of pragmatics there as well. Here is where the class became interesting, as our group exchanged stories and gave some thought to the issue of pragmatics and assessment.

The first story that surfaced was my own, and here it is. Eight years ago, I received a call from a friend who had a part-time job with a testing company ( to my dismay, I cannot remember the name of his employer, but perhaps that’s for the best ). His mission was to track down Japanese speakers who would potentially fit the “top band” of his company’s speaking proficiency standard chart, interview them, and submit the results for assessment. The company was in the process of re-evaluating their standards and looking to see if, in fact, those standards were realistic for Japan. “How about your husband? Do you think he’s proficient enough? Would he do an interview?” my friend asked me with some hesitation. “Well, sure!” I said confidently, with all the faith in the world that my clever husband would ace the interview. My husband, one of those rare people who likes tests, was immediately agreeable, and the three of us met at my school on the appointed day.

The first half of the test went smoothly, as the questions were fairly innocuous. My husband was in his element, speaking confidently and accurately, and I was sure he’d pass with flying colors. My clever husband, and clever me, to have found and married such a clever man!  But then–but then–my friend the tester threw a curve ball. “Here’s the situation,” he said. “I’m your co-worker and friend, and I have a smoking problem. It really bothers you. I want you to convince me to stop smoking.” And everything fell apart. My husband is Japanese. Smoking was still not politically incorrect eight years ago in Japan, and even if it had been, one’s habits in this country are one’s own business. The workplace is not where men have heart-to-heart talks about alcohol or smoking habits. Most men here don’t have those talks anyway! I knew instinctively that things were not going to go well with this question, and I was dead right. My husband very compliantly said a few words, smiled, and dropped the ball. “No, no! Pick it up again! Don’t stop there, be persuasive!!” I thought, burning inside.

The interviewer was encouraging: “Oh, you can come at me stronger than that! The smoking really bothers you–come on!” But to no avail. At that point, my husband capitulated completely, proposing ( while still smiling politely ) that if the smoker really wanted to smoke it was okay, and he wouldn’t bother him anymore. And that was that. The interview ended, we ate a box of donuts together, and talked about our respective families. My friend called some weeks later to report that my husband had not fit the criteria for the top band of proficiency. And it was because of the smoking question. “He just didn’t use the language like a native speaker,” was the way my friend explained it.  As I understood it, then, he didn’t use his English appropriately.

And yet the problem was not my husband’s English skill. The problem was that he was put in the uncomfortable position of engaging in a culturally unnatural and unfamiliar type of discourse. He was sideswiped by his cultural identity rather than his lack of vocabulary, fluency, or comprehension. Literally left tongue-tied. Too bad, I say, but at least the situation was voluntary, with no “high stakes” involved. Not counting, of course, my husband’s pride and my own high expectations.

As soon as I had finished my story, Chunmei chimed in with her own. “I can never get my students to do role plays about smoking, either!” she exclaimed. “I ask them to persuade me to stop smoking and they clam right up and give me a funny look!” We then agreed that skipping the role play ( the book lesson ) and getting students involved in a discussion about the topic would be much more valuable. And we wondered about pragmatics and assessment.

In the case of my husband, meeting the criteria for a “top level of proficiency” required him to behave in a decidedly un-Japanese way. He was judged to be a non-native level English language speaker because his reactions were Japanese. And this, of course, is the heart of the matter: is such a language proficiency assessment valid? Recall the IELTS Band 9 criteria: appropriateness, accuracy, fluency, and comprehension. Again, “appropriateness” is the tricky component, and ( according to my friend ) it was a component of the test my husband took as well.  As I see it, his approach to the smoking question was perfectly appropriate within his own culture. The interviewer did not specifically say, “You’re in the U.S. Address this situation like an American!” Of course, if he had, it might not have made much difference in my husband’s answer, but at least the expectation or the context might have been clearer. And perhaps, given the ambiguity of the question and the informality of the interview in general, the problem was one of reliability rather than validity.

The Ear Book (chapter 10 ) presents a chart of assessment models that I found both interesting and enlightening. Specifically, Rost gives a visual overview of criteria for assessment according to the purpose of second language learning ( EFL vs ESL vs English for Young Learners vs English as a Lingua Franca ). According to this chart, my husband could probably get top scores in an assessment of English as a Lingua Franca, since this is the only category that specifies “expected to maintain national identity through English”. Still, if not for the pragmatic component, I believe he would qualify as a top-level speaker in an ESL assessment as well.

So what’s the moral of the story? I really don’t know. I consider my husband a native-level English speaker, but that’s my own subjective assessment. In the long run, I’m engaged in an ongoing attempt to understand the delicate balance between mastering a second language ( which means tackling the pragmatics and taking on a new identity ) and retaining the foundations of one’s own cultural identity at the same time. Happily, there’s no rush, because I think I’ll be busy with this one for some time to come.

The Drama of Semantic Processing (Class 3)

Dr. Jim’s–he said we can call him whatever we want– third class was about semantic processing. For me, it was all about the drama of everyday communication. For instance, you think you’re just chatting with a friend, but actually you’re uttering intonation units referring to either given information ( which you assume is currently active in the listener’s memory ) or new information ( which might in fact exist in the speaker’s working memory, but you assume is NOT currently active ). This is interesting, but not so amazing. What’s amazing is this: according to chapter 3 of the Ear Book, we signal whether information is given or new by using our voices, with rising tones signalling given, or background information, while falling tones signal new, or focal information. After testing this in class by attempting to tell Hiroko a story involving facts she knew ( about my daughter, Sumire ) and facts she didn’t yet know ( about my cousin Cathy ), both of us decided that if such tone variance exists, it must be extremely subtle, and difficult to consciously reproduce. Try it and you’ll agree.

Taking the subject of given and new information a bit further, we know, of course, that the speaker’s assumptions about what is or isn’t active in the listener’s memory are not always correct; this results in one of two equally frustrating situations. The first would be the speaker who laboriously and painfully explains every twist and turn of a story whose details you are actually quite familiar with ( unnecessary background information ). The other extreme would be the over-excited speaker, jabbering away at top speed and unaware that you lost the thread of the story a full five minutes ago ( unwarranted assumption of a familiarity with background information ). The lesson to be learned? Do not pontificate, do not babble, and do not charge full speed ahead without verifying that your assumptions about the listener’s degree of comprehension are correct. The well-mannered speaker is advised to confirm his own comprehensibility. I recommend pausing to ask, “Do I need to tell you this or are you up to speed?” or “Hey, buddy, your eyes are glazing over….are you following?” On the listening end, things might be facilitated by a bold-but-polite interruption: “Errrr….you told me that story last week, remember?” or “Whooooooah, Nellie! Slow down and back up to the part where your cousin ate what she shouldn’t have and All Hell Broke Loose. I don’t have a clue what you’re talking about!”

And since the topic of communication is far from exhausted, let’s continue with “types of understanding”, six of which are neatly analysed by Rost. From a linguistic point of view, it’s all about how our schemata match up. The best case scenario would be complete understanding ( the listener activates schema that are completely shared with the speaker ), and the worst would be not a “mis” understanding, but a non-understanding ( the listener cannot activate any schemata at all to understand the speaker ). A misunderstanding, then, is when there’s a mismatch between the listener’s and the speaker’s schemata. Here’s an example: before my husband and I were married, he took me on a trip to a resort town in Japan called Karuizawa. “It’s just like camping out in the forest!” he said confidently, sure that I would love it. My “camping” schemata activated, I pictured nothing but trees and worried that we did not have the proper gear. But no worries–when we arrived, I saw that Karuizawa was full of fashionable women in white tennis skirts, bars and restaurants, and even an amusement park! “Just like camping” for my husband meant that it was a cool, pleasant place to relax away from Tokyo. He had actually never been camping in the wild, and our schemata were wildly mismatched. 

Crown Prince Akihito and Michiko Shoda in Karuizawa, 1959. (Photo from Sankei Archive)

Crown Prince Akihito and Michiko Shoda in Karuizawa, 1959. (Photo from Sankei Archive)

There are also degrees of partial, plausible, and acceptable understanding, depending on how much overlap is shared by the speaker’s and the listener’s relative schemata, but let’s return to the state of complete understanding. We all have friendships where communication is so easy it’s like play; according to prototype theory as explained in the Ear Book, this is because we share common activation spaces in our memories. No two friends can share exactly the same schemata, but two friends who share similar world views can easily understand and empathize with each other because they share a similar neurological programming. And when the speaker’s words activate relevant knowledge in the listener’s memory, there’s actually a cognitive commitment involved, which produces an empathetic response and further connects speaker and listener. We don’t just listen to our best friends, we make a cognitive commitment, and I like that thought. 

As a matter of fact, some of my classmates are not so thrilled with all the detail in the Ear Book. I can understand why it makes some folks tear their hair, but so far I’m liking it. I enjoy picking things apart and breaking down concepts, and it looks to be a steady diet of this kind of reading until mid-book, when it gets “more practical for teachers”, according to Dr. Jim. The next chapter I’ll be writing about is on pragmatic processing, which looks to be pretty juicy. If it’s not, I’ll do my best to spice it up, so no worries. Good night, from the Godzilla Hotel in Tokyo (really! ).

Neural Committment, Perceptual Goodness, and Updating our Schemata (classes 1 and 2)

An hour or so of class discussion time didn’t really do justice to the first two chapters of Michael Rost’s Teaching and Researching Listening (which will be referred to from here on in as The Ear Book). At a recent JALT conference, I saw a teacher reading this text “for fun”–his own words– and I now understand why. It IS fun, not just because it’s extremely readable, but because it connects language acquisition theories and experiments with biology and neuroscience. Every TESOL student studies experiments related to critical or sensitive periods in language acquisition and discusses “plasticity”.  It’s interesting, then, to read about the process of neural commitment on page 23 of the Ear Book: 

As basic linguistic functions develop, they become confined to progressively smaller areas of neural tissue, a process called neural commitment. This leads to a beneficial increase in automaticity and speed of processing, but it also results inevitably in a decline in plasticity……It appears that the process of neural commitment leads to a neural separation between different languages in bilinguals and second language learners. The plasticity or neural flexibility required for language reorganization declines progressively through childhood and adolescence and may be the primary cause of some of the difficulties that adults face in second language learning.

What this means to me is that we do lose plasticity as we age, but hey, it’s a trade-off: we gain automaticity and speed (which means greater fluency). That’s pretty important, and actually a good consolation prize; whether the fluency referred to is in one’s mother tongue or in multiple languages, our brains are wired toward fluency, and decline in plasticity is simply an unfortunate result of that process. I also like the image of bilinguals’ different languages as being neurally separated ( like an obsessively neat person’s sock drawer, I imagine ). Mothers of my students would be relieved to know this, since many of them imagine English as something that will interfere with their child’s Japanese learning process ( imagine all the socks unballed and in a colorful mess in the drawer–that’s what they envision ). My husband and I are both late and successful second language learners who raised two fully bilingual children; between the four of us, we’ve had no trouble using either Japanese or English or mixing the two when necessary. The different language systems are firmly established in our minds; they don’t interfere with each other and they often combine to make for a more colorful style of communication.

On to chapter two of the Ear Book, which reveals that a linguist’s favorite snack is full of Perceptual Goodness. No, not really, but that’s the first thing I thought when I saw the phrase used on page 27: it sounds like a TV commercial. Actually, it refers to sound. I learned that phonemes have “identities” that clearly define them, as follows: “…..each individual phoneme of a language has a unique identity in terms of frequency ratios between the fundamental frequency of a sound…and the frequency of the sound in other harmonic ranges.” So, as I understand it, in pronouncing certain phonemes we are unconsciously manipulating sound frequencies. Although there’s a broad range of acceptable ratios between frequencies for each phoneme, if we move out of that range, the phoneme loses its identity (becomes unrecognizable or unintelligible). In other words? When a non-native speaker’s pronunciation is so far off that the listener can’t decode certain words, it’s a matter of mathematics. The ratio of the sound frequencies produced by the speaker is mismatched (not purposely, of course), and the listener must resort to guesswork or hypothesis (top-down processing) to comprehend what’s being said. It’s a very physical explanation for what defines phonemes and how we identify them. 

There are numerous blog-worthy topics detailed in chapters one to three of the Ear Book, but I particularly enjoyed thinking about Schemata. That’s one of those words that we’re familiar with, but perhaps in a fuzzy way. Well, no more fuzzy thinking now that I have Rost’s definition to work with. Here it is: “A schema is a figurative description for any set of simultaneously activated connections (related nodes) in the vast frontal cortex of the brain.” Schemata, then, refers to a set of “memory nodes” that are activated constantly as we attempt to understand the world around us, make decisions, and communicate with others. Read the morning newspaper? You’ve probably created a new schema by processing new facts and linking them together. Just met your new neighbor and exchanged greetings? In that short time, you’ve put a simple schema in place. Even more interesting is the fact that our existing schemata are constantly being undated as we read more, see more, hear more, experience more, and revise our former schema. In linguistic terms, this is being “parsimonius” (following the principle of Occam’s razor), since it is obviously a pain in the neck for your brain to be constantly creating entirely new schemata, and updating is much more efficient. I’m fairly certain that my mother-in-law is constantly updating her schemata in regards to me, her son’s American wife. And that is for the good, since her first schemata was rather alarming and not at all positive: the image of an American bride (she has since confessed to me) triggered fears of early divorce, wild and crazy spending, and greasy, unhealthy cooking. 

And that is enough reflection for now. I vow to not let my schemata get rusty, but to continue updating and creating nice shiny new ones.

 

Post Navigation