Sunday, 23 April 2017

Red flags: Assessing sources

This was originally posted on 'The Scarlet Onion' webpage but it has since ceased to exist :( 

I once had an online 'discussion' with a chap claiming that the twin towers were, in fact, brought down by the US government and that 9.11 was all an inside job. He sent me a link to, in his words, a ‘peer reviewed academic journal article’ to back this theory up. 

The link led to the ‘Journal of 9.11 studies. One of the editors was Kevin Ryan who coincidentally was also the article’s author. Editors do occasionally write articles for their own journals but still...I was dubious. So I googled Kevin Ryan and found that he’d written a book on why 9.11 was actually 'an inside job' and was fired from his previous university job his views. The article was about Nano-thermites (which he said the government used to blow up the twin towers), a type of explosive, and yet Ryan was working on water testing. Things didn’t add up. 

O Red Flags  
Imagine going to a restaurant and seeing no customers, the paint peeling and a smoke coming from the kitchen. None of this necessarily means the food there is bad, -hey it might be great, but these are all the kind of things which Dorothy Bishop refers to as ‘red flags’. These red flags can acts as a kind of early warning system. 

This brings me a conversation about the relative pros and cons of learning styles/MI and the Montessori Method I had on twitter earlier in the year. I was sent a link to a series of educational articles by a teacher who thought they supported her case. When I read them i noticed a number of red flags. I thought it would be useful to blog about these and hopefully this will be useful for other teachers assessing the quality of sources. So here are a few things to look out for.

O Mode of publication
Is the article in a book or on a website or in a journal? 

This matters because anyone can say anything in a book or on a website. Books often seem impressive as if somehow putting something in a book makes it more weighty and serious. Usually, academic journals, which have been through peer-review, are more likely to have credible information than books. Websites are almost always a no-no as there is usually zero quality control. Disclaimer (of course it always depends on exactly what you are looking for and what kind of website/book it is).

What about the papers that I was directed to look at? They seem to be on a website, which is an initial red flag. However, when we get to the articles in the download section they actually seem to be from real journals, namely, the MASAUM Journal of Reviews and Surveys (volume 1), MASAUM Journal of Open Problems in Science and Engineering (volume 1) and the International Journal of Engineering Research and Applications (volume 2); so far so good. But O (first red flag) why publish education articles in journals of engineering? 

O Dubious journals  
So these articles are in journals but what kind of journals?

Ideally the journals would be Peer-reviewed, and well established. The first two of these journals fail the ‘well-established’ test as they are all only seemingly on their second or first volume and are not available anywhere on the web. The third one actually does exist and looks (roughly) like a journal should. The site looks a bit cheap and unprofessional (Gmail address for submissions etc) but it exists, which is a start.

O Pay for play
A little bit of digging around the FAQ section of the third journal and we find this:

Q: How much do I have to pay for publication fee
A: $150

Paying for publication isn’t a great sign. As it’s an open-access journal (anyone can view the articles for free) this doesn’t necessarily mean it’s a bad thing, as long as the quality control isn’t affected. If we go to the ‘stats’ page though, we can see that they accept roughly 40% of all submissions. This is quite high considering journals like ‘Applied Linguistics’ accept only about 10% of submissions. It’s also notable that they record stats monthly and from those we can see they accept about 250 papers each month which means each volume has well over 300 articles. For context that’s about 7 years’ worth of ELTJ articles in one volume. 

Peer-reviewed
Peer-review is sometimes mistakenly thought of as the ‘end’ of the process. But actually all peer review means is that a couple of other people in the profession have read it and think it’s good enough to be published. This doesn’t mean it’s perfect, or that no one can question it, just that it has reached a certain level of acceptability. After peer-review, the academic community at large get their teeth into and then we often see criticisms, repeat studies and sometimes retractions. The International Journal of Engineering Research and Applications is actually peer-reviewed so that seems pretty reassuring. Except, considering they boast that peer-review only takes 4-6 days (not the couple of months journals usually take) and because they read about 800 papers a month and only have about 27 reviewers, they are getting through those papers at an astonishing rate! You have to worry about quality control. 

O The author 
If you’re at all in doubt you can always check out the author’s credentials. According to his website Dr. Qais Faryadi (or ‘Dr. Prince’ as his website calls him) has a PhD in computer science and a master’s in Sharia Law as well as being an expert in curriculum design, criminal law, software engineering and Islam. He lectures on all these subjects and has even published a number of books (or rather ebooks) on such diverse topics as teaching, Islam and “Magnesium The Health Restorer: The Missing Link To Recovery”. 

O The nose-test
When you actually start reading the paper, does it sound plausible or like an academic paper should? 

Dr. Prince’s starts his learning styles paper with the statement that:

This evaluation examines teaching and learning from the lenses of mind blowing scholars such as David Kolb, (1984), Honey, (1982), Dick and Carey model (1990), Anthony, Sudbury Model, VAK Model and Madeline. (Faryadi, 2012:222)

Language like this should set alarm bells off instantly. This paper isn’t going to be an unbiased review of learning styles, not when the writer believes their creators to be ‘mind blowing’. Not only this, the article is littered with errors. In this small section alone we have such oddities as:

  •        ‘Honey’ should be ‘Honey and Mumford’ (as in the reference section),
  •        He starts by talking about ‘scholars’ but then switches to ‘models’
  •        Anthony? Who he?
  •        Madeline? Who she?


Wouldn’t peer-review usually sort errors like these out? Then again, with reviewers getting through 50 or so papers a month in 4-6 days each, it’s perhaps not surprising that the quality suffers.

The article itself is a relatively unremarkable laundry list of things the author believes ‘good teachers’ should do. There is no attempt to critically engage with the various learning styles models presented or to talk about why and how they differ from each other.

So in conclusion we have sloppy education articles written by an expert in computer science and law who claims expertise in a number of other fields. They were published in science and engineering journals that either no longer exist or have very little reputation. The one journal that does exist, charges for publication and has numerous credibility issues.


The food may be great here, it really might…but I’m going to look elsewhere. 

Thursday, 29 December 2016

Enough of experts



"...anti-intellectualism has been a constant thread winding its way through our political and cultural life, nurtured by the false notion that democracy means that 'my ignorance is just as good as your knowledge.'" 

Isaac Asimov, Column in Newsweek (21 January 1980) 


It's been an interesting year for evidence and belief. No, scratch that, it's been a depressing year. We've been shown, and left in no doubt, that people generally do not care about facts, truth and reality, and would rather stick to what feels right to them. The name for this phenomena is 'post truth', (Oxford's word of the year) which is defined as:
Relating to or denoting circumstances in which objective facts are less influential in shaping public opinion than appeals to emotion and personal belief.
So it seems, we’re now living in an era of ‘post-truth’. This came as something of a surprise to me since for the last 5 years I've been writing about linguistic and teaching practices that are widely believed despite having little or no evidence to back them up


I work in EAP which is similar to language teaching but with an academic twist. One thing we have to do is to insist student know the value of criticial thinking and supporting ideas with evidence. We drill 'where is your evidence for this claim' and 'how do you know this?' Here are some of the things my EAP colleagues have told me over the years:
  • Climate change is a hoax. 
  • Microwave ovens destroy the nutrients in food
  • The moon landing didn't happen
  • Wifi causes cancer 
  • Horoscopes are credible 
  • The earth is only six thousand years old
The people who espouse these views are well-educated and thoughtful people. They are not alone in holding beliefs like these. For instance, in order for homoepathy to work, physics would have to be wrong and yet it is a 6 billion dollar a year industry. So to my mind, we haven't suddenly slipped into a 'post truth' era, we've been living here for quite some time. Perhaps the only difference is how in-your-face it is, now? And are things really any different with regards to education?

Enough of experts
Brexiter Michael Gove when confronted with the fact that almost every economist thought Brexit was a bad idea said simply "I think people in this country have had enough of experts". Turns out he was right.

The denigration of experts is nothing new. Climate scientists have predicted dire consequences for us if we continue to put CO2 in the atmosphere. 97% of scientific institutions worldwide agree that human activity is causing this problem and yet 52% of Brits don't believe them (there's that number again). They've had enough of experts, they know better. 



In a discussion with an ELT teacher about learning styles (see picture) the person in question told me that no amount of research could dissuade her of the notion that learning styles were real. She actually tweeted "Why I believe in learning styles despite what researchers say". How much more 'post truth' could you get? 


At IATEFL one teacher trainer stated that he thought the TEFL world was getting too obessed with searching for evidence and trying to prove things. I found this quite a surprising claim since education seems to be one of the least evidence informed professions I can think of. 

He went on to say he was suspicious of any claims that a teaching practice could be said to be proved to work. Interestingly, he also argued that teachers should just try something out in class and then reflect on whether it worked or not. I couldn't help but wonder since 'nothing can be proved to work' how teachers were supposed to know if what they tried in class had been successful or not. 

Speaking of experts Brian Cox recently said:
“[cynicism towards professional expertise] is entirely wrong, and it’s the road back to the cave. The way we got out of the caves and into modern civilisation is through the process of understanding and thinking. Those things were not done by gut instinct. Being an expert does not mean that you are someone with a vested interest in something; it means you spend your life studying something. You’re not necessarily right – but you’re more likely to be right than someone who’s not spent their life studying it.
Caves are dark places, but they're also warm and safe. 



Monday, 21 November 2016

Movie Review: Arrival

I took a trip to my local picture house recently to view one of the talkies. I've not reviewed a movie before but as this one touches on linguistics I felt like it wouldn't be too out of place on this blog. 

There will be **spoilers** below. 



You have been warned. 





Arrival is a sci-fi flick in which Aliens come to Earth and humans try to communicate with them. The hero is a linguist. Predictably this has linguists very excited. Ben Zimmer, for instance has written that the film "does a remarkably good job of depicting how a linguist goes about her work" and Rob Drummond made a similar point on 'Kermode and Mayo's Film Review'. David Adger noted that "The linguistics was very good". 

I can understand the excitement at having a linguistics prof as a hero and I have to agree that it was a well made film. However, the contrarian in me wasn't able to look past some of the linguistics and just enjoy the film. In fact, the whole central premise of the plot falls to pieces because it just isn't backed up by what we know about languages.  Yes, I'm being a kill joy. Yes it's just a movie. But here's a list of the three things that bothered me in the film. The first two peeves are relatively minor, the last one was a clanger. 

The parts that rankled were the following (apologies if I've mis-remembered any parts). 

*peeve 1: linguist means linguist means linguist. 
Our hero was introduced as a linguist. If you're not sure, in academia that means 'someone who studies languages'. It doesn't mean 'someone who speaks a lot of languages'. Those people are usually called polyglots. Our hero is a linguist and a polyglot, speaking Mandarin, Farsi, Sanskrit and is shown giving a lecture on "romance languages". 

OK, sure, some linguists do speak a lot of languages to varying degrees. But the film starts with the military thanking her for a Farsi translation. Translation is a hard job and linguistics is hard job and there isn't that much overlap between them. Though, maybe she's just insanely talented and that's how she pays for her ridiculously expensive house? To her credit the linguist who assisted on the film fought this:
Coon unsuccessfully lobbied the filmmakers to change a line describing Louise, arguing that it misrepresents what linguists do: “You’re at the top of everyone’s list,” Forest Whitaker’s Army colonel says to Louise, “when it comes to translations.”
So well done and bad luck to Jessica Coon. By the by, The Guardian, rather densely asks why Chomsky wasn't a consultant on the film:
Why, you ask, did they not approach Noam Chomsky, with his understanding of “deep structure” in language? Perhaps Prof Chomsky did not care to help America’s military-intelligence complex
Putting aside the fact that Chomsky's work has, in fact been funded by "America’s military-intelligence complex" and that he famously dislikes the kind of field work linguistics that the hero of the film is involved in, it is not clear to me how knowledge of 'deep structure' would have helped a linguistic dealing with a language they had no knowledge of. 

*peeve 2: The white board scene 
Our linguist hero is trying to convince the army that her method is the right way to go about things. She writes on a white board:
What is your purpose on Earth? 
She attempts to explain that in order to ask the aliens questions, she first needs to make sure they know what a question is. She goes on to make other points concerning things like vocabulary. This is all fine and quite sensible but there are a few clangers here. first she says (paraphrase) 'we need to make sure they understand the difference between you singular and (crosses out the 'r' in 'your') you plural.' In English there is no difference between singular and plural you (I -we, s/he -them, you -you). 

She then says 'we need to make sure they understand 'why questions'. Now I've heard of 'wh' questions, but 'why questions'...? The sentence on the white board seemed to be lacking an essential item in what could be defined as a 'why question'...that is, a 'why'. Perhaps the script originally had the question as 'why are you here?' who knows. 

Next, (and this is really nit picky), if we're talking to aliens, do we really need the words 'on Earth'? I mean, are these aliens going to get confused by 'what is your purpose' and think we might be asking what their plans for Sunday afternoon are? 

*peeve 3: Eskimo's and their many many words for snow. 
All of the other issues I could have happily put aside. However, when the words Sapir-Whorf were mentioned I tensed up. It's hard to imagine something annoying me more than learning styles...but here it is. With this in mind, why have I never written about it before? It's such an attractive and widely believed idea, that I really felt I needed quite a powerful response to it...and I'm really quite lazy. Linguist John McWhorter wrote an excellent book on the topic called 'the Language Hoax' and even he was reluctant to criticise the movie: 

If you're not familiar with Sapir-Whorf I'll present a very reductionist overview here. Basically it can be divided into two version. The first (which is the most attractive to click bait headlines) postulates that the language we speak allows us to think certain thoughts. It's also known as linguistic determinism. The second is the idea that the language we speak influences the way we think. This is known as linguistic relativism

There's a lot to say about this, but the TL:DR is that the strong form while being very attractive is patently false, whereas the weak form has some empirical support. The idea that the language you speak makes certain thoughts unthinkable (the strong version) is so seductive, it's very hard to resist.  

Language can obviously influence though (like if I ask you to think about elephants and you do then my language influenced your thought) but it would be a brave soul who argued that the lack of a word for 'he' and 'she' in spoken Chinese means that Chinese people have no concept of whether they are talking about a man or a woman. One would have to wonder how we ever come up with concepts and then name them were this theory true. 

The most commonly known manifestation of this theory is the ever expanding number of words Eskimos are alleged to have for 'snow'. It started off as about 7 words in 1911 and reached 100 by 1984. All of this leading Geoff Pullum to pen his classic 'The Great Eskimo vocabulary hoax'. This even led to the creation of the term 'snowclone' which describes "some-assembly-required adaptable cliché frames for lazy journalists" such as "If Eskimos have N words for snow, X surely have Y words for Z.". 

The Eskimo Snow theory is attractive because it posits that Eskimos are surrounded by so much snow that their language represents their increased sensitivity to it, while us dumb Westerners just see all the subtle variety and diversity as plain old 'snow'. It's an attractive theory and the original Sapir-Whorf hypothesis had good intentions. Back in the days when Latin was considered the pinnacle of language, Sapir introduced the idea that rather than other languages being 'primitive' these languages offered insights into the world we would never be able to even perceive:
“Human beings… are very much at the mercy of the particular language which has become the medium of expression for their society... The fact of the matter is that the ‘real world’ is to a large extent unconsciously built up on the language habits of the group.”(Sapir

 It was social justice of its day, but sadly it was not true. 

The film arrival takes the strong form of the theory to its logical conclusion. Learning the language of the heptopods, literally changes the protagonists outlook to an incredible degree. The Alien language is represented in circles and we are told the aliens have no concept of ordinal numbers. This, we later learn is because the aliens do not share our concept of time. In fact, the heptopods experience all time at once. Thus their sentence appear in circles. Once the protagonist learns their language she starts to be able to see into the future. The language she learns literally changes her perception of the world. 

How would circular sentences lead to a timeless world view? It's hard to say anything about an alien language but as it is translatable, in the film into English sentences like 'we bring a tool' there are certain things we can say. Firstly, word order is essential in English. 'We bring a tool' is a lot different from 'a tool brings us' or 'Bring us a tool'. If there was no case marking in the alien language (for instance 'I' is the subject in English but the same word is 'me' when it's the object.) the there would have to be a lot of guessing as to the meaning of the sentences. If there were markers of case, then the circular nature of the sentence is really just an artistic flourish. 

Lastly, at what point would someone learning a new language obtain the ability to see into the future? After the first lesson? after a few months? Would it happen all at once, or gradually? People don't tend to hit a point at which on Tuesday they were OK at French but on Wednesday they were fluent. So when does the new world view kick in? 

To its credit the film handles this point quite well. As Amy Adams eventually learns a language which makes the concept of 'time' disappear, so it follows she must have always possessed the ability to speak the language. The her in the past who couldn't speak the language exists in the same time as the her who can speak the language. But presumably they also exist simultaneously with the her who couldn't. Best not to think too much about this. 

In short the language can't do what the film proposes it does and even if it could it wouldn't lead to mind altering powers. But really....who cares? It's a good film go and watch it! 








Wednesday, 28 September 2016

boooooo! hurrah!



Penn and Teller's show 'Bullshit' was a favourite of mine. Every week they debunked commonly held beliefs from 12-step-programs to cryptozoology. In one particular episode they asked people to sign a petition to ban Dihydrogen monoxide -a substance found in 'pesticides, baby food and the water supply'.

Hundreds of people signed up to demand the government ban H2O, more commonly known as water. So why would someone want to ban water? Probably because it was presented to them as a scary sounding chemical and 'chemical' is for many people a 'boo' word. 

'Boo' words, and their opposite 'Hurrah' words come from an old theory called Emotivism which holds that "ethical sentences do not express propositions but emotional attitudes". I'm not too concerned about the philosophical theory but I rather like the notion of boo and hurrah words. Put simply boo words are things that are just accepted as bad, and hurrah words, the opposite. When we hear 'Chemical' we mentally relegate it to the pantomime villain category and boo accordingly. 

So what are boo and hurrah words in education? Swan noted that:
the applied linguistic equivalents of democracy and motherhood - include 'learner-centred', 'meaning based', 'holistic', 'discourse', 'discovery', 'process', 'interaction', 'negotiation' and 'strategy'. On the other side of the communicative fence, concepts related to 'bad' pedagogic attitudes felt to be discredited and undesirable include 'teacher-dominated', 'form-based', 'discrete', 'sentence-level', 'transmission model', 'product', memorization', 'repetition', and 'drill'. (2009:167). 
I would probably add 'testing' and 'textbooks' to this list. These words are often placed in 'boo' or 'hurrah' boxes and there they linger with little examination. And it's not just ELT, as a comment on the now defunct 'Web of Substance' blog wryly notes:
I am disappointed in you as well Harry. You should know by now that, in polite education society you label your OWN ideas as "authentic", "innovative", "Child-centred" and "21st Centruy" so that when anyone disagrees they are, essentially, arguing for a counterfeit, old-fashioned, child-hating, Victorian education. 
We often take our views 'off-the-peg', after all, none of us really have the time to go and read up on every single subject which may concern usWhat, for instance, is the link between wanting relaxed gun laws and thinking climate change is a hoax? Seemingly nothing, and yet (American) people with one of these views will often have the other. Have these people really reasoned out the pros and cons of each side, or have they adopted the views of the 'tribe' they most identify with? 

What this boils down to is ideology. Once we choose an ideology to follow, be it socialism, Islamism or environmentalism, we reshape reality to fit that frame. A petition to ban a chemical? Sure, where do I sign!

Is this a problem? As long as our chosen ideology is sound, the views that follow will also be sound, won't they? Perhaps. But I'm uncomfortable, for two reasons. 

Firstly, our views are often unexamined. I can't speak for other teachers, but I often find a lot of the TEFL discourse confusing because I can never sure the terms people are using mean the same thing to them as they do to me.  

Take for instance the discussion on PowerPoint on the Minimal Pair podcast. One of the presenters said something about trying to avoid using PowerPoint because they're so 'teacher centric'. Thassumption in this statement is that 'teacher centric' (whatever that means) is bad and should be avoided. I kept thinking, 'are they teacher centric and if they are is that a problem?' 

Secondly, we've seen this go wrong before. Learning styles rode an ideological wave to success. It is an appealing notion to imagine that every learner has their own special abilities and if we just teach them in the right way, tapping into their unique 'intelligence' they will flourish. It's certainly more appealing than the notion that some people are just smarter than others and will do better than them no matter what we do. Learning styles is attractive, ideologically, but unfortunately its not true. 

Alan Waters, who passed away recently, wrote several articles examining ideology in applied linguistics noting that "a good deal of its discourse promotes or proscribes language teaching ideas on the basis of ideological belief rather than pedagogical value." A view supported by 40 years of learning styles promotion. Dana Ferris, who is perhaps the leading scholar in written error correction notes that, on largely ideological grounds "composition theorists have for decades ignored, minimized, or even openly disparaged any issues related to error treatment in writing courses." (2011:61) And Hyland suggests that although process approaches to writing may be appealing "there is little hard evidence that they actually lead to significantly better writing in L2 contexts." (2003:17-8)

These examples make me wonder, what teaching practices we are currently being ignoring because they don't fit our ideology. And likewise, what teaching practices are popular because they appeal to our world view? Is a teacher-centric lesson bad because it limits learning, makes students unhappy and is boring, or is it because it's 'authoritarian' and 'traditional' while we are modern, democratic, freedom loving sorts? Is there a difference between claiming you teach in a 'a learner-centric, communicative way using only authentic materials' and say claiming that you only eat 'organic, gluten free, locally sourced, food?'

Walters wrote several papers on this theme, taking quite an extreme position at times. He claimed, for instance that the EFL world engages in a kind of Orewelian 'newspeak' where unacceptable views are supressed  and only, "approved’ ways of thinking, such as in the use of the term ‘authentic’" are acceptable. (2015) He argued that getting rid of textbooks or advocating learner autonomy or ELF are not just pedagogical choices, but markers of right thinking people

And perhaps he has a point. Are textbooks disliked more because they present materials in pedagogically unsound ways or because they are written by large companies who make lots of money? Arguably it's a bit of both. So how do we stop ideology slipping into our teaching? I think it's important to carefully scrutinise our beliefs. The first step would be making sure we have a clear and accurate definition of what it is we're talking about. Take autonomy for instance, most teachers would consider it a good thing but as Mike Swan noted at a recent talk, while autonomy can certainly be good, the logical end point of autonomy, is no teacher. 

Next, we need to examine our biases, -what would we like to be true. I correct my students mistakes in class. Therefore I hope that that helps them learn. If I found out it didn't help them, -even hindered them, I'm likely to feel pretty bad about that. Therefore, I have a vested interest in trying to find data that back that view up. I'll also fight harder against, and examine closer articles which contradict that view.

Lastly, we should ask ourselves what our beliefs about teaching are based on. Do you teach the way you do because it's the way you were taught to teach, or because it's how everyone else teaches? What reason do you have to believe the things you do and more importantly, what would it take to change your mind. If the answer to the former is 'I just know' or 'common sense' and the answer to the latter 'nothing' then what you are describing is dogma. 

A chemical like H2O may save your life or, like H2O2 it might be poisonous. Chemicals themselves are not inherently bad, and H2O2 is excellent for dying hair while water may drown you.