Known as the third largest English speaking nation in the world, business and knowledge process outsourcing is booming in the Philippines. But how do you know if your candidate is proficient enough to carry out the job efficiently and the language skills to interact effectively? In this week’s Curran Daly blog post, we take a closer look at language testing.

Automated Language Testing

BPO Employees must have solid English speaking abilities. However, assessing the language skills of thousands of job applicants can be costly and time consuming. With the help of automated language testing, the English ability of job applicants can now be measured in a quick, cost-efficient and standardised way.  The BPO industry comprises of a variety of back office functions, support services and call centres which are outsourced to sites that are more economical to run than those at home. Complex information and services must then be negotiated within the constraints of the telephone and screen through English, as a common language.  Not just by communicating interpersonally, but in the understanding of information passed through in English for certain business processes.

How does a computer grade a test?

There are two approaches for automated language testing –

1. Natural Language Processing (NLP)

   wherein the software identifies and counts linguistic features and does not attempt to gauge content in any way. It is more commonly used for testing writing skills.

2. Speech Recognition

   wherein the software compares speech samples to a large database of samples of the same test question(s). Faster responses are considered “more fluent”. It is more commonly used for testing speaking skills.

Testing System Examples

1.     Versant (Pearson)

• First fully automated oral language test used commercially
• As the leading test of spoken and written language, Versant is being leveraged by leading corporations, government agencies, and academic institutions to provide a fairer, consistent, and efficient option for evaluating language and literacy skills.
• Versant is also a recognised leader in the BPO/KPO industry.
• Test is taken in a computer lab using microphone and headset, or over the telephone
• Computer automatically rates the speech and produces scores
• What a Versant test is like:
• About 15mins long
• Several simple task types – repeating sentences, scrambled sentences, “oral multiple choice”, all responses totally scripted
• Optional “free response” final question (not scored, but saved for reference)

2.     E-Rater (ETS)

• Automated scores of timed essays
• Uses NLP
• Currently used in limited way to rate:
• Students turn in essays, receive scores, revise, repeat
• What E-Rater does with an essay:
• Global measures – Counts total words, total sentences, sentence length, # of paragraphs
• Vocabulary measures – # of unique words used divided by total words (lexical diversity) , # of low-frequency words (lexical depth), # of prompt=specific words (topic appropriateness)
• Grammatical measures – dependent, independent clauses, passive voice, subject-verb agreement, etc.

Main Advantages of using a fully automated language testing systems:

• Reliability – computers don’t get tired, they are not biased for or against individuals; scores are more consistent that with human assessors (Bernstein, Van Moere et al 2010).
• Practicality – automated scoring is much less expensive than human rating; scores and feedback is also obtained instantly (Farnsworth, 2013). You are able to reduce the time and man hours you need to find talents. You can also screen and hire candidates anywhere and at any time as you are able to view scores generated online within minutes.

Machine-human correlation

One of the common criticisms of automated languages tests is that a machine cannot evaluate speaking skills as well as a human can. Another criticism is that these tests do not measure communicative abilities because there are no interaction exchanges between live participants. From these remarks, we suggest that the use of these systems while still invaluable should be supplemented with a more conversational evaluation perhaps during the interview stage.

Over to you, what do you think about automated language testing? If you are already in the BPO industry, what are your tips for ensuring you hire those with a high-proficiency in English?