Any member of RID is aware of all that rides on an interpreter becoming certified. It is the gold standard. Once achieved it is the stamp of approval that permits an interpreter to enter a new phase of his/her career where they are able to take consistent work at a professional rate of pay. It is supposed to be an achievement worthy of respect and in many states, licensure. With so much riding on an interpreter passing this test, it should have meaning. It should represent a standard of excellence that is uniform throughout the profession. Most of all it should be an assessment that is thorough and can withstand scrutiny.
Sign Language Interpreting is a field that requires each interpreter to employ discretion in choosing settings in which to apply their skills. The NIC Certification test is intended to assist in assessing interpreter competency; however, the recently released report on the NIC Performance and Interview Pass Rate has many people questioning the fundamental validity of the test. In light of the 26% NIC Performance pass rate, there must be public acknowledgement that the new “Enhanced NIC” is a flawed test demanding critical inspection.
This is not about bashing. It is important to pay respect to the few selfless interpreters who volunteer to serve on RID committees and on the RID Board. They give their time and confront frustrating criticism while attempting to improve the field. They deserve praise and should be respectfully challenged, as we aim to do today.
Validity of Enhanced NIC in Question
While RID claims that the “Enhanced NIC” is a valid way of evaluating interpreters’ competence, the organization presents incomplete evidence supporting its argument. The board claims that the NIC is valid because it includes items identified in a 2002 role delineation study and goes on to say, “the exams are built such that the most critical knowledge and skills are tested more often than less important skills, as outlined by the test blueprint” (http://www.rid.org/2014/02/nic-validity-reliability-candidate-report/). They do not mention evaluating the quality of these items or any analysis of how validity might be affected by test delivery. This is unusual in evaluating the legitimacy of an assessment. Multiple sources exist explaining assessment validity and reliability. The National Institute of Health offers advice that establishing validity requires multiple routes of analysis, only one being test content (http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3184912/). Is RID arguing that the enhanced NIC is valid based only on the quantity of intended content? Tests attempt to include priority content and intend to prompt test takers to demonstrate knowledge or skill level. Stating a test contains items that are important is not an argument supporting validity. It is a statement that the test intends to measure some identified thing. Intention is not validity. Proving validity requires that a test actually measures those things thoroughly and accurately enough to reflect reality. To claim validity, RID’s must also analyze how the test taking process impacts what the NIC actually measures. Critical questions and answers are missing. How do candidates receive the test? Is a candidate reasonably able to grasp and respond to what is being asked of them based on how the items are crafted and presented? Is test content confusing or rushed? Is performance somehow affected by any other internal or external factors such as testing environment or even the order of questions? Multiple complicated factors may cause invalid scoring of candidates who actually possess knowledge and skills. These are the minimum of what could be asked in establishing the NIC’s validity.
In our analysis, two key factors are very likely to interfere with the validity of the NIC. These factors are test anxiety and short duration of segments. Anxiety is known to interfere with performance of specific skills, and since the NIC is a high stakes and expensive test, candidates are likely to have heightened anxiety beyond what they experience on the job. Since vignettes are only 4 minutes at most, candidates may not have enough time to calm test taking nerves to demonstrate performance skills identified in the candidate handbook. Short segments may also interfere with an interpreter’s ability to use effective interpreting strategies in all 5 vignettes, especially if those vignettes are very different or deal with topics unfamiliar to the candidate. Performance on short segments also easily suffers because of inadequate preparation. 90 seconds of preparation should not be industry standard and is unrealistic to properly prepare an interpreter for a 4 minute segment. With much pressure, little preparation, and insufficient time to prepare for or settle into the vignette, many candidates will be unable to demonstrate their true skills. Therefore, we believe the NIC is actually measuring how test anxiety, insufficient preparation, and lack of warm-up combine to impact interpreter performance during the first 4 minutes of an interpretation. It cannot measure an interpreter’s overall skill as it intends and is therefore invalid. It seems that in the past year, some 26% of NIC attempts successfully demonstrated the ability to achieve a passing score under the test conditions. 74% of those who felt ready enough to spend over $300.00 on the exam were not.
Discrepancy between NIC and Enhanced NIC Pass Rate
In 2009, RID reported that nearly 80% of NIC attempts received passing scores. Compare that to this year’s 26% pass rate. Multiple explanations could explain this discrepancy, but none have been provided. It is very possible that one or both tests are invalid since both tests draw from the same content but produce very different results. Another explanation that could partially explain this difference is who was and is taking the test. Can RID provide statistics on test candidates for the old exam? Did the NIC levels create such an incentive to repeatedly retake the test that it might account for the 53% difference in pass rate between the old NIC and the new? If NICs and NIC-As doing retakes to level up are eliminated from the old NIC pass rate, what is the new statistic? Is it anywhere close to the 26% pass rate of the new NIC? Who is taking the test now? Who is passing, and who is failing?
There are many unanswered questions in recent years surrounding RID and its certification programs. Below is a timeline.
September 2010 – Guy Motley, an RID employee, is caught performing illegal financial activity relating to RID certification.
Fall 2010 – The RID Board approves NIC Task Force recommendations to enhance the NIC.
June 2011 – RID announces a certification department employee falsified scores and forged NIC rater checks to take RID’s money for himself.
July 2011 – Longtime RID Executive Director, Clay Nettles, is terminated. Other longtime employee, Matthew O’hara, fills in as Interim Executive Director.
July 2011 – At RID National Conference in Atlanta GA: RID announces enhanced NIC exam being developed. Enhanced NIC is pass/fail and NIC levels to be discontinued. Enhanced version to be rolled out in less time than it had been taking a hand full of raters to score 1 NIC exam. Why the rush? Also in this mad dash to implement Enhanced NIC delivery and scoring system, RID contracts with certification firm, Caviart Group. Founding partner Clarence “Buck” Chaffee promises streamlined test uploaded to a secure internet line with raters who can instantaneously provide faster and cheaper scoring. He likens testing of interpreters with his company’s testing of crane operators. RID praises the partnership.
March 18, 2012 – Longtime interpreter, CODA, former RID President, and Interpreter Educator, Dr. Dennis Cokely submits a letter to the Board of Directors seeking answers that he assumed RID had at its disposal before the “New Enhanced NIC” was implemented. RID said they needed time to carefully consider their response. The membership is still waiting. Why didn’t RID have these answers?
May 15, 2012 – Dr. Dennis Cokely follows up by writing a Street Leverage article on enhanced NIC and includes his letter to Board. On RID’s admission that it did not have answers, Dr. Cokely states, “…there are even more serious questions about the process by which this iteration of the NIC was developed and implemented”. (http://www.streetleverage.com/2012/05/defenders-of-certification-sign-language-interpreters-question-enhanced-rid-nic-test/)
November 29, 2012 – RID announces Shane Feldman to take the helm as RID Executive Director effective January 1, 2013.
May 20, 2013 – RID releases Youtube video of Certification manager Dan Ebeling announcing partnership with The McCann Associates who will manage NIC interview and performance exams. McCann will take over where Caviart left off. McCann is to upload “New Enhanced NIC” to a secure internet line to streamline the rating process. Testing centers to double. Partnership with Caviart did not produce results.
January 2014 – RID announces McCann testing locations cannot all offer a private space and/or the internet speeds required for “New Enhanced NIC”. Some McCann centers are unable to offer “New Enhanced NIC”. Final list of locations that can to be released soon. Partnership with McCann does not produce results.
January 30, 2015 – RID President announces contract with Executive Director, Shane Feldman, has ended. Search for a new director to begin.
April 2015 – Anna Witter Merithew becomes Interim Executive Director.
Spring 2015 – Longtime employee and former Interim Executive Director and Deputy Director of RID, Matthew O’hara leaves the organization.
June 2015 – RID releases the 2014 annual report. The Interview/Performance portion of the “New Enhanced NIC” has a pass rate of 26%. Of the 771 interpreters who took the test, 50 already had certification. No data is given as to how many of the certified interpreters passed the “New Enhanced NIC”. Any of the certified interpreters who did pass drops the effective pass rate even lower.
July 2015 – “New Enhanced NIC” still being burned to DVDs and shipped via US Mail. Partnership with McCann has not produced a more secure test with faster results.
Meanwhile, interpreters and RID have been pushing for state licensure. These regulations will require professional interpreters to obtain national certification. While RID bylaws say that an interpreter has 5 years from passing the written exam to pass the Interview/Performance exam, many states will only give a provisional license for a limited period of time. (Kentucky, for example, only allows a provisional license to be held for one to two years based on the level of your training program, and Washington State is currently proposing to allow a provisional license to only be held for 24 months before requiring certification to continue working. Many other states top off at 3 years.) RID should encourage states to draft licensure laws that are in keeping with certification timelines, particularly in light of the low pass rate for the current NIC.
Interpreter trainers have asked the Certification Committee to release the rubric used to score the “New Enhanced NIC”. The committee responds that they are looking for interpreters with “strong clear ASL and strong clear English”. They promise to release more information in future.
Summer 2015 – RID announces that the cost to retake the “New Enhanced NIC” Interview/Performance portion will rise from $335 to $360. Indeed RID has chosen to raise the cost of the NIC, yet the test itself has not been improved. A non-member of RID must pay $510 to take the current NIC for the first time. For that cost their test is still burned to a DVD, still shipped to raters via US Mail, the feedback remains obscure and pithy and RID is still only obligated to give results within 120 days. How is that fiscally or ethically responsible to the membership?
In light of our points clearly calling into question the validity of the “New Enhanced NIC”, we ask the NAD/RID Certification Committee and members of the RID Board to consider the following list of demands.
- Immediately suspend the deeply flawed NIC Interview and Performance testing until validity can be demonstrated using thorough analysis, including at minimum questions such as those above. If the NIC Interview / Performance exam does not stand up to legitimate validity measures, we call for an extended collaboration with CIT, CCIE, and NAD to create an exam that can and that reflects the real work of community interpreters.
- Immediately publish a press release or position paper calling on states to extend licensure time limits until a psychometrically valid NIC credential can be established to the satisfaction of RID members.
- Create a rubric OR clear examples of demonstrable skills for each skill item in The Role Delineation Study (published in the NIC Candidate Handbook, 2011). The language is too vague. Interpreters need concrete examples of how specific skills can be demonstrated to follow good practice and assess their own readiness for the exam. RID is under no obligation to describe exactly how the exam measures these skills, but should provide some measure to assess one’s readiness. If statistical gathering and analysis identified important traits interpreters should exemplify, RID should make the community explicitly aware of what it finds to be most important among interpreter knowledge and skills. Withholding this valuable information is irresponsible and unfair to the community.
- Create longer vignettes reflecting settings encountered by generalist interpreters (of the breadth of community interpreting). Longer segments would give candidate more time to compensate for the realities of the test and to accurately demonstrate their skills within the realities of a testing environment. Longer segments should not impact scoring or rating time, only test taking time and the amount of test content. There may need to be additional vignettes to keep test content from leaking.
Testing for certification should be hard. It should raise the profession as a whole. Mentors and novice interpreters should have their skills bettered by striving to achieve the standard set by the test. Fundamentally that standard must be fair and transparent to the membership.
Long before there was RID, an interpreter’s skill level was assessed by native users of ASL. People with Deaf parents or siblings or other people who were Hearing but also with deep ties to the Deaf community would put eyes and ears on novice interpreters’ work. Feedback that is clear and in-depth leads to the greatest growth. Modern testing for certification must reflect these roots. The interpreting profession has grown tremendously since that first conference at Ball State more than a half century ago. Sadly, certification has become not only the gold standard but the only standard for community interpreters to strive for. Because the stakes for passing certification testing are so incredibly high, shouldn’t such a test provide the most in-depth, carefully considered feedback and assessment a professional interpreter can receive in the course of their career?
The current iteration of the National Interpreter Certification test is not transparent. It is unclear to mentors, teachers of interpreters and interpreters striving for certification what precisely is the threshold and what can be done to rise to that level. Given all of this confusion and the abysmal pass rate that no psychometrics can support, we urge the full membership to demand a suspension of this test and the careful crafting of a test that is valid.