A Psychologist’s Ethical Examination of the EPPP

“Examinees shouldn’t be forced into research that benefits the company already providing a high-priced assessment tool”

By: Ryan Blackstock Psy.D., LP, CAADC

Chief Editor: Baron Crespo, Psy.D.

An idea occurred to me some time ago that a specific facet of the Examination for Professional Practice in Psychology (EPPP) may violate ethical boundaries, and this idea became exacerbated when in August 2011 the parameters of the EPPP changed. The problem exists in the now 50 experimental (also called “pre-test” by Association of State and Provincial Psychology Boards) questions that are included in the exam. In a review of the American Psychological Association’s Ethical Principles of Psychologists and Code of Conduct (2010), I found no data that seemed to satisfactorily explain my concerns.


My main dispute is that the 50 experimental questions are thrust upon participants under the guise that “this is just part of the examination,” however, ethically these 50 questions are not involved in the evaluation of “decisional capacity” (APA Ethics Code 9.03). The 50 questions do not count against the participant, but may in fact be damaging to the participant’s performance.


Under the APA Ethics Code, informed consent is necessary except when:  1) testing is mandated by laws or governmental regulations; (2) informed consent is implied because testing is conducted as a routine educational, institutional, or organizational activity (e.g.  when participants voluntarily agree to assessment when applying for a job), or (3) one purpose is to evaluate decisional capacity” (APA Ethics Code 9.03).


One hundred seventy-five of the questions on the EPPP easily fall into this category, in that the purpose of the EPPP is to measure an examinees decisional capacity across a spectrum of domains. The problem is that the 50 research questions embedded in the exam are not mandated by law, despite their routine distribution as part of the EPPP.  I argue that these 50 experimental questions are unethically routine, and although they may provide the examiners with a notion about appropriate future questions (i.e, . item difficulty, item discrimination, etc,) they are not used to determine overall decisional capacity (e.g., getting them correct/incorrect does not affect your score). One’s performance on the 50 experimental questions does not relate to issues of competence or ensuring public safety.


I am certain that examinees are informed that there are 50 experimental questions. This information is most likely covered in the tutorial before taking the exam. I was aware of this fact before taking the exam from my purchased study materials as well as from a survey of several online forums.  My concern is not that informed consent is lacking, but more specifically that the EPPP operates ethically under two different part of the ethics code. I believe that the 175 questions used to determine professional competence is overseen under section 9 of the ethics code concerning assessment. I contend that the 50 experimental questions are and should be governed by section 8.0 (Research and Publication).


In order to highlight my concerns, and abbreviate the task of reproducing the ethics code in its entirety, I have condensed my comments to the subsections that seem most relevant. Under Section 8.02, the informed consent states that psychologists must “inform participants about. . . (2) their right to decline to participate and. . . (3) the foreseeable consequences of declining or withdrawing; (4) reasonably foreseeable factors that may be expected to influence their willingness to participate such as potential risks, discomfort, or adverse effects” (APA Ethics Code, 8.02).


The quandary is that although 175 of the questions on the EPPP are used to assess a candidate’s professional competency, one cannot withdraw from the 50 experimental questions without predictable damage to him or herself. I have no doubt that withdrawing from the EPPP has foreseeable negative consequences. The EPPP is one of the final steps to full licensure in psychology. Without full licensure, at least in the State of Michigan, one can only renew their doctoral educational limited license five times. This typically means that within five years of graduation one must pass the EPPP, have his or her license reduced to a masters limited license, or cease working in the field until they apply to take the exam. Other state boards have been successful at restricting their future competitors’ from taking the EPPP up to three attempts


Being unlicensed for most practicing psychologists is career suicide. I know there are unlicensed practitioners, but I do not know of any insurance panels that pay for unlicensed professionals. When I survey jobs in the field, a license is a standard requirement. I wonder how one would pay back student loans without a career in which they can generate income? Again, this criticism has nothing to do with the 175 questions that determine competency, however; in order to have your competency determined, you must participate in 50 experimental questions. Declining the 50 experimental questions is declining your chance at full licensure.This is the foreseeable consequence of declining or withdrawing.


I am not even certain, in the event an examinee had no knowledge that there would be 50 experimental questions, that once they are in the testing center and have read the informed consent, that they could choose to withdraw or decline without significant financial penalty. The cost of the exam is $600 and the testing centers administration fee (in my case) was $88. Those fees were paid in advance of the test date. I doubt that getting a refund would be a simple or convenient process, if that possibility exists at all.


I assert that it would be a completely different scenario if the 50 experimental questions were an option at the end of the examination. The questions would still be protected under the test security protocols, but would be voluntary. That is not currently the case. The 50 questions are scattered throughout the EPPP. I acknowledge that getting these questions wrong does not count against your score. Perhaps what the ASPPB has not accounted for is that these questions can contribute to frustration on the examinees behalf. Frustration can have negative effects on test performance. I am certainly not contending that the 50 questions are the only source of frustration on the exam. There are a variety of issues related to the 175 other questions that are frustrating, however that is a discussion for another time. In the process of reading and listening to others experiences of the EPPP, this frustration is evident. The frustration itself or even the potential of it that is generated from the way in which these 50 experimental items are incorporated into the EPPP is also completely avoidable.


A common theme exists in the identification of questions by self-reported prepared examinees, who despite their methodical and timely preparatory groundwork, have no adequate base of knowledge by which to approach answering a question. This is not the same as the dilemma of “which is the most correct answer.” This dynamic is, despite preparation, being aware that the question can only be answered correctly by chance. The stressor that accompanies this dynamic is that if examinees experience this repeatedly in the exam, they may become discouraged or frustrated, which likely impacts performance.


I do not think that removing the 50 experimental questions will magically alleviate frustration with the exam. I think that there is a simple way to lessen unnecessary frustration with the exam (i.e., minimize any harmful effects in a reasonable manner,) and that we should do this. This is ethical behavior as described in section 8.08 under Debriefing.


The second aspect that appears unethical, when we consider that the 50 experimental questions fall under the guise of research, is that there is no adequate debriefing. I acknowledge that within 4 to 6 weeks, I received a letter in the mail from the state that identified my score on the EPPP as well as what the standardized score was to pass. I agree that test security is important and I am not lobbying for some chance to review my test in its entirety and see what questions I answered incorrectly. That is the purview of the various EPPP preparation solicitors. What I cannot avoid looking at is that there is no debriefing about your performance on the experimental questions. One could assume that the “good” questions with an item difficulty of p = .625 will be incorporated into newer versions of the EPPP, but that is the extent of it, an assumption at best.


I am aware that one can order a role feedback report from Pearson Vue, however that report is focused on the 175 questions. The problem is that with the EPPP, there has been a marriage between questions measuring competency and the experimental (“pre-test”) items. I understand that it must be a great business advantage to have examinees pay for your examination, and on top of financial profits, the business gets research data for free, arguably at the examinees expense. It seems this union between the EPPP’s assessment function and its built in research tool has resulted in an obfuscation of the ethics involved. The EPPP, being mainly an assessment tool, has been under the purview of the ethics in relation to assessment in psychology. The experimental questions are a different entity altogether.


When I was an undergraduate student at Michigan State University, I participated in a number of psychological studies. Sometimes it was for the extra credit, and other times it was out of interest coupled with a large gap of time between classes. In every case, we were adequately debriefed, even if it was through subsequent contact via student email which would direct us to a website that spoke about the research project and results. I understand that for test security reasons ASPPB cannot send examinees a list of test items they decided to keep. Doing so would be ironically futile.


Debriefing would be a complicated issue, particularly deciding how to debrief examinees on their performance on experimental items. Perhaps this is impossible, and is why there is no debriefing process, although the absence of debriefing does not dismiss the issue. At a minimum, a statement could be included in the informed consent that clearly acknowledges that there will be no debriefing. Currently, this part of the process seems to remain implicit.


I believe the route to solving this dilemma of mixed ethics within the EPPP relies upon extracting the 50 experimental items and placing them as an optional portion after the examinee has completed the exam or run out of time. In all fairness, participation in this should be voluntary. I believe this fixes every ethical problem I have mentioned, with the exception of adequate debriefing. Examinees should not be forced into research that benefits the company already providing a high-priced assessment tool.


I imagine that most people would take the time to complete the experimental questions if they were placed at the end of the exam, but I admit I could be wrong. It is just as likely that people would be glad to be done with the exam and decline to participate. The burden of getting participants should be on the shoulders of ASPPB and PES, NOT the examinee. I wonder if a lack of voluntary participants would force them to provide some reasonable incentive. As it stands now, the examinee is being exploited to a small degree, as they are held hostage and must participate in the research questions. Although this practice may be profitable, I do not see how it is ethical.


Points to Consider:

  1. If it is such a good predictor, why are they adding a second test?
  2. If both Academic Review and AATBS are owned by the same parent company: What accounts for the variance in study materials?
  3. Another ethical question is that only clinicians are tested on this, but have to know I/O and Social psych material, though according to the APA Ethics code, they would not be allowed top practice in these areas without specific training (generally seen as a Masters Degree)…so why test these areas?
  4. Critics say: “Well, tests like the GRE do this all the time:. My answer. Does the GRE fall under the APA Ethics Code? 


Full Credit to Source: Dr. Ryan Blackstock Psy.D., LP, CAADC

References:  American Psychological Association (2010). Ethical Principles of Psychologists and Code of Conduct. Retrieved from http://www.apa.org/ethics/code/index.aspx



We thank all of you psychologists coming forward lately with these abuses and fighting against these cowards, And we encourage You to continue sending us Your stories and leaks. You can do so completely anonymously by sending your documents via www.sendspace.com, directly to our e-mail at info@modernpsychologist.com, or any other PGP encrypted messaging that you may use. Together we will continue dismantling  these cartel schemes, which are essentially harming the public.


#TheNewAPA #APA2018

Newsletter Subscription

© 2014-2018 ModernPsychologist.com. All Rights Reserved.
All Photos Published Licensed Under Creative Commons Zero