Online psychological testing and assessment

---

Recently Anthony S. Ragusea, Psy.D. offered this informative email to several of us inquiring about this topic. I asked Dr. Ragusea if I could blog his response to all of us and he graciously gave permission. You can visit his profile at the Online Therapy Social Network.  Enjoy his wise words on a timely topic! Thanks Dr. Ragusea!

Introduction.

…the Myers-Briggs is indeed online and has been for years, but it is not considered a serious test of personality by experts on test design.  It is not admissible in court and the research clearly shows that it’s not much better than a horoscope.  So let me draw a distinction between serious, rigorously designed tests, and those that are not rigorously designed and thus the designers/publishers are less concerned about the issues I’m about to summarize.  That said, there ARE tests available via computer that are good, solid tests (e.g., the Strong Interest Inventory (career-aptitude test), behavioral rating tests like the CBCL and BASC-2, etc.) but these tests don’t NEED to be administered in person, and also in my experience tests are more EXPENSIVE by computer than paper.  The MMPI-2 is also available for computer administration, but not on the Internet, the client needs to be in the office with the computer.  And it’s more expensive than paper administration, again.  Companies tend to charge high fees for software, plus fees for each test “form,”  and sometimes subscription fees to maintain the service.

In articles I’ve written, I’ve argued that as of now it is not ethical for psychologists to try to conduct certain kinds of psychological evaluations (like forensic evals) solely via telehealth.

Here are some of the issues.  One is that some tests MUST be administered in person as they are currently designed because they involve interaction between the examiner and client.  IQ and memory tests, for example, have puzzle pieces, test booklets that require pencil writing and drawing by the client, etc.  In addition, the examiner needs to have control over the testing environment and how the test is administered–the examiner may pick and choose which subtests are administered, for example, or alter the test protocol for a specific purpose.  The examiner also needs to be able to observe HOW the client is behaving and dealing with test items, the test items are only part of the evaluation.  Behavioral observation during test administration is crucial to evaluating people.  For example, the Rorschach Inkblot test could be adapted to computer administration, but then I can’t SEE a potentially schizophrenic client becoming afraid and anxious in response to an inkblot that looks like Satan to the client.  That’s important information.   It might be possible to adapt IQ and memory tests to a computer-based format, but the therapist would still need control over the administration in real-time, and would need to be able to observe the client.

Another issue relates to STANDARDIZED ADMINISTRATION.  Rigorous tests like IQ, memory, neuropsychological, and others are often designed to be administered under highly specific conditions.  Manuals specify the kind of environment the test should be taken in, where the client should be sitting, exactly what the examiner should say, etc.  Now, some aspects of standardization are better enforced via computer than by a live person, but what can’t happen is that the client be permitted to, for example, take these tests on their laptop in their bedroom with music blasting.  The reason is that it is crucial that all clients take the tests under the same conditions, so that the test validly assesses what’s designed to assess and to permit legitimate comparisons between different subjects who take it.  Changing the protocol potentially ruins the test’s validity.

The next issue relates to what Mark mentioned, is a computer equivalent to a piece of paper?  I don’t think we have a good body of research on this topic yet.  Intuitively I would agree with Mark, a test that is administered just on paper without any interaction with the examiner should likely be equivalent on a computer (assuming the test-taking conditions are otherwise the same).  But we don’t KNOW that.  For example, does eye fatigue affect performance when taking a test on a computer monitor?  It may depend on the test.  Some tests may need to be entirely re-normed on a population that takes it on the computer, and that’s expensive.

The next issue is very important, and relates to TEST INTEGRITY.  There are hundreds of tests out there, many of them not very well designed, and many that don’t worry about test integrity.  By that I mean are the test items and information about how they are scored and interpreted available to the public.  The Myers-Briggs items, for example, are easily accessible by anyone, as is the scoring and interpretive method.  Many serious, rigorously designed tests require complete test integrity.  It is potentially illegal, and certainly a violation of ethical standards, for a psychologist to make MMPI-2 items, for example, available for anyone to see.  The MMPI-2 must NEVER leave the control of the examiner.  The MMPI-2 is the world’s most well-respected and researched personality test, and part of its validity comes from the fact that whoever takes the test has no prior familiarity with it.  There are examples of people who illegally post MMPI-2 items online so that, for example, people in child custody cases can see them and be coached on how to answer them in a custody evaluation.  Obviously, that can’t be permitted.  Results of personality, IQ, and achievement tests can have extremely serious consequences in real life, so it is CRITICAL that the public does not have access to them and is naive to how the tests are scored and interpreted.  This is why these tests can’t be administered on the Internet, because somebody could copy the items down, take screenshots of items, etc.

I once consulted with a psychologist who was doing online work and wanted to administer the MMPI-2 to clients located at distant locations.  The only way I could tell her to do it ethically was to have the client take the test at a health clinic in their hometown and arrange with staff there about how to handle the test materials to maintain test integrity.  But that’s difficult and time consuming, and I don’t know if she ever followed through.

The last issue is one that I touched on earlier–the importance of live observation.  If I’m evaluating a person and the outcome of my eval will have serious consequences, it is absolutely necessary that I see the client in person, at least for part of the eval.  Otherwise, I can’t smell alcohol, I may miss subtle behaviors that are clinically important, I may not be able to see subtle signs of neurological damage like difficulty walking, writing, grasping things, I may miss family interactions that are clinically important, etc.  Even video can’t replace it.  It’s similar to the reasons why the AMA doesn’t want physicians prescribing online to people they’ve never evaluated in person.

So, to summarize, there are certainly tests that can be administered online appropriately, but others that, as of right now cannot, either because the design of the test makes it impossible to adapt to a computer or because of concerns about security and test integrity.  Sometimes there are concerns that a computer-based test is NOT equivalent to taking it on paper.  Bottom line is that as of right now I can’t administer the core battery of tests that I administer in a psychological evaluation, which really limits what I can do with telehealth.  Other mental health professions aren’t qualified to administer the tests that psychologists are, so they don’t feel the constraint.  But I do think this limitation puts a damper on psychologists’ enthusiasm about moving their practice into the Internet age..