Psychological Testing on the Internet New Problems, Old Issues
Jack A. Naglieri George Mason University Fritz Drasgow University of Illinois at Urbana–Champaign Mark Schmit SHL Litigation Support Group Len Handler University of Tennessee
Aurelio Prifitera Harcourt Assessment Amy Margolis Brooklyn Learning Center
Roberto Velasquez San Diego State University
The Internet has significantly changed the way people conduct business, communicate, and live. In this article, the authors’ focus is on how the Internet influences the practice of psychology as it relates to testing and assessment. The report includes 5 broad sections: background and context, new problems yet old issues, issues for special populations, ethical and professional issues, and recommendations for the future. Special attention is paid to implications for people with disabling conditions and culturally and lin- guistically diverse persons. The authors conclude that eth- ical responsibilities of psychologists and current psycho- metric standards, particularly those regarding test reliability and validity, apply even though the way in which the tests are developed and used may be quite different.
At their spring and fall 2000 meetings, members ofthe Committee on Psychological Tests and As-sessment (CPTA) discussed issues related to psy- chological testing and assessment on the Internet. They recognized that as psychological test instruments become more readily available via the Internet, issues arise con- cerning, for example, test reliability, validity, administra- tion, item security, and test-taker confidentiality. Members of CPTA’s parent boards—the Board of Scientific Affairs (BSA), the Board of Professional Affairs (BPA), the Board of Educational Affairs, and the Board for the Advancement of Psychology in the Public Interest—reviewed and en- dorsed the idea of creating a task force to examine these issues. Specifically, BSA and BPA members agreed that it was important for the American Psychological Association (APA) to be a leader in discussing new and emerging technologies in psychological testing, assessment, and re- search and in providing information about Internet-based testing and related issues. They noted that a jointly spon- sored BSA–BPA task force on Internet-based testing would complement other APA groups focusing on related issues. The Task Force on Psychological Testing on the Internet was therefore formed, with a broad mission of reviewing current practices on Internet-based psychological testing and determining psychometric, ethical, legal, and practical implications of this approach to testing particularly for
individuals from culturally and linguistically diverse populations.
Task force members were chosen to reflect expertise across a broad range of testing areas (e.g., educational, school, employment, forensic, career–vocational, clinical, cross-cultural, neuropsychological), to be knowledgeable in Internet technology, and to represent the concerns of diverse groups that may be affected by testing. Their pri- mary objective was to prepare a report, which informs the profession of psychology about emerging issues and prob- lems in Internet testing and actions psychologists can take to protect the integrity of testing and the consumer. An accompanying objective was to devise mechanisms for informing and educating the public about potential prob- lems with Internet testing.
This article is organized into the following five broad sections: (a) background and context, (b) new problems yet old issues, (c) issues for special populations, (d) ethical and professional issues, and (e) recommendations for the fu- ture. Throughout these subsections, both practical and sci- entific issues are discussed with careful consideration of the consequences of decisions based on information obtained from Internet tests. The goal was not to provide a thorough summary of all Internet testing practices, but rather to describe broadly the current state of practice. It will be- come obvious to the reader, as it did to the committee, that many issues about Internet testing practices are similar to those faced by the profession in the past. This report also contains recommendations for the profession’s response to current developments in Internet-based psychological testing.
Jack A. Naglieri, Department of Psychology, George Mason University; Fritz Drasgow, Department of Psychology, University of Illinois at Urbana–Champaign; Mark Schmit, SHL Litigation Support Group, Boul- der, Colorado; Len Handler, Department of Psychology, University of Tennessee; Aurelio Prifitera, Harcourt Assessment, San Antonio, Texas; Amy Margolis, Brooklyn Learning Center, New York, New York; Roberto Velasquez, Department of Psychology, San Diego State University.
Correspondence concerning this article should be addressed to Jack A. Naglieri, Department of Psychology, George Mason Univer- sity, MS 2C6, 4400 University Drive, Fairfax, VA 22030. E-mail: [email protected]
150 April 2004 ● American Psychologist Copyright 2004 by the American Psychological Association 0003-066X/04/$12.00
Vol. 59, No. 3, 150 –162 DOI: 10.1037/0003-066X.59.3.150
T hi
s do
cu m
en t i
s co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. T
hi s
ar tic
le is
in te
nd ed
s ol
el y
fo r t
he p
er so
na l u
se o
f t he
in di
vi du
al u
se r a
nd is
n ot
to b
e di
ss em
in at
ed b
ro ad
ly .
Background and Context Growth of the Internet The past decade has witnessed a rapid expansion of the Internet. The foundations of the Internet began in the 1960s as part of the U.S. defense system development and the inception of new data-passing technologies. Since then it has grown from a university-based network to a worldwide network of interconnected computers accessible to people through many media in all civilized parts of the world (Abbate, 1999). The number of sites available on the In- ternet and the number of users grow larger each day. This relatively new and revolutionary communication network has significantly changed the way people conduct business, communicate with others, and live.
Over the past 10 years, the number of Internet sites and users has grown from hundreds to millions. As of July 2001, there were over 125 million Internet sites (Internet Software Consortium, 2001). Approximately 143 million, or about 54%, of all U.S. citizens have home access to the Internet (U.S. Department of Commerce, 2002). In fact, a recent report suggests that only 24% of Americans are truly offline, with no direct or indirect experience with the In- ternet (Pew Internet & American Life Project, 2003a). Although access varies worldwide by country, it is clear that similar percentages hold, on average, across the globe and that more than 500 million people around the world have access (Nielsen/NetRatings, 2001). Further, close to 40 million U.S. citizens have access to the Internet at work. At home, the average U.S. user spends about 26 hours a month on the Internet; while at work, the average user spends over 75 hours per month on the Internet (Nielsen/ NetRatings, 2003). Clearly, the Internet is quickly becom- ing a medium as pervasive as radio and television, with the capacity for infinitely more two-way communication.
There is no doubt that the Internet is quickly becoming one of the most pervasive communication and commerce media in the world (Abbate, 1999). It is also quite clear that usage and access will continue to grow for years to come. For example, although ethnic minority members in this country were sometimes among the last to utilize the In- ternet, their usage has grown significantly over the last five years (U.S. Department of Commerce, 2002). Individuals with disabling conditions are also connecting in growing numbers (U.S. Department of Commerce, 2002). Further advances in technology, such as broadband, allowing quicker and larger transmissions of data, are likely to increase Internet activities. The impact of the Internet has been, and will continue to be, substantial for all people and organizations, including professional groups. The profes- sion of psychology is among the groups that are beginning to explore opportunities and issues, both positive and neg- ative, concerning the Internet (Barak, 1999). And so we should, as many individuals look to the Internet for psy- chological information. A recent report suggested that 23% of Americans with Internet access have looked for infor- mation about a mental health issue such as depression or anxiety (Pew Internet & American Life Project, 2003b).
Growth of Internet Testing Why has Internet testing created so much interest? Internet advocates stress better, faster, and cheaper services and products, and Internet testing provides many good illustra- tions of this principle. For example, a new test with ac- companying translations could be made available around the world almost instantly. Test publishers can download new tests to secure testing sites in a matter of moments, while other test developers can put tests on their Web sites and make them available to anyone with an Internet con- nection. Updating a test is also much easier. For example, revising a paper-and-pencil test requires printing and dis- tributing new test forms and answer keys and printing new or revised test manuals, an expensive process that may take several months or years. Revisions of a test that appears on the Internet can be downloaded to testing sites around the world in a few minutes at virtually no cost.
In many paper-and-pencil testing and assessment pro- grams, examinees typically receive their scores and inter- pretive reports a month or two after they take a test. Their answer sheets must first be mailed to the test publisher, where they are scanned and scored, and perhaps inter- preted. Then reports are created, printed, and mailed back to the examinees. In an Internet setting, responses are recorded in computer files as examinees answer each item. Software that computes test scores and generates interpre- tive reports can be run as soon as the last item is answered, with examinees receiving feedback within a few seconds of completing the test.
Internet testing is more scalable than paper-and-pencil testing. In the language of the Internet, scalable means that adding volume results in very little additional cost. There- fore, over the course of a year, the number of times people who visit a Web site and respond to a test may increase, for example, from 5,000 per month to 10,000, but the test
Jack A. Naglieri
151April 2004 ● American Psychologist
T hi
s do
cu m
en t i
s co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. T
hi s
ar tic
le is
in te
nd ed
s ol
el y
fo r t
he p
er so
na l u
se o
f t he
in di
vi du
al u
se r a
nd is
n ot
to b
e di
ss em
in at
ed b
ro ad
ly .
publisher does not incur the costs associated with printing, distributing, and scoring 5,000 additional paper-based tests. Of course, eventually an additional server may be required, but additional test administrations are much lower in cost in an Internet setting compared with paper-and-pencil admin- istration. Additionally, because of the minimal costs in- volved, tests and assessments could be made available at no cost to respondents. For example, researchers may put tests and assessments on their Web pages hoping that people will complete the assessment in order to receive a score report. In exchange, the researcher obtains the data pro- vided by the respondents. Test publishers sometimes put free assessments on their Web sites as a means of attracting potential customers.
Goals of Testing Internet tests and assessments can be classified into three broad categories according to their goals. First, many In- ternet tests, instruments, and surveys are designed for per- sonal development and growth and may or may not be scientifically based. Measures such as these are usually designed for the layperson or public consumer. These in- struments may be used to identify specific personality characteristics or traits (e.g., motivation for success; match- making), determine suitability for a particular type of job or trade, or facilitate psychotherapy (e.g., a personal rating of depression or anxiety). Second, many traditional psychod- iagnostic measures like the Minnesota Multiphasic Person- ality Inventory (MMPI), the Minnesota Multiphasic Per- sonality Inventory—2 (MMPI–2), and the Beck inventories now appear on the Internet. These instruments are typically used to make important diagnostic and treatment decisions regarding individuals. In addition, there are Web pages that are devoted to ways to respond so as to create a desired
result, especially in forensic settings where instruments like the MMPI–2 may be pivotal. Finally, cognitive ability tests, certification tests, and licensing exams can be admin- istered via the Internet with the purpose of identifying the best candidates to be awarded some scarce resource (e.g., a job or admission to graduate school) or credential. Here the test or assessment is used to make an important decision about the examinee, usually related to access into a pro- fession or area of study (e.g., medicine, psychology, etc.).
The goals of a test have important implications for how the measure should be administered. For example, a test that is designed for personal development or growth is less likely to be affected or contaminated by some response style. The person is likely to be candid and open, and not defensive or guarded. In these situations, there is little motivation for dissimulation on the part of the respondent. On the other hand, when a measure is used to make a decision about the examinee, the process is more likely to elicit a motivation to obtain a better score. If test takers are sophisticated, they will answer items in a way that they believe maximizes their positive results. For example, an individual may fake good on a personality inventory or cheat on a cognitive ability test. Much more care in test administration is needed in such situations.
Benefits of Internet Testing The benefits of Internet testing are speed, cost, and conve- nience. Testing over the Internet provides rapid communi- cation of findings to clients, patients, researchers, and the public. It also allows researchers to collect data rapidly, conveniently, and at lower costs than in face-to-face re- search settings. Internet testing is cheaper and more effi- cient; it saves valuable time and provides results more rapidly and easily compared with face-to-face testing. Ben- efits of Internet testing also include sensitization and fa- miliarization of testing to potential clients and the presen- tation of test materials in a consistent, uniform manner. The more that potential clients become familiarized with these procedures, the more comfortable their approach to the tests can be, reducing spurious sensitization and situational effects.
Internet testing is also beneficial in that it allows patients in rural settings to be tested, where it would be difficult or impossible to travel to a testing center or to the office of a testing professional. Internet testing is of value to patients who lack transportation to such sites or to those who cannot travel because of physical limitations. In addi- tion, tests may be presented in a precise manner or in interesting and novel ways, so that the client’s attention to the testing task is enhanced, compared with face-to-face administration.
Differences Between Testing and Psychological Assessment The important distinction between testing and psychologi- cal assessment (Matarazzo, 1990) is particularly important for Internet testing sites. Internet site developers as well as many others unfortunately use the terms testing and psy- chological assessment synonymously when actually these
Fritz Drasgow Photo by Harry Zanotti
152 April 2004 ● American Psychologist
T hi
s do
cu m
en t i
s co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. T
hi s
ar tic
le is
in te
nd ed
s ol
el y
fo r t
he p
er so
na l u
se o
f t he
in di
vi du
al u
se r a
nd is
n ot
to b
e di
ss em
in at
ed b
ro ad
ly .
terms mean quite different things. Testing refers to the administration, scoring, and perhaps the interpretation of individual test scores by applying a descriptive meaning based on normative, nomothetic data. The focus here is on the individual test itself. Administering a test is typically a relatively simple process that can be conducted by psychol- ogists and possibly testing technicians, sometimes with relatively little training, or even by a computer. Although more than one test may be given, the emphasis in each case is the comparison of each individual test score with the scores of an appropriate normative group.
Conversely, in psychological assessment, the empha- sis is typically on the person being assessed and the referral question, rather than on specific test results. Typically, an array of tests is given with an emphasis on their integration, taking many factors other than normative findings into account. The results of the tests are integrated among themselves, in the context of additional available patient– client data, such as history, observations, referral source, and information from friends and/or relatives. The eventual goal of the assessment is to answer the referral question or questions. Tests are typically used in the psychological assessment process, but much more information and often much more complexity is involved. The integration and interpretation of data in the assessment process requires a high degree of skill, psychological sophistication, and education.
The distinction between testing and psychological as- sessment is important because most of what is available on the Internet is testing, not psychological assessment. There- fore, the issue of method variance is an important one; for example, the test results obtained on the Internet may be inaccurate because of the specific method used in the testing and because there is no psychologist available to
assist in interpretation. Although it is conceivable that assessment on the Internet may be possible someday, the requirements for appropriate psychological assessment ex- ceed current Internet capabilities.
Psychometric Advantages
Computerized tests provide some psychometric advantages in comparison with paper-and-pencil assessments. In fact, considerable research has been conducted to document and demonstrate these advantages. A brief summary is pro- vided here; more detail can be found in Sands, Waters, and McBride (1997) and Drasgow and Olson-Buchanan (1999).
An Internet test and assessment provides more accu- rate scoring compared with a traditional paper-and-pencil test. Optical scanning of paper test forms encounters diffi- culties with stray pencil marks, incomplete erasures, and insufficiently darkened answers. In computerized testing, an examinee enters a response, the response is displayed on the screen, and the examinee is provided an opportunity to change the answer. Suppose an examinee has selected “B” as his or her response. The computer monitor will then display a darkened circle next to option “B” and will allow the examinee to change the response or proceed to the next item. If the examinee goes to the next item, well-designed software will correctly record and score the “B” response. A significant source of errors is removed when optical scanning is not used.
Internet testing and assessment is especially well suited for the use of item response theory (IRT; Hambleton & Swaminathan, 1985; Hulin, Drasgow, & Parsons, 1983; Lord, 1980). For example, computerized adaptive tests that tailor difficulty to the ability level of each examinee can be efficiently delivered through this medium. In this process, IRT technology would be used to select which items are given so that they are of appropriate difficulty for each examinee. Internet assessment can also offer the potential for assessing abilities and skills not easily assessed by paper-and-pencil methods. For example, Vispoel (1999) developed a computerized assessment of musical aptitude; Ackerman, Evans, Park, Tamassia, and Turner (1999) cre- ated a dermatological test that allows examinees to pan in and out as they examine color images of skin disorders; and Drasgow, Olson-Buchanan, and Moberg (1999) developed an assessment that uses video clips to assess respondents’ conflict-resolution skills. These and many other needs can be effectively met using computerized technology that can be delivered via the Internet.
New Problems Yet Old Issues Test–Client Integrity
In the same way that we do not allow clients to take tests at home, given that they might not take them privately, Internet testing encounters this old problem with a new twist. When the goals of the test taker differ from the goals of the test user, it is important to confirm the identity of the person answering items. The simplest and most effective method is to require test takers to go to a secure test site and show a government-issued photo ID such as a driver’s
Mark Schmit
153April 2004 ● American Psychologist
T hi
s do
cu m
en t i
s co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. T
hi s
ar tic
le is
in te
nd ed
s ol
el y
fo r t
he p
er so
na l u
se o
f t he
in di
vi du
al u
se r a
nd is
n ot
to b
e di
ss em
in at
ed b
ro ad
ly .
license or passport. Of course, test administration at such sites is inconvenient and expensive. When a test or assess- ment is not administered at a secure test center, there are a number of ways to check a test taker’s identity (e.g., “What is your mother’s maiden name?”). Unfortunately, such methods can easily be circumvented; a more talented ac- complice can sit with the supposed test taker and provide answers to items.
Segall (2001) suggested a clever means of confirming the validity of test takers’ administered tests remotely via the Internet. He has proposed Internet administration of the lengthy Armed Services Vocational Aptitude Battery (ASVAB) enlistment test used by the U.S. military. Seg- all’s idea is that individuals could take the ASVAB at their convenience in nonsecure locations. Individuals who ob- tain scores qualifying for enlistment would then travel to secure test centers, where they would be administered a much shorter confirmation test composed of highly dis- criminating items. A statistical procedure developed by Segall would then be used to check whether the test taker’s original responses are consistent with the responses from the confirmation test. This method was found to be very effective at detecting cheating in a simulation study. A combination of informing examinees that a confirmation test will be administered as well as applying Segall’s sta- tistical analysis may also prove to be effective in discour- aging cheating.
Test Security Levels of security can range from highly secure and re- strictive (e.g., high-stakes testing programs) to unsecured and permissive (low-stakes testing). As might be expected, the greater the level of security, the higher the cost for implementing and maintaining an application. The level of
security implemented for a given test or test site should be appropriately matched to the usage of the test. Secure test environments should use a three-tier server model. Within this model, the test system is actually made up of three independent servers: an Internet server, a test application server, and a database server. It is imperative that the application server is solely dedicated to the test application. In order to maximize the security of client data, a separate data server should be maintained behind a secure firewall. This configuration reduces the possibility of unauthorized intrusions into client test data. If scoring and reporting services are required, it is recommended that these appli- cations be placed on yet a fourth server in the middle tier with the application server in order to minimize processing bottlenecks that may affect the test application or data access. Regular and frequent backups of all collected data should be conducted, and the provider should be able to give prospective customers a detailed disaster recovery plan. Redundancy allows a site to continue to operate even if one of its components completely fails. A reputable provider will have redundancy on all systems throughout its site including incoming and outgoing communications lines. As with any secure application, client and adminis- trator password formats need to be robust (nontrivial) and actively maintained. Finally, server traffic should be ac- tively and continuously monitored for intrusions.
On the client side, one of the most important security considerations is the prevention of unauthorized copying by the examinee or an observer and printing of test items. This can partially be achieved within a browser by dis- abling access to menu selections such as cut, copy, paste, export, save, save as, print, print screen, and so forth. Hot keys and right mouse context menu selections should also be disabled. However, it is only possible to partially secure items by controlling browser functions. Even with such controls in place, it is still possible for more technically knowledgeable examinees to make use of operating system features and other applications to capture items from the screen. Therefore, where full client-side security is re- quired, it is necessary to install a test security agent on the client’s desktop, which completely prohibits an examinee from dropping out of the test application while it is in operation. Such an application prevents users from launch- ing screen recorders, word processors, e-mail applications, and any other unrelated application that may be used to compromise the security of test items.
Issues for Special Populations The delivery of psychological tests through the Internet provides the opportunity to meet the needs of a wide variety of individuals, in particular, important special pop- ulations including people with disabling conditions and culturally and linguistically diverse persons.
People With Disabling Conditions
A critical issue in determining appropriate accommoda- tions for a person with a disability is demonstrating the clear relationship between the individual’s deficit and the
Len Handler
154 April 2004 ● American Psychologist
T hi
s do
cu m
en t i
s co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. T
hi s
ar tic
le is
in te
nd ed
s ol
el y
fo r t
he p
er so
na l u
se o
f t he
in di
vi du
al u
se r a
nd is
n ot
to b
e di
ss em
in at
ed b
ro ad
ly .
nature of the accommodation. The challenge of determin- ing the type of accommodations required for Internet-based assessment arises in part because little is known about the unique aspects of testing in this format. Although many of the accommodations developed for paper-and-pencil test- ing can be used for Internet assessments, new issues will likely arise. As psychologists begin to make recommenda- tions to institutions on behalf of individuals with disabili- ties or on behalf of institutions attempting to design fair testing practices for groups of individuals with disabilities, it is important to consider new types of accommodations to address the unique problems inherent in Internet assessment.
Accommodations may be considered in terms of op- erating at the level of the individual or at the level of the group. At the individual level, adequate accommodations include alterations in the testing environment. Although this is standard practice with paper-and-pencil testing, unique challenges may be encountered with Internet assess- ments because the computer may be permanently affixed in one position. Adjusting the height and placement of the table on which the computer sits is critical for an individual in a wheelchair. For some disabilities, accommodations require alterations to test administration itself rather than alterations to the environment. For example, a reader is often recommended for individuals who are sight impaired or who have a specific reading disability (e.g., dyslexia). During Internet-based assessment, there is likely to be a new vocabulary to describe the spatial layout of the mate- rial and the actions taken by the reader (e.g., rather than stating, “I am filling in Answer A on the scantron sheet,” the reader might say, “I am clicking Choice A on the answer screen”). These accommodations are not unique to testing over the Internet, but are unique to testing on a
computer platform, the frequency of which will likely increase as Internet technology advances.
At the group level, a lack of equal access to technol- ogy may result in poorer test performance for some groups. Households with lower incomes have reduced access to computers and therefore the Internet. Assessment over the Internet, therefore, may be confounded by the novelty of the format. For example, during cognitive testing, individ- uals who are less familiar with computers will have a greater cognitive load due to divided attention than indi- viduals who are familiar with computers. Further, a lack of familiarity with the security and privacy features of the Internet may influence performance. In this sense, low access to computers may be viewed as a disability that requires accommodations to ensure fair testing.
Culturally and Linguistically Diverse Groups
Many culturally and linguistically diverse groups, includ- ing Latinos and African Americans, have been among the last to connect to the Internet because of economic and/or access issues. Yet, the number of people from these and other minority groups who have access to the Internet is increasing dramatically. For these groups, the Internet is proving to be a tool that connects them to their country of origin, resources in a particular language or dialect, and so forth.
Like the majority Euro-American population, mem- bers of these groups have also begun to access the Internet for information related to mental health and psychology. For example, it is not unusual for a Spanish-speaking Latino to seek out information about a particular mental condition or even a psychological test instrument through the Internet. The person is now likely to find information in Spanish, usually from an Internet site in Latin America. Similarly, the person may seek information about a partic- ular test that he or she is about to take that will be admin- istered by a psychologist (e.g., for employment screening or child custody purposes).
There remain many unanswered questions regarding the psychological testing and assessment of these groups via the Internet. In many ways, these issues are similar to concerns related to test use with culturally diverse or mi- nority groups (e.g., fair assessment). For example, it is unclear if it is necessary to have separate norms, including norms for minorities, for an instrument that is administered via the Internet versus administered in the traditional man- ner. Also, a review of various Web pages indicated that many instruments are poorly translated, or have been mod- ified for use with Latinos in the United States or Latin America, by Spanish-speaking professionals usually out- side of the United States. Additionally, older measures, such as the MMPI, can still be found in Spanish despite the appearance of recent translations of the MMPI–2 that are superior to the translations of the older MMPI. People may use old and outdated instruments in a manner that is inap- propriate or problematic, which may then result in negative consequences for clients and the public.
Aurelio Prifitera
155April 2004 ● American Psychologist
T hi
s do
cu m
en t i
s co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. T
hi s
ar tic
le is
in te
nd ed
s ol
el y
fo r t
he p
er so
na l u
se o
f t he
in di
vi du
al u
se r a
nd is
n ot
to b
e di
ss em
in at
ed b
ro ad
ly .
Ethical and Professional Issues Ethical issues abound for psychologists who use the Inter- net in their practice. Many of these issues are being ad- dressed by specific APA committees (American Psycho- logical Association, 1997). Although all of the ethical issues surrounding the use of the Internet in the practice of psychology are important, we confine our discussion to the issues specifically raised around Internet testing. Our dis- cussion is framed by the current APA Ethical Principles of Psychologists and Code of Conduct (American Psycholog- ical Association, 2002), specifically Section 9, Assessment, which covers most of the issues surrounding Internet testing.
The first ethical issue to be considered is the profes- sional context in which the Internet testing takes place. The associated ethical principle is as follows: 9.01 Bases for Assessment
(a) Psychologists base the opinions contained in their recommen- dations, reports, and diagnostic or evaluative statements, includ- ing forensic testimony, on information and techniques sufficient to substantiate their findings.
(b) Except as noted in 9.01c, psychologists provide opinions of the psychological characteristics of individuals only after they have conducted an examination of the individuals adequate to support their statements or conclusions. When, despite reasonable efforts, such an examination is not practical, psychologists doc- ument the efforts they made and the result of those efforts, clarify the probable impact of their limited information on the reliability and validity of their opinions, and appropriately limit the nature and extent of their conclusions or recommendations.
(c) When psychologists conduct a record review or provide con- sultation or supervision and an individual examination is not warranted or necessary for the opinion, psychologists explain this
and the sources of information on which they based their conclu- sions and recommendations. (American Psychological Associa- tion, 2002)
The issue raised by Internet testing is how testing is placed into a professional context when conducted on the Internet. Many tests on the Internet are accompanied by little other than some broad statements about the use of the test. Further, test takers may not read instructions or may ignore disclaimers more than in face-to-face situations (Barak & English, 2002). Under these principles, test materials that are posted for self-administration and interpretation on the Internet should be accompanied by a statement to the test taker that clearly defines the bounds and limitations of the professional relationship with the client that can be achieved through this medium. This may seem a bit coun- terintuitive given the impersonal nature of Internet com- munications. However, a potential client who is browsing the Internet for professional advice is seeking a trust rela- tionship. Providing preliminary test materials for diagnos- tic or evaluative purposes therefore implies an offer to form this trust relationship. Thus, the limitations of the relation- ship that can be developed through an impersonal medium such as the Internet should be clearly described in an opening statement to the test taker. In addition, test pro- viders may need to make available contact information (e.g., e-mail address, phone number) for those who do not completely understand directions or the purpose of the test. Finally, the limits of the feedback provided to the test taker following the test should be clearly described both before the test and preceding feedback. This description should clearly describe the potential limitations of conclusions and recommendations that can be made as a result of a very limited and potentially nonpersonal Internet approach.
The next area of ethical consideration involves the appropriate use of Internet testing and assessment. The ethical principle states the following:
9.02 Use of Assessments
(a) Psychologists administer, adapt, score, interpret, or use assess- ment techniques, interviews, tests, or instruments in a manner and for purposes that are appropriate in light of the research on or evidence of the usefulness and proper application of the techniques.
(b) Psychologists use assessment instruments whose validity and reliability have been established for use with members of the population tested. When such validity or reliability has not been established, psychologists describe the strengths and limitations of test results and interpretation.
(c) Psychologists use assessment methods that are appropriate to an individual’s language preference and competence, unless the use of an alternative language is relevant to the assessment issues. (American Psychological Association, 2002)
Internet testing, in many cases, has been simply a process of putting paper-and-pencil or computerized tests onto a new medium. However, although research has explored the equivalence of some forms of computerized and paper-and- pencil tests (e.g., Mead & Drasgow, 1993), very little research has been conducted on the equivalence of Internet
Amy Margolis
156 April 2004 ● American Psychologist
T hi
s do
cu m
en t i
s co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. T
hi s
ar tic
le is
in te
nd ed
s ol
el y
fo r t
he p
er so
na l u
se o
f t he
in di
vi du
al u
se r a
nd is
n ot
to b
e di
ss em
in at
ed b
ro ad
ly .
testing with these other formats. This may call into ques- tion the evidence for the usefulness of these tools. Further, tests that may have been developed and researched in a proctored setting are now often being used in an unproc- tored context that is facilitated by the Internet and its widening accessibility. This approach calls into question the proper application of the techniques. The effects of both the medium and the context require additional research to ensure appropriate use of tests and assessment on the Internet.
As noted earlier, an advantage to using the Internet to deliver tests is that it may provide greater accessibility and reach than an approach that requires an individual to be at a certain place, at a particular time. This advantage can also create a challenge. Wider access may cause a difference in the populations for which the test was developed versus the ultimate population that has access. For example, a preem- ployment test may be specifically developed and re- searched for a management population. Under more tradi- tional conditions, applicants for these management positions may be required to test at a specific location where a significant effort and commitment is involved. However, the Internet may provide easy access to a differ- ent population where a nonqualified candidate could decide that he or she might just take the test on the off chance that he or she might gain entry to an otherwise inaccessible position.
Normative issues are also a related concern for Inter- net test delivery. With good intentions, a test may be placed on a Web site by a psychologist in the United States, but someone in China may have access to it and complete the test. Feedback may be based only on U.S. norms. An inadvertent, but inappropriate, use of norms is the result. This is clearly an area of great potential for the inappro-
priate use of tests and associated norms. Psychologists will need to make substantial efforts to collect demographic information prior to testing and to provide feedback only to individuals in groups for which normative data are available.
The next area of ethical consideration involves in- formed consent. The ethical principle states the following: 9.03 Informed Consent in Assessments
(a) Psychologists obtain informed consent for assessments, eval- uations, or diagnostic services, as described in Standard 3.10, Informed Consent, except when (1) testing is mandated by law or governmental regulations; (2) informed consent is implied be- cause testing is conducted as a routine educational, institutional, or organizational activity (e.g., when participants voluntarily agree to assessment when applying for a job); or (3) one purpose of the testing is to evaluate decisional capacity. Informed consent includes an explanation of the nature and purpose of the assess- ment, fees, involvement of third parties, and limits of confiden- tiality and sufficient opportunity for the client/patient to ask questions and receive answers.
(b) Psychologists inform persons with questionable capacity to consent or for whom testing is mandated by law or governmental regulations about the nature and purpose of the proposed assess- ment services, using language that is reasonably understandable to the person being assessed.
(c) Psychologists using the services of an interpreter obtain in- formed consent from the client/patient to use that interpreter, ensure that confidentiality of test results and test security are maintained, and include in their recommendations, reports, and diagnostic or evaluative statements, including forensic testimony, discussion of any limitations on the data obtained. (American Psychological Association, 2002)
Gaining true informed consent through electronic means is likely to create unique challenges for psychologists. As noted earlier, the impersonal and standardized nature of Internet testing programs are not likely to fit all individuals the same. In other words, it may be very difficult to provide true informed consent to all individuals completing tests through the Internet. In many cases, it will not be known whether the person completing the test is capable of giving informed consent or whether permission is required from a legally authorized person. Take, for examples, a preteen who poses as an individual over 18 years old or a patient under the legal guardianship of another who gives consent in order to gain access to testing. Psychologists who wish to use testing on the Internet, other than for excepted practices, will need to find ways to deal with this thorny problem of how to authenticate informed consent over the Internet.
The next area of ethical consideration involves the appropriate release of test data. The ethical principle states the following: 9.04 Release of Test Data
(a) The term test data refers to raw and scaled scores, client/ patient responses to test questions or stimuli, and psychologists’ notes and recordings concerning client/patient statements and behavior during an examination. Those portions of test materials that include client/patient responses are included in the definition
Roberto Velasquez
157April 2004 ● American Psychologist
T hi
s do
cu m
en t i
s co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. T
hi s
ar tic
le is
in te
nd ed
s ol
el y
fo r t
he p
er so
na l u
se o
f t he
in di
vi du
al u
se r a
nd is
n ot
to b
e di
ss em
in at
ed b
ro ad
ly .
of test data. Pursuant to a client/patient release, psychologists provide test data to the client/patient or other persons identified in the release. Psychologists may refrain from releasing test data to protect a client/patient or others from substantial harm or misuse or misrepresentation of the data or the test, recognizing that in many instances release of confidential information under these circumstances is regulated by law. (See also Standard 9.11, Main- taining Test Security.)
(b) In the absence of a client/patient release, psychologists pro- vide test data only as required by law or court order. (American Psychological Association, 2002)
Psychological test data reveal very personal details about human characteristics, behaviors, preferences, and capabil- ities. This type of data is not only valuable to psychologists, but is also valuable to sales, marketing, political, and other groups who may or may not have the individual’s best interest in mind when deciding how to use these data. Therefore, it is imperative that measures be taken to pro- vide secure sites for the collection of psychological test data on the Internet. Without secure sites, test data could be intercepted, corrupted, or changed by unscrupulous data thieves and hackers. There are three major principles of Internet security, and psychologists using the Internet for testing should take proactive steps in each of these areas to protect test takers (Howard, Paridaens, & Gramm, 2001). The first principle is confidentiality, which deals with keep- ing information from being viewed by unintended readers. Encryption technology is designed to provide confidential- ity by scrambling data so that only the appropriate senders and receivers can read the data. The second principle is integrity, which is concerned with keeping information from being altered. Message digests are fingerprints that do not allow the changing of information or at least can detect when information has been changed. The final principle is authentication, which relates to identifying the origins of the data. Digital signatures can provide the authentication through a system of keys that are used both in the sending and receiving of messages to identify the sender as authen- tic. Given the value and highly sensitive nature of psycho- logical test data, psychologists should use technologies in each of these areas to secure data.
The sharing of data and reports is infinitely easier as a result of Internet-based access to databases. People any- where in the world can access databases anywhere else at the click of a button, given the correct security clearance. This ease of access provides more opportunity than in the past for both the intentional and the inadvertent release of data to unqualified individuals. Safeguards must be put in place by psychologists, in conjunction with information technologists, to avoid the release of data to those who are unqualified to use it. Although there are many technolog- ical safeguards available to protect data, the psychologist’s responsibility goes further. It extends to building lines of communication, to the use of training materials, and to other means of safeguarding data that involve the human element that is more likely to result in security breaches compared with the failure of technology.
The next ethical issue of note in the use of tests on the Internet has to do with test development efforts. The ethical principle reads as follows:
9.05 Test Construction
Psychologists who develop tests and other assessment techniques use appropriate psychometric procedures and current scientific or professional knowledge for test design, standardization, valida- tion, reduction or elimination of bias, and recommendations for use. (American Psychological Association, 2002)
Traditional test construction techniques are appropri- ate for administration over the Internet in proctored test environments. However, as noted earlier, tests developed in a paper-and-pencil format and researched in monitored and controlled situations cannot be assumed to provide equiv- alent measurement when administered over the Internet in unmonitored and uncontrolled situations. Therefore, addi- tional studies of test equivalence and norming should be conducted over the Internet, with subjects completing the test under conditions that represent those that the intended target population will experience (Epstein, Klinkenberg, Wiley, & McKinley, 2001).
Advances in computer and Internet technologies have provided the medium for many new advances in testing. Video-based simulations, virtual reality, computer adaptive testing, and precision measurement of physiological re- sponses are examples of technological advances that are shaping the new testing landscape (Olson-Buchanan, 2002). Integrated systems approaches are also beginning to appear. For example, in the area of preemployment testing, methods and tools such as job analysis, recruitment sys- tems, prescreening tools (i.e., questionnaires, application blanks, etc.), selection tests, interviews, and hiring deci- sions are being integrated in a fully Internet-based or en- terprise system that challenge the existing psychometric knowledge base (Drasgow & Olson-Buchanan, 1999; Funke & Schuler, 1998). Therefore, it is incumbent on psychologists to understand the bounds of current psycho- metric methods and to establish, research, and report on new methods that support emerging technological ad- vances. It would be unethical to develop new measurement tools that cannot be held to existing psychometric standards (American Educational Research Association, American Psychological Association, & National Council on Mea- surement in Education, 1999) without providing arguments and evidence for new or revised standards. Advances in testing spurred by the Internet should be encouraged, but associated advances in psychometric theory may be a man- datory part of this advancement (i.e., in cases where exist- ing models are inappropriate).
The interpretation of Internet test results poses some unique ethical considerations. The relevant ethical princi- ple is as follows:
9.06 Interpreting Assessment Results
When interpreting assessment results, including automated inter- pretations, psychologists take into account the purpose of the assessment as well as the various test factors, test-taking abilities, and other characteristics of the person being assessed, such as
158 April 2004 ● American Psychologist
T hi
s do
cu m
en t i
s co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. T
hi s
ar tic
le is
in te
nd ed
s ol
el y
fo r t
he p
er so
na l u
se o
f t he
in di
vi du
al u
se r a
nd is
n ot
to b
e di
ss em
in at
ed b
ro ad
ly .
situational, personal, linguistic, and cultural differences, that might affect psychologists’ judgments or reduce the accuracy of their interpretations. They indicate any significant limitations of their interpretations. (American Psychological Association, 2002)
Internet testing will often be conducted in unproctored and in variable environments. Test takers will likely be in unstand- ardized settings (e.g., home, library, school), and psycholo- gists will have little or no way of knowing exactly what conditions might exist that could influence or limit interpre- tations. This problem may be alleviated to some extent by the use of instructions to test takers, but it is likely that this will only reduce a small amount of irrelevant variability in scores. Further, when tests are completed in unmonitored situations, there is currently no way to guarantee the true identity of the test taker (Schmit, 2001). Thus, psychologists will need to weigh carefully the importance they place on tests adminis- tered over the Internet. Confirmatory methods, administration of equivalent forms, or gathering of data through additional methods will almost always be necessary before making any- thing other than preliminary evaluations, diagnostic, or pre- dictive decisions.
Gaining an understanding of the test-taking skills and specific personal characteristic of the test taker poses an even greater challenge, given the impersonal approach that characterizes most Internet-based testing and assessment. For example, in preemployment testing situations, a pro- vider may have no way of knowing whether an applicant has a particular disability that might affect the test results and invalidate the possible interpretation of those results. Similarly, a test may be posted in English for use in coun- seling, but the test taker speaks English only as a second language. Unless the test taker is asked about this condi- tion, the interpretation of results will likely be flawed. The point is that psychologists using Internet testing and assess- ment must make provisions for understanding the unique needs of test takers that may ultimately affect the interpre- tation of results.
In addition, test takers must be given information that clearly identifies the purpose of the test so that they can determine whether the test is appropriate for their situation. However, this may not be as easy as providing a purpose statement. The test taker will need help understanding whether the test is a fit for his or her situation. There will likely be a need for prescreening the test taker to help him or her understand whether the test or assessment is right for his or her situation.
The next set of ethical issues to be considered with regard to Internet testing involves the use of Internet tests by unqualified persons. The ethical principle states the following: 9.07 Assessment by Unqualified Persons
Psychologists do not promote the use of psychological assessment techniques by unqualified persons, except when such use is con- ducted for training purposes with appropriate supervision. (Amer- ican Psychological Association, 2002)
The Internet has made it very easy for anyone to publish any kind of material in the public domain. This freedom has led many to assume that anything published on
the Internet is in the public domain and can be copied and used by anyone who chooses to do so. These and other Internet crimes are raising significant challenges for many professions (Reno, 2000). Whole tests, scales, and test items posted on the Internet can be copied and used by unqualified people. It is the responsibility of psychological test publishers and authors to keep their works under tight control and to report copyright violations. Most do this well with customers who use appropriate channels to gain ac- cess to the materials. However, publishers and authors must scan the Web for whole and partial elements of tests that require professional training for administration or interpre- tation. Partial tests are likely to be the most difficult to identify, yet they may be the most damaging, as the original psychometric properties are likely denigrated. Consistent with Principle 9.11 (cited below) publishers must also protect their copyrights on test materials. It is the duty of the psychology profession to protect the public from un- scrupulous vendors who exploit the Internet with tests of others or, worse yet, with bad renditions of the original test.
Principle 9.07 is written in a way that may suggest that psychologists take reactive steps rather than proactive steps in the protection of the profession. However, others have taken more proactive measures to protect the public. For example, a mental health consumer advocacy and educa- tion program has stepped up a process for checking cre- dentials of online counselors (Ainsworth, 2002). Online therapists can register with this organization and have their credentials (e.g., education, experience, background) checked. Therapists who pass this check are issued a spe- cial icon for posting on their Web site. Clients can go to the advocacy group’s Web site to verify the authenticity of the therapist. A similar program could be established by a consortium of test publishers who have or plan to have their psychological test products administered on the Internet.
The next ethical principle to be considered deals with outdated test materials. It reads as follows:
9.08 Obsolete Tests and Outdated Test Results
(a) Psychologists do not base their assessment or intervention decisions or recommendations on data or test results that are outdated for the current purpose.
(b) Similarly, psychologists do not base such decisions or recom- mendations on tests and measures that are obsolete and not useful for the current purpose. (American Psychological Association, 2002)
The Internet is full of obsolete and outdated information. Consumers often have difficulty sorting out the current from the outdated pages available on the Internet. Consis- tent with the discussion of the previous principle, when partial or whole replication of test materials is made through uninformed or fraudulent acts, these tests materials are likely to become obsolete or outdated, because the original publisher updates the materials. Further, it is quite easy for Web publishers to forget about published pages on the Internet that may be updated in different places, yet the old materials remain available to the public. Finally, psy- chologists who do not closely watch the literature and other
159April 2004 ● American Psychologist
T hi
s do
cu m
en t i
s co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. T
hi s
ar tic
le is
in te
nd ed
s ol
el y
fo r t
he p
er so
na l u
se o
f t he
in di
vi du
al u
se r a
nd is
n ot
to b
e di
ss em
in at
ed b
ro ad
ly .
materials from test publishers may inadvertently use out- dated materials online. Others may resist change and in- tentionally use outdated materials. As previously noted, test publishers and authors must carefully monitor the Internet for obsolete and outdated materials and take both proactive and reactive steps to curb and eliminate these practices.
Third-party vendors of Internet tests and associated services also have a set of ethical issues to consider. The relevant ethical principle states the following: 9.09 Test Scoring and Interpretation Services
(a) Psychologists who offer assessment or scoring services to other professionals accurately describe the purpose, norms, valid- ity, reliability, and applications of the procedures and any special qualifications applicable to their use.
(b) Psychologists select scoring and interpretation services (in- cluding automated services) on the basis of evidence of the validity of the program and procedures as well as on other appropriate considerations.
(c) Psychologists retain responsibility for the appropriate appli- cation, interpretation, and use of assessment instruments, whether they score and interpret such tests themselves or use automated or other services. (American Psychological Association, 2002)
The Internet is full of psychological, para-psychological, and pop-psychology tests, as described in earlier sections. Psychologists must find ways to differentiate themselves from the mass of alternatives that do not meet professional standards (American Educational Research Association et al., 1999). Providing the information described in this ethical principle is the first step in overcoming such con- fusion. Psychologists who provide tools to other trained professionals should go beyond the simple provision of providing basic psychometric information to potential us- ers. Steps could be taken to show the equivalence of Internet testing with traditional forms of the measure (Ep- stein et al., 2001). Efforts should also be made to provide consultation and training to test users regarding the chal- lenges faced in using tests on the Internet (Barak & En- glish, 2002). The training should be specific to tests and populations that will take the tests. Processional vendors of psychological tests to be used on the Internet may be able to overcome some of the noise of Internet marketing by becoming professional Internet test consultants. Producers of pop-psychology tests should be made to issue more detailed disclaimers, or warnings, describing their tests as entertainment and not as true tests, just as tobacco manu- facturers must issue store warnings on cigarette packages.
The advent of technological breakthroughs and the ease of conducting a professional practice that results from these innovations can occasionally blind adaptors to the fundamental qualities that comprise quality tools. Psychol- ogists must learn to discriminate among efficient delivery tools, flashy format, face-valid content, and psychometric quality. All of these qualities may be important to a psy- chologist in choosing a vendor, but the foundations of psychometrics are still necessary conditions that should be the first hurdle in a multihurdle decision process. Technol- ogy advances should not be considered in a vacuum when choosing an Internet test.
Perhaps one of the greatest challenges of Internet testing will involve the explanation of results to test takers. The ethical principle dealing with this issue is as follows:
9.10 Explaining Assessment Results
Regardless of whether the scoring and interpretation are done by psychologists, by employees or assistants, or by automated or other outside services, psychologists take reasonable steps to ensure that explanations of results are given to the individual or designated representative unless the nature of the relationship precludes provision of an explanation of results (such as in some organizational consulting, preemployment or security screenings, and forensic evaluations), and this fact has been clearly explained to the person being assessed in advance. (American Psychological Association, 2002)
Providing feedback to test takers over the Internet is a topic of concern to many psychologists. There are at least three major ethical issues to consider. First, there are limited ways to understand the conditions under which the test taker completed the test. Did the individual complete the test or did someone else help or do it for him or her? Under what environmental conditions was the test taken? These and many other questions should be answered in order to provide accurate feedback. Second, it is very difficult to provide feedback, particularly negative feedback, to a test taker without knowing the person’s emotional and mental state. The wrong type of feedback could exacerbate the individual’s condition. Third, it is difficult to provide test takers with immediate emotional support in cases where the feedback has traumatic effects on an individual. It is also difficult to know the extent of these reactions in the first place. Given these severe limitations and many other pos- sibilities, psychologists should rarely provide feedback over the Internet. When they do provide feedback, pro- cesses for resolving these ethical issues either in “real time” or within a reasonable time period should be established. Feedback should generally be limited and should include directions for seeking additional information and help through other means. Ultimately, feedback should rely on multiple methods of evaluation to provide assessment re- sults consistent with professional best practice.
Another rather charged area of ethical concern is the maintenance of test security when tests are delivered over the Internet. The ethical principle covering this set of issues is as follows:
9.11. Maintaining Test Security
The term test materials refers to manuals, instruments, protocols, and test questions or stimuli and does not include test data as defined in Standard 9.04, Release of Test Data. Psychologists make reasonable efforts to maintain the integrity and security of test materials and other assessment techniques consistent with law and contractual obligations, and in a manner that permits adher- ence to this Ethics Code. (American Psychological Association, 2002)
As the music industry can attest, the ease of posting ma- terial on the Internet has led to widespread violations of copyright laws. Many psychological tests and assessments are copyrighted because much effort was expended in their
160 April 2004 ● American Psychologist
T hi
s do
cu m
en t i
s co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. T
hi s
ar tic
le is
in te
nd ed
s ol
el y
fo r t
he p
er so
na l u
se o
f t he
in di
vi du
al u
se r a
nd is
n ot
to b
e di
ss em
in at
ed b
ro ad
ly .
development. These instruments constitute much of test publishers’ intellectual capital and must be safeguarded. It is unethical and illegal for unauthorized parties to distribute or use such copyrighted materials. In fact, a quick search of www.ebay.com on any given day will produce quick access to many copyrighted and sensitive test materials. For ex- ample, a quick search on June 18, 2003, produced the opportunity to bid on an MMPI Manual for Administration and Scoring together with unused testing materials as well as the opportunity to purchase Rorschach Psychodiagnostic Plates.
In this section, we have made an attempt to raise issues, offer guidance, and delineate some of the ethical issues surrounding Internet testing. Although we have touched on many issues, this is certainly not a comprehen- sive list. We have shown, however, that the current APA Ethical Principles of Psychologists and Code of Conduct (American Psychological Association, 2002) provide strong guidance for Internet testing. We believe most issues can be resolved by studying these principles and making conservative interpretations that protect both clients and the general public.
Recommendations for the Future This examination of the issue of testing on the Internet leads to several conclusions. First, and perhaps most im- portant, is that the current psychometric standards, includ- ing test reliability and validity, apply even though the way in which the tests are developed and delivered may be quite different. Unfortunately, because there are many more tests that are now available via the Internet, there is much variability in the quality of these tests. The extent to which there is documented evidence of the reliability and validity of these tests is also quite variable because many Internet tests do not seem to meet standards established by the profession. This puts consumers in the unfortunate position of having the responsibility of evaluating the quality of the information they receive, often with little knowledge and skill to do so. One conclusion is obvious: Internet testing should be subjected to the same defensible standards for assessment tools (American Educational Research Associ- ation et al., 1999) as paper-and-pencil tests when their results are used to make important decisions. Still, new methods and combinations of methods that are made pos- sible by emerging technologies will push the boundaries of existing psychometric theory, and it is up to psychologists to test and expand the limits of psychometrics to keep pace with these innovations.
The Internet provides a tremendous opportunity for testing, and with that opportunity comes a corresponding need for the ethical and professional use of these tests and a responsibility to expand our science to test the usefulness of these interventions. Despite the flash and sparkle of Internet testing, critical questions of the validity of the inferences made from test scores must be demonstrated. This is a fundamental issue of test validity that must be weighed in relation to the ease of availability, cost, and convenience of Internet testing. All of these advantages become irrelevant if scores are used in ways that are not
supported by evidence of validity. The Standards for Ed- ucational and Psychological Testing (American Educa- tional Research Association et al., 1999) provides extensive information about what is needed to justify a particular use of a test. Internet test developers and test users should carefully read the Standards and ensure that their tests are used in appropriate ways.
Although the Internet has considerable potential as a means of testing, assessment will require the integration of information obtained via this medium with other relevant information. For this reason, what is typically available on the Internet is testing in contrast to psychological assess- ment. The test results obtained on the Internet may be inaccurate for a variety of reasons and therefore there must be a professional available to verify the validity of the information and assist in interpretation. Although it is conceivable that future Internet testing methods may ap- proach a psychological assessment, the requirements for appropriate psychological assessment exceed current Inter- net capabilities. Practitioners must, therefore, be mindful of this distinction and utilize the Internet for its strength and augment it with their assessment skills.
Tests can be placed on the Internet in a manner that suggests authority and conveys confidence, although many of these tests may have little to no documentation of reliability and validity, test takers often ignore disclaimers that might appear, and self-administered tests can yield inaccurate interpretations. What is needed is considerably more accountability of the Internet site authors so that the user receives the same kind of protections obtained in traditional assessment sessions. Similarly, test developers and publishing companies that enter into Internet testing programs should ensure that Internet tests are held to the same psychometric standards as traditional tests. This would include, for example, documentation summarizing standardization samples, reliability, and validity as well as additional evidence, such as equivalence of tests delivered on Internet and paper, uniformity of stimulus quality on different displays, and so forth, to ensure high quality test administration.
There are tremendous opportunities provided by In- ternet testing. This article has described many of them, and other innovations await discovery. The importance of this new method of testing and assessment is clear, as is the need for formal guidelines for Internet-based tests and the many ways in which psychologists may use this environ- ment for a variety of applications. We encourage psychol- ogists to think creatively about how their research and practice can be improved by Internet testing. Times have changed as the Internet has brought testing out of the secure environment controlled by a licensed professional psychol- ogist or psychometrician. As testing becomes more acces- sible, it is important to realize that the principles of good testing still apply and the ethical standards for psycholo- gists are still fundamental. Balancing widespread accessi- bility with good practice presents a critical challenge to psychologists for the new millennium. There are many issues that await resolution. Over the years to come, much research and critical thinking will be required to address
161April 2004 ● American Psychologist
T hi
s do
cu m
en t i
s co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. T
hi s
ar tic
le is
in te
nd ed
s ol
el y
fo r t
he p
er so
na l u
se o
f t he
in di
vi du
al u
se r a
nd is
n ot
to b
e di
ss em
in at
ed b
ro ad
ly .
these issues. We believe that psychologists should look forward to this work with excitement and enthusiasm.
REFERENCES
Abbate, J. (1999). Inventing the Internet. Cambridge, MA: MIT Press. Ackerman, T. A., Evans, J., Park, K. S., Tamassia, C., & Turner, R.
(1999). Computer assessment using visual stimuli: A test of dermato- logical skin disorders. In F. Drasgow & J. B. Olson-Buchanan (Eds.), Innovations in computerized assessment (pp. 137–150). Mahwah, NJ: Erlbaum.
Ainsworth, M. (2002). Credentials check. Retrieved January 11, 2002, from http://www.metanoia.org/imhs/identity.htm
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: Author.
American Psychological Association. (1997). Services by telephone, tele- conferencing, and Internet: A statement by the Ethics Committee of the American Psychological Association. Retrieved January 11, 2002, from http://www.apa.org/ethics/stmnt01.html
American Psychological Association. (2002). Ethical principles of psychol- ogists and code of conduct. American Psychologist, 47, 1597–1611. Avail- able from the APA Web site: http://www.apa.org/ethics/code1992.html
Barak, A. (1999). Psychological applications on the Internet: A discipline on the threshold of a new millennium. Applied & Preventive Psychol- ogy, 8, 231–246.
Barak, A., & English, N. (2002). Prospects and limitations of psycholog- ical testing on the Internet. Journal of Technology in Human Services, 19, 65– 89.
Drasgow, F., & Olson-Buchanan J. B. (Eds.). (1999). Innovations in computerized assessment. Mahwah, NJ: Erlbaum.
Drasgow, F., Olson-Buchanan, J. B., & Moberg, P. J. (1999). Develop- ment of an interactive video assessment: Trials and tribulations. In F. Drasgow & J. B. Olson-Buchanan (Eds.), Innovations in computerized assessment (pp. 177–196). Mahwah, NJ: Erlbaum.
Epstein, J., Klinkenberg, W. D., Wiley, D., & McKinley, L. (2001). Insuring sample equivalence across Internet and paper-and-pencil as- sessments. Computers in Human Behavior, 17, 339 –346.
Funke, U., & Schuler, H. (1998). Validity of stimulus and response components in a video test of social competence. International Journal of Selection & Assessment, 6, 115–123.
Hambleton, R. K., & Swaminathan, H. (1985). Item response theory: Principles and applications. Boston: Kluwer Academic.
Howard, B., Paridaens, O., & Gramm, B. (2001, 2nd Quarter). Informa-
tion security: Threats and protection mechanisms. Alcatel Telecommu- nications Review, 117–121.
Hulin, C. L., Drasgow, F., & Parsons, C. K. (1983). Item response theory: Application to psychological measurement. Homewood, IL: Irwin.
Internet Software Consortium. (2001, July). Internet domain survey. Re- trieved January 11, 2002, from http://www.isc.org/ds/WWW-200107/ index. html
Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Erlbaum.
Matarazzo, J. D. (1990). Psychological assessment versus psychology testing: Validation from Binet to the school, clinic, and courtroom. American Psychologist, 45, 999 –1017.
Mead, A. D., & Drasgow, F. (1993). Equivalence of computerized and paper-and-pencil cognitive ability tests: A meta-analysis. Psychological Bulletin, 114, 449 – 458.
Nielsen//NetRatings. (2001). Hot off the net. Retrieved January 11, 2002, from http://www.nielsen-netratings.com/hot_off_the_net.jsp
Nielsen//NetRatings. (2003). Monthly usage statistics. Retrieved May 1, 2003, from http://www.nielsen-netratings.com/news.jsp?section�dat_to
Olson-Buchanan, J. B. (2002). Computer-based advances in assessment. In F. Drasgow & N. Schmitt (Eds.), Measuring and analyzing behavior in organizations (pp. 44 – 87). San Francisco: Jossey-Bass.
Pew Internet & American Life Project. (2003a). The ever-shifting internet population. Washington DC: Author.
Pew Internet & American Life Project. (2003b). Internet activities. Re- trieved May 1, 2003, from http://www.pewinternet.org/reports/chart .asp?img�Internet_A8.htm
Reno, J. (2000, September). Statement by the Attorney General. Sympo- sium of the Americas: Protecting intellectual property in the digital age. Retrieved January 11, 2002, from http://www.usdoj.gov/archive/ ag/speeches/2000/91200agintellectualprop.htm
Sands, W. A., Waters, B. K., & McBride, J. R. (Eds.). (1997). Comput- erized adaptive testing: From inquiry to operation. Washington, DC: American Psychological Association.
Segall, D. O. (2001). ASVAB testing via the Internet. Unpublished manuscript.
Schmit, M. J. (2001, September). Use of psychological measures for online recruitment and pre-employment selection. In L. Frumkin (Chair), Internet-based assessment: State of the art in testing. Sympo- sium conducted at the 109th Annual Conference of the American Psychological Association, San Francisco, CA.
U.S. Department of Commerce. (2002). A nation online: How Americans are expanding their use of the Internet. Washington, DC: Author.
Vispoel, W. P. (1999). Creating computerized adaptive tests of music aptitude: Problems, solutions, and future directions. In F. Drasgow & J. B. Olson-Buchanan (Eds.), Innovations in computerized assessment (pp. 151–176). Mahwah, NJ: Erlbaum.
162 April 2004 ● American Psychologist
T hi
s do
cu m
en t i
s co
py ri
gh te
d by
th e
A m
er ic
an P
sy ch
ol og
ic al
A ss
oc ia
tio n
or o
ne o
f i ts
a lli
ed p
ub lis
he rs
. T
hi s
ar tic
le is
in te
nd ed
s ol
el y
fo r t
he p
er so
na l u
se o
f t he
in di
vi du
al u
se r a
nd is
n ot
to b
e di
ss em
in at
ed b
ro ad
ly .
Public Health Agency |
Mission Statement and Link |
Logo |
1. Action on Smoking and Health (ASH) |
Action on Smoking and Health (ASH) seeks to publicize the risks associated with tobacco smoking and campaign for greater restrictions on cigarette and tobacco sales.
http://ash.org/ |
|
2. AIDS Action |
AIDS Action’s mission is to advocate on a national level for people living with or affected by HIV/AIDS and the organizations that serve them.
http://www.aidsaction.org/ |
|
ATSDR's mission is to prevent exposure and adverse human health effects and diminished quality of life associated with exposure to hazardous substances from waste sites, unplanned releases, and other sources of pollution present in the environment.
http://www.atsdr.cdc.gov/ |
|
|
4. All India Institute of Hygiene and Public Health |
All India Institute of Hygiene and Public Health (AIIH&PH), is a pioneering Indian institute for research and training in public health and allied sciences in Kolkata.
http://aiihph.gov.in/ |
|
5. American Cancer Society |
The American Cancer Society (ACS) is a nationwide voluntary health organization dedicated to eliminating cancer.
Cancer.org |
|
6. The American Diabetes Association (ADA) |
The American Diabetes Association (ADA) is a United States-based association working to fight the consequences of diabetes and to help those affected by diabetes.
Diabetes.org |
|
7. American Heart Association (AHA) |
The American Heart Association (AHA) is a non-profit organization in the United States that fosters appropriate cardiac care in an effort to reduce disability and deaths caused by cardiovascular disease and stroke.
http://www.heart.org/HEARTORG/
|
|
American Public Health Organization (APHO) is the oldest and most diverse public health professional organization in the world.
|
|
|
9. Association of State and Territorial Health Officials (ASTHO)
|
Association of State and Territorial Health Officials (ASTHO): The largest nonprofit group in the US for representing public health agencies in the country. http://www.astho.org/ |
|
10. American Society of Tropical Medicine and Hygiene (ASTMH) |
American Society of Tropical Medicine and Hygiene (ASTMH) is a worldwide group of scientists, clinicians, and program professionals who promote public health and prevent tropical diseases.
http://www.astmh.org/ |
|
11. California Center for Public Health Advocacy |
The California Center for Public Health Advocacy tackles the underlying factors that perpetuate childhood obesity and undermine parents’ desire to keep their children health.
http://www.publichealthadvocacy.org/ |
|
12. California Association of Public Health Laboratory |
The mission of the California Association of Public Health Laboratory Directors is to improve and protect public health through the promotion and advancement of public health laboratory practice.
http://www.caphld.org/ |
|
13. Centers for Disease Control and Prevention |
Centers for Disease Control and Prevention (CDC) is the leading national public health institute of the United States.
http://www.cdc.gov/
|
|
14. Center for Medicare and Medicaid Services (CMS) |
Center for Medicare and Medicaid Services CMS, a major component of HHS, was created in 1977 to bring together the two largest Federal health care programs—Medicare and Medicaid.
https://www.cms.gov/ |
|
Council of State and Territorial Epidemiologists is an organization of member states/territories that represent public health epidemiologists.
http://www.cste.org/ |
|
|
16. The Consortium of Universities for Global Health (CUGH)
|
The Consortium of Universities for Global Health (CUGH), established in 2008, is a consortium of institutions in high, middle and low income countries involved in global health.
http://www.cugh.org/ |
|
17. Equal Access |
Equal Access is a non-governmental organization that address the most critical challenges affecting people in the developing world such as women & girls’ empowerment, youth life skills & livelihoods, human rights, health and civic participation & governance.
http://www.equalaccess.org/ |
|
18. European Centre for Disease Prevention and Control (ECDC) |
The European Centre for Disease Prevention and Control (ECDC) is an independent agency of the European Union (EU) whose missionis to strengthen Europe's defenses against infectious diseases.
http://ecdc.europa.eu/en/Pages/home.aspx |
|
19. The European Public Health Association (EUPHA) |
The European Public Health Association (EUPHA) is an international, multidisciplinary, scientific organization, bringing together around 14,000 public health experts for professional exchange and collaboration throughout Europe.
https://eupha.org/ |
|
20. Food and Drug Administration |
FDA ensures the safety of foods and cosmetics and the safety and efficacy of pharmaceuticals, biological products, and medical devices.
http://www.fda.gov/ |
|
21. The George Institute for Global Health |
The George Institute for Global Health is an independent medical research institute dedicated to improving global health.
http://www.georgeinstitute.org/ |
|
22. Global Coalition Against Child Pneumonia |
The Global Coalition Against Child Pneumonia exists to raise global awareness about the deadly toll of the number 1 killer of children - pneumonia.
http://worldpneumoniaday.org/ |
|
23. Global Footprint Network |
Global Footprint Network develops and promotes tools for advancing sustainability, including the Ecological Footprint and biocapacity, which measure the amount of resources we use and how much we have.
http://www.footprintnetwork.org/en/index.php/GFN/ |
|
24. Global Health Security Initiative (GHSI) |
The Global Health Security Initiative (GHSI) is an international partnership between countries in order to supplement and strengthen their preparedness to respond to threats of biological, chemical, radio-nuclear terrorism (CBRN) and pandemic influenza.
http://www.ghsi.ca/english/index.asp |
|
25. The Genome-based Research and Population Health International Network (GRaPH-Int) |
The Genome-based Research and Population Health International Network (GRaPH-Int) is an international collaboration of experts and researchers focused in the area of population health.
http://www.cdc.gov/genomics/hugenet/publications/populationhealth.htm |
|
26. The Health Threat Unit |
The Health Threat Unit is responsible for terrorism surveillance and early warning of biological, chemical, and radiological threats within the European Union.
http://ec.europa.eu/health/index_en.htm |
|
27. The Hispanic Health Council (HHC) |
The Hispanic Health Council (HHC) was established in 1978 in response to the health care crisis in the Puerto Rican community.
http://www.hispanichealth.com/hhc/ |
|
28. Howard Center |
Howard Center offers professional crisis and counseling services to children and adults; supportive services to individuals with autism.
http://www.howardcenter.org/home |
|
29. The Immunization Alliance |
The Immunization Alliance is an American vaccine advocacy consortium, assembled under auspices of the American Academy of Pediatrics (AAP) in May 2008.
http://vaccinealliance.org/ |
|
30. India HIV/AIDS Alliance |
India HIV/AIDS Alliance operates in partnership with civil society, government and communities to support sustained responses to HIV in India that protect rights and improve health.
http://www.allianceindia.org/ |
|
31. Indian Heart Association (IHA) and Indian Stroke Association |
The Indian Heart Association (IHA) and Indian Stroke Association (ISA), is an NGO and non-profit dedicated to raising cardiovascular and stroke health awareness among the South Asian population.
http://indianheartassociation.org/ |
|
32. Institute for Health Metrics and Evaluation (IHME) |
Institute for Health Metrics and Evaluation (IHME) is a research institute working in the area of global health statistics and impact evaluation at the University of Washington in Seattle.
http://www.healthdata.org/ |
|
33. The International Association of National Public Health Institutes (IANPHI) |
The International Association of National Public Health Institutes (IANPHI) is a member organization of government agencies working to improve national disease prevention and response.
http://www.ianphi.org/ |
|
34. The International Coalition for Trachoma Control (ICTC |
The International Coalition for Trachoma Control (ICTC) is a non-profit organization that was established in 2004 for the purpose of contributing to the global effort to eliminate blinding trachoma and to advocate and implement the SAFE strategy.
http://www.trachomacoalition.org/ |
|
35. The International Epidemiological Association (IEA) |
The International Epidemiological Association (IEA) facilitates communication amongst those engaged in research and teaching of epidemiology throughout the world, and to encourage its use in all fields of health including social, community and preventative medicine.
http://ieaweb.org/ |
|
36. The International Vaccine Institute |
The International Vaccine Institute (IVI) is an international nonprofit organization that was founded on the belief that the health of children in developing countries can be dramatically improved by the use of new and improved vaccines.
http://www.ivi.org/web/www/home |
|
37. Meds & Food for Kids (or MFK) |
Meds & Food for Kids (or MFK) is a nonprofit organization dedicated to treating and preventing child malnutrition in Haiti by producing fortified peanut-based foods.
https://mfkhaiti.org/
|
|
38. National Association of County and City Health Officials |
National Association of County and City Health Officials (NACCHO): This organization is the comprehensive representative body for more than 2,700 local health departments in the US. http://www.naccho.org/
|
|
National Association of Local Boards of Health (NALBOH): This national association represents more than 1,000 local health boards located in all 50 US states. http://www.nalboh.org/
|
|
|
National Cervical Cancer Coalition (NCCC): Founded in 1996 as a grassroots group that helps to serve women with cervical cancer and HPV disease, or at risk. http://www.nccc-online.org/
|
|
|
41. National Environmental Health Association (NEHA) |
National Environmental Health Association (NEHA): A national professional society of environmental health practitioners. Started originally in the state of California. http://www.neha.org/ |
|
42. National Institutes of Health (NIH |
National Institutes of Health (NIH): NIH is the nation’s top medical research organization, and it makes many important discoveries that boost health and save people’s lives.
http://www.nih.gov/ |
|
43. Office of the Assistant Secretary for Preparedness and Response (ASPR) |
The Office of ASPR (formerly the Office of Public Health Emergency Preparedness) serves as the Secretary's principal advisory staff on matters related to bioterrorism and other public health emergencies.
http://www.phe.gov/about/Pages/default.aspx |
|
Pan American Health Organization (PAHO): A health agency that has over a century of experience is boosting the health and living conditions of the countries in the Americas.
http://www.paho.org/ |
|
|
The Public Health Foundation (PHF): Dedicated to creating communities of good health through the use of training, research and technical support. http://www.phf.org/Pages/default.aspx |
|
|
46. Science & Environmental Health Network |
The Science & Environmental Health Network (SEHN) is a non-profit organization founded in 1994. Its principal aim is to use law and best practices to combat cumulative impacts, especially in matters relating to public health and the environment.
http://www.sehn.org/about.html |
|
47. SOPHE |
SOPHE brings together public health education professionals to promote the health of society. SOPHE advances research and best practices and advocates to put policy into action. |
|
48. The Stop TB Partnership |
The Stop TB Partnership was established in 2000 to eliminate tuberculosis as a public health problem.
http://www.stoptb.org/stop_tb_initiative/ |
|
49. Substance Abuse and Mental Health Service Administration (SAMHSA) |
Substance Abuse and Mental Health Service Administration (SAMHSA) works to improve the quality and availability of prevention, treatment, and rehabilitative services in order to reduce illness, death, disability, and cost to society resulting from substance abuse and mental illnesses.
http://www.samhsa.gov/ |
|
World Health Organization (WHO): WHO directs and coordinates health activities within the system of the United Nations. http://www.who.int/en/ |
|
1
Zuzana Bic, Dr.P.H., MUDr. E-‐‑mail: [email protected]
2019
PubHlth 1 – Principles of Public Health Online Class, Fall 2019
Department of Population Health and Disease Prevention, Program in Public Health, College of Health Sciences, University of California, Irvine
Cover design by UCI student FahrzadAlikoza, alumni PubHlth 199 & 198 Public Health for the Corporate World
2
Instructor Zuzana Bic, DrPH, MUDr. Email:[email protected]
Online office hours: Mon. & Fri. from 7:00 9:00 a.m.; Chat or Zoom
Contact Information
Accessing midterm &final exams:contact Proctor U (see class website) Canvas technical issues: https://eee.uci.edu/help/faq/general/
TA Name: TBA &information will be on the class website
Course Prerequisites
Official: N/A Non-‐Official:
1. Self-‐motivated and self-‐disciplined 2. Able to learn without personal, face-‐to-‐face interaction with the instructor 3. Able to follow written instructions 4. Not easily frustrated when there are technical problems
Course Description
Introduces the major concepts and principles of public health and the determinants of health status in communities. Emphasizes the ecological model that focuses on the linkages and relationships among multiple natural and social determinants affecting health. 4 hours lecture
Course Summary
This is an online class. There are no meetings in the classroom. Audio-‐lectures, Power Point presentations, additional readings, web-‐links, study materials, assignments and exams are online. Communication with instructor is by e-‐mail, via discussion board on the class website, and on Skype. All assignments are submitted, graded and returned through the class website.
Course Material
E-‐textbook Available Directions on class website
REQUIRED TEXT: Online version as an E-‐book and/or hard copy is available at your University bookstore or directly from the publishing house at: http://www.jblearning.com/ More information will be on your class website.
INTRODUCTION TO PUBLIC HEALTH, Mary-‐Jane Schneider, 5th edition, 2017, ISBN: 978-‐1-‐284-‐ 08923-‐3, Jones and Bartlett Publishers, 40 Tall Pine Drive, Sudbury, MA 01776, Phone: 978-‐443-‐ 5000,www.jbpub.com
Lectures, links and videos are available on the class website.
Course Objectives
COURSE OBJECTIVES:
UPON SUCCESSFUL COMPLETION OF THIS COURSE, STUDENTS WILL BE ABLE TO: q Understand “What is Public Health”. DISCUSSION FORUM q Synthesize and apply the concepts of public health for health-‐related conditions. ACTIVE
PARTICIPATION q Understand the role of public health by analyzing topics on public health and society from
current media/current research, and suggest/develop various types of intervention. RESEARCH PAPER
q Speak about public health problem and evaluate it based on PH concepts. PRESENTATION WITH PEER REVIEW q Synthesize and apply information pertaining to public health in the terms of science, politics,
policy, epidemiology, statistics, prevention and society. MIDTERM AND FINAL EXAM q Gain a stronger understanding of the concepts of public health. WEEKLYQUIZZES
3
Email Info • Check your uci.edu email on a regular and consistent basis! • Configure your uci.edu account to forward your emails to your personal email account if you
do not usually check your uci.edu email over the summer. • On all email correspondence, always include your first and last name, the class in which you
are enrolled in (“PH 1” in either in the text of the message or in the subject line), and the original email(s) if a reference is necessary.
• Allow 24 hours for a response from your instructor and/or TA
Course Requirements
Course Requirements • Lectures, Links and Videos each week • Discussion Forum assigned topic each week • Presentation a 5-‐minute audio narrative PowerPoint presentation with 6-‐10 slides and a
peer review evaluation of a classmate’s presentation • Research Paper based on the specific public health problem and will be the same topic
as presentation • Active Participation each week • Quizzes 10 questions each for 7 quizzes • Exams midterm and final exam; $15 fee for Proctor U for each exam • Extra Credit up to 9 points of possible extra credit
*More detailed information at the extended syllabus at your class website
Estimated Weekly Time Commitment
Lectures Discussions Research Paper Exams Quizzes Presentation with peer review TOTAL
120 minutes 120 minutes 120 minutes 150 minutes 30 minutes 60 minutes Approx. 10 hours each week for assignments
Note: Online classes should take the SAME amount of time as traditional classroom courses.
Due Dates Write down these dates on your calendar. Do NOT rely on reminders from your instructor!! Please note: All due dates are in Pacific Daylight Time.
Week 1 Active Participation Discussion Forum Quiz
Every Sunday @ 11:00 p.m. Every Wednesday (Post 1) and every Sunday (Post 2) @ 11:00 p.m. Friday: Open from 1:00 a.m. – 11:00 p.m.
Week 2 Active Participation Discussion Forum Quiz (Extra Credit) Syllabus Quiz
Every Sunday @ 11:00 p.m. Every Wednesday (Post 1) and every Sunday (Post 2) @ 11:00 p.m. Friday: Open from 1:00 a.m. – 11:00 p.m. Sunday @ 11:00 p.m.
Week 3 Active Participation Discussion Forum Quiz
Every Sunday @ 11:00 p.m. Every Wednesday (Post 1) and every Sunday (Post 2) @ 11:00 p.m. Friday: Open from 1:00 a.m. – 11:00 p.m.
Week 4 Active Participation Discussion Forum Quiz
Every Sunday @ 11:00 p.m. Every Wednesday (Post 1) and every Sunday (Post 2) @ 11:00 p.m. Friday: Open from 1:00 a.m. – 11:00 p.m.
4
Week 5 Active Participation Discussion Forum MIDTERM EXAM
Every Sunday @ 11:00 p.m. Every Wednesday (Post 1) and every Sunday (Post 2) @ 11:00 p.m. Friday: Open from 1:00 a.m. – 11:00 p.m.
Week 6 Active Participation Discussion Forum Presentation Mandatory Peer Review
Every Sunday @ 11:00 p.m. Every Wednesday (Post 1) and every Sunday (Post 2) @ 11:00 p.m. Friday @ 11:00 p.m. Sunday @ 11:00 p.m
Week 7 Active Participation Discussion Forum Quiz
Every Sunday @ 11:00 p.m. Every Wednesday (Post 1) and every Sunday (Post 2) @ 11:00 p.m. Friday: Open from 1:00 a.m. – 11:00 p.m.
Week 8 Active Participation Discussion Forum Research Paper
Every Sunday @ 11:00 p.m. Every Wednesday (Post 1) and every Sunday (Post 2) @ 11:00 p.m. Friday @ 11:00 p.m.
Week 9 Active Participation Discussion Forum Quiz
Every Sunday @ 11:00 p.m. Every Wednesday (Post 1) and every Sunday (Post 2) @ 11:00 p.m. Friday: Open from 1:00 a.m. – 11:00 p.m.
Week 10 Active Participation Discussion Forum Quiz
Every Sunday @ 11:00 p.m. Every Wednesday (Post 1) and every Sunday (Post 2) @ 11:00 p.m. Friday: Open from 1:00 a.m. – 11:00 p.m.
Week 11 FINAL EXAM Monday: Open from 1:00 a.m. – 11:00 p.m.
Late Policy • Active Participation/Discussions: Late submission (-‐ 5) points will be deducted for 24-‐hour
period after 11:00 p.m. No acceptance after 24 hours. • Late quizzes will NOT be accepted • Research Paper: Late submission (-‐ 5) points will be deducted for 24-‐hour period after 11:00
p.m. No acceptance after 24 hours. • Presentation: Late submission (-‐ 5) points will be deducted for 24-‐hour period after 11:00
p.m. NO acceptance after 24 hours.
Grading System
Discussion Forum (9%) Research Paper (24%) 7 Quizzes (2%) 2 Exams (50%) Presentation with review (6%) Active Participation (9%) TOTAL (100%)
*Extra credit is added to (Active Participation 9% value)
100 points (10 points each week) 75 points 35 points (5 points each week) 120 points (60 points each) 30 points (25 points for presentation; 5 points for review) 100 points (10 points each week) 460 points
*TBA points
Mandatory Turnitin Submission
Please note you need to submit your Research Paper to turnitin via Canvas under Assignments – in your class website. Your paper will be NOT accepted if it is not submitted to turnitin through Canvas link. This means you must use the Canvas link to submit your paper to turnitin.
5
Grading Scale NOTE: Grades are not based on a curve. Your grade will be calculated by adding the points you earn on each assignment. Please note each assignment has its own VALUE!!! Grade is calculated as a summation of %.
Out of 100% Scale:
A+ ≥ 100 > A ≥ 93.5 > A-‐ ≥ 90 B+ ≥ 86.5 > B ≥ 83.5 > B-‐ ≥ 80 C+ ≥ 76.5 > C ≥ 73.5 > C-‐ ≥ 70
D+ ≥ 66.5 > D ≥ 63.5 > D-‐ ≥ 60 > F
Lecture Schedule
Weeks 1 & 2: Weeks 3 & 4: Weeks 5 & 6: Weeks 7 & 8: Weeks 9 & 10:
Chapters 1, 2, 3, 4, 5, 6 and links Chapters 7, 8, 9, 10, 11, 12 and links Chapters 13, 14, 15, 16, 17, 18 and links Chapters 19, 20, 21, 22, 23, 24 and links Chapters 25, 26, 27, 28, 29, 30, 31 and links
CODE OF CONDUCT: All participants in the course are bound by Code of Conduct, https://aisc.uci.edu/students/academic-integrity/index.php (link to browser) Anyone caught cheating or in any other way in violation of the university policy on academic honesty will receive an F in the class. There are no exceptions to this rule. "Learning, research, and scholarship depend upon an environment of academic integrity and honesty. This environment can be maintained only when all participants recognize the importance of upholding the highest ethical standards. All student work, including quizzes, exams, reports, and papers must be the work of the individual receiving credit. Academic dishonesty includes, for example, cheating on examinations or any assignment, plagiarism of any kind (including improper citation of sources), having someone else take an examination or complete an assignment for you (or doing this for someone else), or any activity in which you represent someone else’s work as your own. Violations of academic integrity will be referred to the Office of Academic Integrity and Student Conduct. The impact on your grade will be determined by the individual instructor’s policies. Please familiarize yourself with UCI’s Academic Integrity Policy (https:// aisc.uci.edu/policies/academic-integrity/index.php) and speak to your instructor if you have any questions about what is and is not allowed in this course."
INSTRUCTIONS
How to listen to the LECTURES, see video clips, access links and print lecture notes 1. Log into the class homepage. 2. Click on the appropriate lecture, video clips and links. 3. Click on lecture notes – you may print the information.
How to do the DISCUSSIONS 1. Discussions: Week 1-‐‑2 discussions: Please find your assigned group under Course Documents section 2. Weeks 3-‐‑10: You will be automatically assigned by the TA through Canvas. 3. You will write and respond to the assigned question by submitting your Discussion Post #1 every Wednesday
before 11:00 p.m. (minimum 150 words – maximum 200 words and type “word count” and provide an accurate word count at the end of the discussion post)
4. As soon as you post your discussion response in the Forum, you will see your classmates’ discussion posts. Please choose one and write your Discussion Post #2 in response to it and post it every Sunday before 11:00 p.m. (minimum 150 words – maximum 200 words and type “word count” and provide an accurate word count at the end of the discussion post)
5. Each “Discussion Post” should include your thoughts and opinion. See rubric for more information. See sample under course document.
How to do the ACTIVE PARTICIPATION 6. Write 1 paragraph with your thoughts, opinion, (see rubric below and sample at course document) with a
minimum of 150 words and maximum of 200 words and type “word count” and provide an accurate word count at the end of the paragraph.
7. Submit every Sunday before 11:00 p.m. How to write and submit the RESEARCH PAPER
1. Follow the rubric and type your research paper in a Word document (doc or docx) or pdf 2. To submit: Click on class homepage, click on “Assignments”, which is linked to turnitin, and upload your paper. 3. You may submit your final paper only once and only paper submitted into canvas link will be accepted.
6
How to take the QUIZZES From the homepage, click on “Quizzes” under each week, open only every Friday from 1:00am – 11:00 p.m. You will have 20 minutes for 10 questions – only T/F and/or multiple-‐‑choice questions.
How to check your GRADE BOOK and your points 1. Check your updated score every Tuesday after 11:00 p.m. From the class homepage, click on Gradebook. 2. Please review your points and feedback for each assignment. If you have questions, please email your instructor
and request for feedback so you do not repeat the same mistake for the remaining assignments. How to take the EXAMS
1. Review the study guide that is posted under the Study Guide for midterm and for final exam. 2. Log into the class homepage and click on the exam you need to take. Follow the instructions for saving your
answers and submitting your exam. 3. Please be aware that the midterm and final exams are open from 1:00 a.m. – 11:00 p.m. You will have 60 minutes
for 60 questions for each exam. The first time you can start to take exam is at 1:00 a.m. and the last time when you can start to take the exam is at 11:00 p.m. You need to finish at 11:59 p.m.
4. You will take the midterm and final exam online, in a proctored setting. You will be connected to a live person during your exam. He or she will be there to guide you through the process and assist with any technical problems. Please read detailed instructions on the course home page – in the “Course Documents” section. You need to pay to Proctor U for your exams: $15 for each 60-‐‑minute exam (60 questions).
5. If you would like to discuss the questions you missed, you will need to talk to the instructor during office hours or send an email to the instructor.
How to do the PRESENTATION and PEER REVIEW 1. You will prepare a 5-‐‑minute audio narrative with 6-‐‑10 slides and post your presentation on the class website. 2. You will follow the rubric and complete a peer review evaluation for one selected presentation in your group. 3. This peer review is suppose to be a critical analysis of the assigned presentation. Based on the rubric -‐‑ Please
answer the categories in a paragraph style. 4. Please write a (minimum 150 words – maximum 200 words and type “word count” and provide an accurate
word count at the end of the per review. How to do EXTRA CREDIT (TBA points)
The information regarding extra credit will be available at your class website under course documents and also under specific week.
GRADING CRITERIA Research Paper (75 points) Point Deductions Failed to submit the assignment Please see rubric below
-‐‑ 75 points
Discussions (10 points/week) Failed to post 2 discussions Failed to post 1 discussion Not reaching the minimum of 150 words or going over the maximum of 200 words Failed to type “Word Count” at the end and include an ACCURATE number of words Failed to include your opinions, thoughts/ideas in your responses (see rubric below) Failed to post Discussion Post #1 by Wednesday by 11:00 p.m. (see late policy page 3)
-‐‑ 10 points -‐‑ 5 points -‐‑ 5 points -‐‑ 0.5 points -‐‑ 5 points -‐‑5 points
Active Participation (10 points) Failed to submit Not reaching the minimum of 150 words or going over the maximum of 200 words Failed to type “Word Count” at the end and include an ACCURATE number of words Failed to include your opinions, thoughts/ideas in your responses (see rubric below) Failed to submit on time, Sunday at 11:00 p.m. (see the late policy page 3)
-‐‑ 10 points -‐‑ 5 points -‐‑ 0.5 points -‐‑ 5 points -‐‑5 points
7
Scores for assignments are based on the grading rubrics below.
RESEARCH PAPER RUBRIC You will choose your topic from the pool of topics at your class website. Required Elements 5 Points Separate cover page
• Must include public health topic, title, name, ID, class, year and instructor’s name / 1
Total pages: 3 • Single spaced with minimum of 2 & 2/3 pages of written text and 1/3 page of references
/ 1
Paragraph titles • Paper must be organized into paragraphs with the following titles: (1) Public Health
Problem; (2) How to Solve the Problem; (3) Public Health in the Future
/ 1
Correct spelling and grammar / 1 Format 12 pt. font, Times New Roman, 1 to 1.25 in. margins on sides, top and bottom)
Research Paper total pages: 4 (cover page and 3 pages)
/ 1
References 10 Points 5 references 3 scholarly references References in AMA/APA format All references in-‐text citations
/ 5 / 3 / 1 / 1
1. Public Health Problem 20 Points Description and analysis of the specific PH problem. Please include: #1. Why does this problem exist in this society? #2: How did this happen #3: Where did this happen? #4: What are the risks for the community in the future? #5: What are/were the etiological factors?
/ 4 / 4 /4 /4 /4
2. How to Solve the Public Health Problem 20 Points Description of how the problem was solved and needs to be solved Please include: #1: Include specific 2 examples, such as public health intervention programs and public health policies #2: Include 2 examples of how the problem was solved from other cultures/countries and from history/from the past
/ 10
/10
3. Public Health in the Future 20 Points Discussion on how to prevent public health (PH) problem Please include: #1: Conclusion with justification from research literature: what should be done to prevent the recurrences of the problem – give 2 examples #2: Add your opinion on the PREVENTION of this PH problem in the future – give 2 examples
/ 10
/10
Miscellaneous Point Deductions Each quotation mark -‐ 10
8
Late submission -‐ 5 for 24-‐hours after 11:00 p.m. NO acceptance after 24 hours. Turnitin
1-‐5% No point deduction 6-‐10% -‐ 10 > 11% Score of 0 on paper
ATTENTION: Please be sure that your research paper includes only your own creative words! You are not allowed to use your previous work with a similar topic for this paper. You are not allowed to copy entire sentences from published sources. Quotation marks are not allowed. Please explore the turnitin website: http://turnitin.com/
TOTAL 75 Points
RUBRIC FOR DISCUSSION POST #1 AND DISCUSSION POST #2 Points Category Explanation 5 Thought-‐‑provoking or challenges new
idea informed by reading or lesson This rating is given to posts that present a new idea or challenges others posts based on information from lesson or textbook or other scholarly source.
3 Opinion based on information from reading or lesson
This rating is given when a person writes a fact-‐‑based forum post. The facts could come from a lesson or a chapter from the textbook, or another scholarly external source.
2 Answered as required, but nothing more
This rating is given when a post answers all parts of my question, but does nothing more. May show an absence of depth or thought.
0 Inappropriate or insufficient postings This rating is given to posts that do not meet my grading requirements. Used for: agreement without new substance, general humor, posts that do not fit into the current discussion.
RUBRIC FOR ACTIVE PARTICIPATION FOR WRITTEN PART Points Category Explanation 10 Thought-‐‑provoking or challenges new
idea informed by reading or lesson This rating is given to posts that present a new idea or challenges others posts based on information from lesson or textbook or other scholarly source.
6 Opinion based on information from reading or lesson
This rating is given when a person writes a fact-‐‑based forum post. The facts could come from a lesson or a chapter from the textbook, or another scholarly external source.
4 Answered as required, but nothing more
This rating is given when a post answers all parts of my question, but does nothing more. May show an absence of depth or thought.
0 Inappropriate or insufficient postings This rating is given to posts that do not meet my grading requirements. Used for: agreement without new substance, general humor, posts that do not fit into the current discussion.
0 or 10 Only if there is a question, this will apply
Correct or incorrect answer
PRESENTATION AND PEER REVIEW RUBRIC LOGISTICS 8 Points
9
Clarity,organization, informative level and presentation style Topic on the title slide Full name Time limit (5 -‐ 6 minutes) Presentation style Clarity No. of slides: 5 -‐ 10
/ 1 / 1 / 2 / 1 / 1 / 2
PRESENTATION SLIDES 17 Points Final presentation should include the following: 1. Public Health Problem Description and analysis of the specific PH problem. Please include: #1. Why does this problem exist in this society? #2: How did this happen, #3: Where did this happen? #4: What are the risks for the community in the future? #5: What are/were the etiological factors?
2. How to solve the Public Health problem How the problem needs to be solved or how was it solved with specific examples referenced. Please include: #1: Description of 2 strategies, such as PH intervention programs and PH policy #2: Please add 2 examples of how the problem was solved from other cultures/countries or from history
3. Public Health in the Future How to prevent the PH problem. Please include: #1: Conclusion with justification from research literature: what should be done to PREVENT the reoccurrence of the PH problem. #2: Add your opinion on the PREVENTION of this PH problem in the future.
4. References Use AMA/APA citation format.
1. Public Health problem
#1: Why? #2: How? #3: Where? #4: What? #5: Factors
2. How to solve the problem
#1: 2 strategies
#2: 2 examples
3. Public health in the future
#1: Prevent
#2: Opinion
4. References on the slide
/1 /1 /1 /1 /1
/2.5
/2.5
/ 2.5
/ 2.5
/2
PEER REVIEW 5 Points Check if all components are included in another presentation. This review is suppose to be a critical analysis of the assigned presentation. Please answer the following categories in a paragraph style. #1:PH problem #2:How to solve the problem #3:Public Health in the future #4:References #5: Additional thoughts, opinion
“word count” word count listed at the end word count of a minimum of 150 or not going over the maximum word count of 200
Peer Review
/0.5 /1 /0.5 /0.5 /1
/0.5 /1
10
TOTAL 30 Points (25 points for presentation; 5 points for peer review)
SOME EXAMPLES OF THE CASES AND TOPICS TO BE COVERED IN PUBLIC HEALTH 1
What is Public Health? Public Health: Science, Politics, and Prevention
Analytical Methods of Public Health Epidemiology: The Basic of Science of Public Health The Role of Data in Public Health
Biomedical Basis of Public Health The Resurgence of Infectious Diseases
Social and Behavioral Factors in Health Do People Choose Their Own Health? How Psychosocial Factors Affect Health Behavior
Environmental Issues in Public Health A Clean Air Environment: The Basis of Public Health Safe Food and Drugs: An Ongoing Regulatory Battle Population: The Ultimate Environmental Health Issue
Medical Care and Public Health Why the U.S. Medical System Needs Reform Public Health and the Aging Population
The Future of Public Health Public Health in the Twenty-‐First Century: Achievements and Challenges
- Kline_077
- Kline_078
- Kline_079
- Kline_080
- Kline_081
- Kline_082
- Kline_083
- Kline_084
- Kline_085
- Kline_086
- Kline_087
- Kline_088
- Kline_089
- Kline_090
Week 8: Data Collection
What does it mean to say that someone is tall or short or to say that a person’s cholesterol level is high or low? Relative descriptions such as these only have meaning because an individual is being compared to a reference group. An American woman with a height of 70 inches might be considered tall because she is being compared to other American women. A man with the same height would be considered average if compared to other American men. If that same man were on a basketball team, he might be considered short.
Along the same lines, it is only meaningful to talk about high and low test scores when the scores are compared to a standard. A frequently used standard is the scores of a normative sample. This week, you explore characteristics of a good normative sample and examine how to address sampling considerations related to psychological tests.
Objectives
Students will:
· Analyze sampling considerations related to psychological tests
· Justify the use of sampling and data collection methods for psychological testing situations
Required Resources
Readings
· Kline, T. (2005). Psychological testing: A practical approach to design and evaluation . Thousand Oaks, CA: Sage. Kline, T., Psychological testing: A practical approach to design and evaluation. Copyright 2005 Sage Publications Inc. Books. Used with permission from Sage Publications via the Copyright Clearance Center.
. Chapter 4, "Collecting Data: Sampling and Screening"
· Buchanan, T., Ali, T., Heffernan, T. M., Ling, J., Parrott, A. C., Rodgers, J., & Scholey, A. B. (2005). Nonequivalence of on-line and paper-and-pencil psychological tests: The case of the prospective memory questionnaire. Behavior Research Methods, 37(1), 148–154. Retrieved from the Walden Library databases.
· Naglieri, J., Drasgow, F., Schmit, M., Handler, L., Prifitera, A., Margolis, A., & Velasquez, R. (2004). Psychological testing on the Internet: New problems, old issues. American Psychologist, 59(3), 150–162. Retrieved from the Walden Library databases.
Week 8 Discussion
Discussion 1: Sampling
In order to compare individuals to a population, that population must be sampled using a standardization sample, also referred to as a norm group. This group represents the population for which a test is intended. There are a number of important considerations when choosing a test’s standardization sample, such as how large your sample needs to be and how you plan to access your participants and induce them to participate. You may also need to consider whether the participants should be representative of the general population or of a specialized subpopulation and how diverse the sample should be. You may have general norms or separate norms according to demographic characteristics, such as age and gender. How to address these considerations depends on the goals of your assessment and will affect the test interpretation.
With these thoughts in mind:
Post by Day 3 an explanation of the three most important considerations related to sampling that you would need to address in your Final Project. Explain how you might ensure that your sample represents the population of interest.
Be sure to support your postings and responses with specific references to the Learning Resources.
Discussion 2: Optimizing Sampling and Data Collection
The data that you collect from your norm group are called normative data. These data allow you to see what the results of your test instrument should be, so that you have a standardized score against which to compare others. In other words, the data allow you to equate scores across different tests of the same construct and let you compare individuals to each other. Now that you have considered sampling the population of interest for your proposed test instrument, you explore how you might optimally define your sample and collect your data if you had unlimited resources. Although it may not always be possible for practical reasons to do this in real life, this exercise nevertheless provides you with a valuable thought experiment.
As you complete this experiment, keep in mind that even with optimal sample definition and data collection, test norms are not absolute. They can change with time or with a different norm group.
With these thoughts in mind:
Post by Day 4 a description of the information you anticipate collecting through your proposed Final Project instrument. Then explain how you would optimally define your sample and collect your data if you had unlimited resources. Finally, explain advantages and disadvantages of your chosen data collection method. Support your response using the Learning Resources and the current literature.
Be sure to support your postings and responses with specific references to the Learning Resources.
Read a selection of your colleagues' postings.

Get help from top-rated tutors in any subject.
Efficiently complete your homework and academic assignments by getting help from the experts at homeworkarchive.com