How an Examination for Professional Practice in Psychology® (EPPP) Item is Developed

Most people taking a licensing examination focus on preparing for test day, not on how the examination itself came to be. Yet every question on a high-stakes licensure examination represents years of planning, expert collaboration, statistical analysis, and ongoing evaluation. That is precisely the expertise that the Association of State and Provincial Psychology Boards® (ASPPB) provides for its 66 jurisdictions in its mission to support its members in fulfilling their responsibility of public protection.

The EPPP® is the standardized licensing examination used by psychology licensing boards throughout the United States and Canada. Developed by ASPPB and first introduced in 1965, the EPPP assesses the foundational knowledge required for entry-level independent practice and supports greater consistency and mobility in psychologist licensure across jurisdictions.

“More than six decades after its introduction, the EPPP continues to serve as a cornerstone of psychology licensure throughout the United States and Canada,” said Mariann Burnetti-Atwell, PsyD, ASPPB CEO. “Throughout its history, ASPPB has remained committed to strengthening the examination through ongoing research, expert review, and evidence-based validation to help ensure it reflects contemporary professional practice. Each examination item is developed through a structured process involving licensed psychologists, psychometricians, and multiple levels of quality review to help ensure the examination remains valid, reliable, fair, and reflective of entry-level professional practice.”

The questions included on the EPPP today have been under development and testing by subject-matter experts (SMEs), who evaluate how individual questions perform after administration. Every stage is guided by established testing standards and the expertise of practicing psychologists.

The Foundation: Defining What the Examination Measures

The development process begins with a Job Task Analysis (JTA), conducted approximately every 7 to 10 years to establish the EPPP blueprint. Informed by panels of SMEs and validated through survey responses from thousands of licensed or registered psychologists, the JTA identifies the knowledge and skills most relevant to contemporary entry-level psychological practice. The resulting blueprint defines the examination specifications and serves as the foundation for all EPPP item development.

The next step in the process is a standard-setting study to establish the examination’s passing score. Using the Angoff method, SMEs evaluate each examination item and estimate the likelihood that a minimally competent candidate would answer it correctly. These ratings are used to determine the performance standard required for licensure.

Step 1: Subject Matter Experts Write New Items

Once the examination blueprint has been established, new examination items are developed to match its specifications.

Each year, dozens of licensed psychologists with expertise in the EPPP domain areas and training in item writing participate in an item-writing workshop. These psychologists are trained in best practices for developing examination items before drafting questions that align with specific blueprint requirements.

Importantly, EPPP items are written by qualified psychology professionals—not by artificial intelligence or automated systems.

Step 2: Collaborative Review and Refinement

The development of examination items begins with the individual item writers within each domain of the exam drafting items anchored directly to their specifications. Next, during item writing workshops, items prepared for new exam forms are reviewed and revised by the entire domain’s team to ensure items:

• Reflect entry-level professional practice
• Present accurate content
• Lack bias in content and wording
• Measure the intended knowledge domain (psychometric adequacy).

This collaborative refinement helps improve both the quality and fairness of each item before it moves forward twice per year when new exam forms are developed, and once every two years during an item bank maintenance meeting.

Step 3: Independent Expert Review

Before a newly written item can appear on an examination form, it undergoes an additional independent review by SMEs.

These reviewers evaluate each item for:

• Relevance to entry-level practice
• Accuracy of psychological content
• Clarity of wording
• Absence of cultural or linguistic bias.

Items may be revised further before being approved for use as pretest items.

Step 4: Pretesting on Live Examinations

The approved draft items are initially included on operational examination forms as pretest items. Although candidates answer these questions during the examination, pretest items are not scored and therefore do not affect a candidate’s examination result. Instead, they allow ASPPB to collect real-world statistical data on how the items perform with actual examination candidates.

Step 5: Psychometric Evaluation

After an examination form has been in use for approximately two years, it is retired and subjected to comprehensive statistical analysis.

Using Item Response Theory (IRT), psychometricians evaluate how effectively each item measures candidate knowledge. At the same time, Differential Item Functioning (DIF) analyses are conducted to identify whether any item functions differently across candidate groups.

Items identified through fairness analyses receive additional review by SMEs to determine whether revisions are necessary or whether the item is removed from future use.

Only pretest items that demonstrate strong psychometric performance and meet fairness standards are promoted to the operational, or calibrated, item bank.

Step 6: Selection for Future Examination Forms

Twice each year, new EPPP forms are assembled. Operational items are selected according to the examination blueprint and meet psychometric requirements established through IRT analyses. Before inclusion on a new examination form, selected items undergo another review by SMEs to confirm they continue to:

• Reflect current entry-level practice
• Accurately measure the intended knowledge
• Meet fairness expectations
• Maintain appropriate statistical performance.

In addition, the operational item bank undergoes periodic maintenance reviews by SMEs approximately every two years to ensure the examination continues to reflect contemporary psychological practice.

An Ongoing Validation Process

The development of an EPPP item does not end once it appears on an examination. Instead, every item is part of a continuous quality assurance cycle that includes expert review, field testing, psychometric evaluation, fairness analyses, and periodic re-evaluation.

At the examination level, the blueprint and passing standard are periodically revalidated through job task analyses and standard setting studies. At the item level, newly developed questions move through multiple stages of expert review and statistical evaluation before becoming scored operational items.

This ongoing development and validation process helps ensure that the EPPP remains aligned with current professional practice while maintaining high standards of validity, reliability, fairness, and defensibility in assessing candidates seeking licensure as psychologists.

Want to learn more about the EPPP? Visit EPPP News.