Taking advantage of earlier work by a member of the research team (Garand), the EPPR investigators began with the population of journals used in earlier research. Because this list was now dated, we supplemented it with a second set of journals collected by one of the members of the research team. This second list was specifically designed to extend the original list and add balance to the slight positivist and Americanist tint of the original corpus of journals.
After combining the lists together, venues that did not employ peer-review were culled. Each member of the research team separately ranked the remaining venues, balancing the presence and nature of journal impact factors, subfield coverage of the journal, and whether a journal was related to a major scholarly association. Each journal received at least two scores involving subjective familiarity and appeal or prestige. Using these scores as a guide, each member of the research team then made an independent evaluation regarding whether a journal should be included in the survey corpus.
Publication venue inclusion decisions were then compared across the research team. Journals that appeared on at least two of three members’ lists were included in the final list of venues. For journals with a single vote of support, the research team adjudicated inclusion on a case-by-case basis, taking into account the factors above until a decision about inclusion was reached.
This process terminated in the creation of a “core” and “supplementary” list of journals. Journals included in the “core” list of venues are rated by all EPPR survey respondents (~50 journals). From the remaining list of roughly 100 journals, participants are further randomly assigned 50 venues to evaluate. This approach was necessary to balance journal breadth with respondent fatigue.
For scholarly presses, the research team developed a similar inclusion approach, albeit on a smaller scale. Drawing from a population of presses surveyed in earlier work, the research team compiled a list of 15 publishers and scholarly presses that all respondents rank at the conclusion of the survey.
As EPPR is meant to be a regularly updated project, the survey instrument includes questions in which respondents can nominate and rank journals not included in their questionnaire. (Note that, because of randomization, a journal so nominated may have already been included but not presented to the respondent.) Future iterations of the journal and publisher rankings will take these responses into account.
Evaluation of Publication in Political Research 