Peer Review Management - An Interview with Craig Raybould, COO, Hindawi

Posted on : August 14th 2021

Craig is Chief Operations Officer at Hindawi, managing teams responsible for the full article lifecycle. We spoke with him about Hindawi’s peer review processes and people.

Straive: How have you structured your JEO – do you have in-house team members working on peer review or is it entirely outsourced?

CR: Hindawi operate a mixed structure. The outsourced JEO is separated into three groups, with the team members supplied by Straive: an editorial screening team who check submissions, editorial assistants who ensure the peer review process goes smoothly, and a quality assurance team. All three teams have rules about when they need to escalate a situation to our internal staff - that might be to the editorial team if the question is content related, operations for a process question, or the research integrity team for ethical issues. We also do regular audits to maintain oversight of the whole workflow and that helps us to catch issues as early as possible, before they become real problems.

Straive: With all the effort that goes into making sure everyone has the best training and documentation, what do you see as the main benefit of outsourced JEO?

CR: The big issue for us is the simple question of scale. We anticipate having around 300 outsourced JEO staff by the end of 2021, which is just impractical for us to have in-house. It's not just the question of cost, though obviously outsourcing is less costly than on-shore hires, but also the HR practicalities of recruitment, training, retention, etc. The other benefit of outsourcing is that we have been able to spread our teams across multiple locations worldwide, which is great risk mitigation as well as allowing us to provide researchers with service pretty much around the clock.

Straive: You mention risk mitigation. What are the biggest challenges Hindawi is facing with peer review?

CR: Much like other publishers, avoiding backlogs by making sure that peer review is quick and efficient is always a challenge. Particularly where we rely on volunteer editors and reviewers, there's often a need to extend deadlines or manage nonresponses to messaging, both of which can create delays. Extending our pool of peer reviewers is a key part of efficiency, and we've chosen to work with Publons to help us do that.

Straive: Do you think there is a bigger role for technology in peer review, and is Hindawi planning to invest in it?

CR: Yes, we are heavily investing in the Phenom publishing tool suite, which includes peer review. We have around 40 staff members in a development team working on it, and our goal is to automate as much of the peer review process as possible to reduce editor and reviewer workload. For example, we have automated the technical checks that happen at submission. There are hundreds of features on our roadmap and wishlist, so we are also assessing third party tools that have the potential to be integrated with Phenom. The major challenge with some of the tools is not developing or integrating them, but training editors to interpret the reports the tools produce and make full use of their capabilities.

Straive: The pressure to grow market share and the desire among some journals to attract media attention for articles making big claims has led to some research integrity issues over the years. How would you like to see publishers address questions of research integrity, either during peer review or after publication?

CR: We have a dedicated research integrity team of six in-house staff, backed up with a team at Straive. The outsourced group help by undertaking the background investigation, pulling up email trails and so on, while the in-house team are responsible for decision making in integrity cases. Hindawi takes a really proactive approach to identifying research integrity issues as early as possible in the process, and ideally during the post-submission screening. One safeguard is that whenever an editor wants to accept an article for publication, the acceptance message will not be sent to the authors until after the JEO has checked the article and peer review process complies with our standards.

Many of the research integrity issues we come across are related to plagiarism. That might be identified at submission, but we do a second plagiarism check before acceptance, to pick up cases where authors have plagiarised a recently published article that wouldn't have been available for us to check at submission. It's not common, but it does happen.

A really difficult part of research integrity issues relates to the thresholds for what is deemed 'offensive'. Different editors have different ways of defining offense and there are limited mechanisms for picking this up automatically.

Straive: Do you think editors or JEO should edit reviewer reports when they contain offensive language?

CR: We do ask the JEO and our editors to edit reports if they find offensive language (e.g. swearing). In those cases we will simply remove the text, and escalate the changes to the in-house editorial team to review before the report is shared with the author.

Straive: Finally, as an OA publisher what is your take on the varieties of open peer review?

CR: Hindawi are an Open Access publishing house so we're all about transparency, but our peer review is single anonymised at the moment, and we don't publish reports or reviewer names. We are investigating non-anonymised review and other aspects of open review, but adoption will be very dependent on what researchers in different disciplines want us to do. Having said that, we do address a lot of other aspects of openness, like open data, funding statements, etc. The focus has been on opening up content, so peer review is a next step.

Download our whitepaper going deep into the opportunities and challenges with Peer Review Management