How to: Peer Review for CHI (and Beyond)

Overview

A key challenge for people that are new to reviewing is pitching the review at the right level, and getting the tone and structure of a review right. This course aims to help participants understand a) the different expectations of different venues and submission types, b) the processes they use to make decisions, and c) good techniques for producing a review for these different circumstances. Combined with developing a good understanding of these different expectations, participants have a chance to critique anonymised proto-reviews, and try to guess the venue they are written for and the recommendation they make.

Learning Outcomes

Understand the different types of submissions that can be reviewed
Understand the different processes used by different venues (for different types of submissions)
Reflect on what senior reviewers want from a review (and therefore how they might be structured)
Critique example reviews for different types of sub- missions for different types of venues

Intended Audience: no prior experience

This course is aimed at people with little or no prior experience in reviewing. This includes, but is not limited to, new research students.

Structure

image shows the two halves of the course as described below.

Part 1 focuses on expanding the understanding of different types of publication venue and what their expectations are. This includes everything from community workshops that want stimulating content, to full peer-reviewed venues like journals that want a focus on detail and rigour. Further, we discuss the different people in the process and what their tasks and expectations are. This part is largely taught as interactive group discussions.

Part 2 begins by focusing more specifically on the process of actually reviewing an article and ways to structure reviews. This half includes the majority of the 'taught' content, which makes up the first hour of the afternoon. The rest of the afternoon is focused on examining reviews, critiquing them for how well they support the review process and the people reading them.

Practical Work and Resources

This course is primarily interactive, involving a range of exercises in small groups using pen and paper materials. The workshop includes discussions of example anonymous reviews, which participants can keep for subsequent reference.

Instructor Bio

Dr. Max L. Wilson Dr Max L. Wilson, as Associate Professor at the University of Nottingham. Max, who sits on the CHI Steering Committee, has been a reviewer for CHI for ~15 years, and has reviewed for many other conferences including CSCW, UIST, SIGIR, CHIIR (and its former IIiX), ISWC, WWW, UbiComp and MobileHCI. Max has also reviewed for journals including: JASIST, JWS, IJHCI, IP&M, TOIS, TOCHI. Max has served as a senior reviewer (AC) for CHI and CSCW since 2014, and nas a Subcommittee Chair (SC) for the Understanding People subcommittee for 3 years. Max has been an Associate Editor for IJHCS and IP&M, and now serves as the Deputy Editor for IJHCS. Max has also acted as Papers Chair for IIiX2014, posters chair for IIiX2012, Courses Chair for CHI2016 and CHI2017, Panels Chair for CHI2018, and on the Best Paper Committee for CHIIR2018. Max has delivered this particular course on five prior occasions, at national PhD student events, and at specific universities in the United Kingdom.