by Shiping Chen
Summer has brought not only sunshine but also the excitement of the annual CHIWORK conference! I’m thrilled to be a paper presenter at CHIWORK this year. CHIWORK 2025 is being hosted in Amsterdam, The Netherlands. Although I wasn’t able to attend in person due to other commitments, I still felt the vibrant academic energy of CHIWORK as an online presenter.
My presentation was scheduled on the morning of 24 June, in the “AI IN HIGH-STAKES AND INSTITUTIONAL WORK” session. As a hybrid conference, CHIWORK provided a seamless and well-coordinated experience, making it easy for both in-person and online participants to engage with the programme. I really appreciated how smooth the transition was between talks, with Zoom rooms and Slack channels well-organised for announcements and Q&A.
My paper, Envisioning the Future of Peer Review: Investigating LLM-Assisted Reviewing Using ChatGPT as a Case Study, co-authored with my supervisors Prof. Duncan Brumby and Prof. Anna Cox, explores how Large Language Models like ChatGPT can support human reviewers during the peer review process. Rather than treating AI as a standalone reviewer, our study takes a human–AI collaboration perspective to understand how reviewers actually use ChatGPT in practice. We conducted a within-subject experiment with 24 HCI reviewers, each completing two review tasks—one with ChatGPT assistance and one without—followed by post-task questionnaires and interviews. Our findings show that ChatGPT can significantly reduce the perceived workload in drafting reviews and making judgments, and reviewers appreciated its support for summarisation, information retrieval, idea generation, and confidence-building. However, it did not lead to substantial time savings or improvements in review quality. These insights contribute to ongoing discussions around responsible AI integration in scholarly publishing and offer design implications for future peer review tools that complement human expertise.
The Q&A after my presentation gave me the opportunity to respond to insightful questions from the audience that touched on the core motivations and assumptions behind our study. It was a valuable chance to elaborate on the reasoning behind our experimental design, particularly why we chose to focus on ChatGPT as a case study. I was encouraged by the genuine interest in the practical implications of LLMs for real-world peer review, and the discussion helped me further reflect on how such tools might be responsibly integrated into academic workflows.
Although I joined the conference remotely, I still came away with a great deal of inspiration, thoughtful questions, generous feedback, and new ideas to take forward. I’m grateful to have been part of such a welcoming and intellectually engaging community, and I look forward to staying connected with CHIWORK in the years to come.
