Workshop Proposal: Recruitment of Participants for Empirical SE Studies (RoPES 22)
Abstract
All else being equal, scientific studies generate stronger conclusions when more participants are involved. For positivist, frequentist approaches, the ability to detect an effect is directly related to the number of subjects tested*. However, finding relevant human subjects for studies is challenging. Our field is composed of many highly skilled and highly compensated professionals. Many studies focus on very specific topics with limited populations. Reliance on students may not be appropriate.
- there are many many other possible problems beyond absolute magnitude of N, of course!
Organizer Contact Information
- Neil Ernst
- Jeff Carver
- … tbd
Interested Parties
- Matthew Smith, Bonn
- Jeff Carver, Alabama
- Valentina Lenarduzzi, TUE
- Bonita Sharif
- Fabiano Dalpiaz
- Nicole Novielli
- Alexander Serebrenik
- Andy Zaidman
- ALberto Bacchelli
- Davide Fucci
- Kelly Blincoe
- Tamara Lopez, OU
- Fabian Gilson, UC NZ
- Igor Wiese
- Kevin Moran
- Melina Vidoni
- Richard Paige
- David Shepherd
- Paul Ralph
- [Daniel Méndez]
- Janet Siegmund
Curious
- Fabiano Dalpiaz
- Bonita Sharif
Motivation and Objectives
- Software studies benefit when more high quality participants are recruited. However, participant recruitment is challenging. There are new tools (Mechanical Turk, Prolific)
- Anticipated goals and outcomes (e.g., open research problems to pursue, validation objectives, empirical studies).
- Topics include: strategies for new platforms, specifically filtering; students vs practitioners; determining adequate sample sizes; ethical considerations of participant involvement (GDPR, use of emails); budgeting for recruitment costs.
Format and Required Services
- The workshop is a working session dedicated to sharing strategies and challenges. Participants will prepare a 2 page position paper and expderience summary. Organizers will create themes around those position papers. We hope to solicit a keynote from a researcher working in a field with greater experience (such as psychology or sociology). The outcome of the workshop will be a SIGSOFT participant recruitment working paper.
- Intended length (1 or 2 days).
- no special requirements
Target Audience
- Industry and academic researchers doing participant recruitment for software studies (e.g., of developers, of
- Plans regarding the mix of industry and research participation.
- Expected minimum and maximum number of workshop participants.
- Plans for soliciting and selecting workshop participants, including whether the workshop will be open or closed (in case of a closed workshop, provide a justification).
Proceedings
- We intend to focus on short non-proceedings position papers to motivate open discussion at this point. No formal proceedings.
- Our program committee will check the submissions for alignment with the workshop topics but formal review will not be conducted.
- Online submission system to be used by the workshop (e.g., Easychair or HotCRP). Link to the preliminary website of the workshop. A proceeding chair, who will be responsible to send the list of accepted papers (paper titles, authors, emails) to the publishers.
Program Committee
(indicated interest on Twitter)
Workshop History
No previous history
Organizers’ bios
- A brief description of each organizer’s background, including relevant past experience in organizing conferences and workshops.