Role of code comments in acceptance or rejection of Pull Requests

Seminar topic, University of Zurich, 2023

Context:

Software documentation is an inexplicable part of the software. It helps developers understand and modify the software in the development and maintenance phases. Code comments are an important type of software documentation, and developers trust code comments more than other forms of documentation. A Pull Request (PR) is one of the key ideas to support collaboration. Developers are encouraged to submit PRs to contribute. Not all PRs are merged into the main codebase. Researchers found various factors for this behavior, including code comments.

Problem:

However, what role code comments or software documentation play in the acceptance or rejection of PR is unexplored.

Steps:

  • Select projects with over 200 closed PRs.
  • Reject projects that have less than 20 closed issues in Github.
  • Identify insiders and outsiders in a project. Keep projects that have both.
  • Check if the project has a README file.
  • Formulate comment criteria based on comment convention work.
  • Check the criteria before and after merging of the PR.
  • For each PR, extract the files modified by the PR.
  • Prepare two versions (original and modified).
  • Measure the commenting style of both versions and compare them.
  • Identify which particular criteria impact the most.