Review guidelines
Thank you for agreeing to review for ICMR 2018. We appreciate your service: your time and effort directly contributes to maintaining the high quality of the conference and strengthening the multimedia information retrieval community.
As a TPC member, we expect that you are already experienced with writing excellent reviews. However, in practice we find that guidelines can help streamline the process. As in the past, ICMR 2018 will be announcing awards for the best reviewers of the conference. The guidelines also serves as a basis for the best reviewer decisions.
The Golden Rule of reviewing: Write a review that you would like to receive yourself.
A review should be helpful to the authors, even if the review recommends rejection of the paper.
Make sure that you would be happy to put your name on the review, even if it weren't anonymous.
Best practices for reviewing:
Check that the paper falls into the topical scope of ICMR, as defined by the Calls for Papers:
- http://www.icmr2018.org/cfp_paper.html
Support your statements:
- Reviews should not just state, "It is well known that...", but rather, they should include citations;
- Reviews should not just state, "Important references are missing…", but rather, they should include examples;
- Authors appreciate if reviewers are generous with their feedback.
Respect the authors:
- Reviews should critique "the paper", and not the authors;
- Reviews should not address the authors directly, esp. not as "you". (A direct address can be interpreted as an affront by the reader);
- During the review process, no attempt should be made to guess the identity of the authors.
Please include in your review:
- Statement of novelty: What does the paper contribute? Has the related work been covered?
- Statement of scientific rigor: Are the experiments well designed? Are they reproducible?
- Fixes that the authors should make for the camera ready. We can trust the authors to correct minor errors. Major flaws must lead to rejection, since it is not possible to confirm that the authors have actually corrected them successfully (i.e., the paper does not go back to the reviewers).
Policy on arXiv papers
We consider a "publication" to be a manuscript that has undergone peer review and has been accepted for publication. This means that the following points apply to arXiv papers (and any other papers available online that have not been peer reviewed):
- If the paper that you are reviewing is available on arXiv, and has not been published elsewhere, it is an acceptable submission to ICMR, since arXiv papers are not peer reviewed and are not publications;
- Please do not insist that the authors cite a paper that is only on arXiv and has not otherwise been published. Since arXiv papers are not publications, missing an arXiv paper does *not* count as missing related work;
- Likewise, if the authors do not compare their work with an approach described in an arXiv paper, it does *not* count as a weakness in their experimental evaluation of their own approach;
- If you know of an interesting arXiv paper relevant to the paper you are reviewing, you are more than welcome to tell the authors about it, but make sure you mark the reference as FYI "for your information" so that the authors know that you do not regard it as missing related work.
If you have any questions about the guidelines, please contact the Technical Program Chairs at program-chairs@icmr2018.org.
