Artifact

Deadline: Thursday 16 Dec 2021 23:59:59.

Submit (a zip archive) via command line:

curl http://jyywiki.cn/upload \
  -F course=ISER2021 \
  -F module=A4 \
  -F token={{your token}} \
  -F stuid={{student id}} \
  -F stuname={{name (chinese)}} \
  -F file=@{{path to your submission}}

You need to submit only one artifact of either PA1 (codesim) or PA2 (jmtrace).

Open Science Policies

The open science policies are encouraged:

The research track of ICSE 2022 is governed by the ICSE 2022 Open Science policies. In summary, the steering principle is that all research results should be accessible to the public and, if possible, empirical studies should be reproducible. In particular, we actively support the adoption of open data and open source principles and encourage all contributing authors to disclose (anonymized and curated) data to increase reproducibility and replicability. Note that sharing research data is not mandatory for submission or acceptance. However, sharing is expected to be the default, and non-sharing needs to be justified. We recognize that reproducibility or replicability is not a goal in qualitative research and that, similar to industrial studies, qualitative studies often face challenges in sharing research data. For guidelines on how to report qualitative research to ensure the assessment of the reliability and credibility of research results, see the Q&A page.

Upon submission to the research track, authors are asked

• to make their data available to the program committee (via upload of supplemental material or a link to an anonymous repository) – and provide instructions on how to access this data in the paper; or
• to include in the paper an explanation as to why this is not possible or desirable; and
• to indicate if they intend to make their data publicly available upon acceptance.

Supplementary material can be uploaded via the HotCRP site or anonymously linked from the paper submission. Although PC members are not required to look at this material, we strongly encourage authors to use supplementary material to provide access to anonymized data, whenever possible. Authors are asked to carefully review any supplementary material to ensure it conforms to the double-anonymous policy (described above). For example, code and data repositories may be exported to remove version control history, scrubbed of names in comments and metadata, and anonymously uploaded to a sharing site to support review. One resource that may be helpful in accomplishing this task is this blog post.

Artifact Evaluation

Accepted top-tier conference papers are encouraged to submit supplementary material for artifact evaluation. A committee will download and evaluate your artifact to see whether the artifact meets a high standard of usability. Below is the badge for “reusable” level artifacts, indicating that there is sufficient documentation for other researchers to reuse the artifact:

The criteria for reusable badge (defined by ACM) is:

The artifacts associated with the paper are of a quality that significantly exceeds minimal functionality. That is, they have all the qualities of the Artifacts Evaluated – Functional level, but, in addition, they are very carefully documented and well-structured to the extent that reuse and repurposing is facilitated. In particular, norms and standards of the research community for artifacts of this type are strictly adhered to.

In this lab, you should pack your programming assignment (either one of PA1 or PA2) into a zip-format artifact. Keep your artifact private: don't publicize them, e.g., on Github. Your artifact should contain at least:

• Your programming assignment source code
• A README file for usage, dependencies, and build instructions
• A .gitignore file

You should submit a minimal artifact of only source code without dependencies or binaries. Since our programming assignments have clear specifications, you do not need to prepare detailed documentation for your project. Focus on high-quality and easy-to-build code.

Your project may have specific dependencies, e.g., LLVM toolchain or a Python package. Please try your best to depend on widely accessible toolchain (e.g., the default version in Ubuntu LTS). It's not a good idea that your project only compiles on Ubuntu 12.04. We also strongly recommend standard build systems like CMake or Maven.

Your classmates will be your artifact users and will compile and evaluate them. Do test your artifact before submitting them.