Call for Artifact Evaluations
A scientific paper consists of a constellation of artifacts that extend beyond the document itself, possibly including software, hardware, evaluation data, documentation, survey results, mechanized proofs, models, test suites, and benchmarks. In some cases, the quality of these artifacts is as important as the document itself.
MLSys’26 invites authors of accepted papers to participate in the Artifact Evaluation (AE) process. The AE process will assess the availability and functionality of artifacts associated with each paper, as well as the reproducibility of the paper's key results and claims using those artifacts. Artifact evaluation is single-blind. Artifacts will be held in confidence by the evaluation committee. We encourage all authors to publish their artifacts to benefit the broader community.
Participation in the artifact evaluation is optional, but we hope all accepted papers will participate. The decision of artifact evaluation does not affect the paper acceptance decision. However, we encourage authors to make the best efforts to obtain the “Results Reproduced” badge.
You need to prepare your artifacts and appendix using the guidelines below. Please write a brief abstract describing your artifact, the minimal hardware and software requirements, how it supports your paper, how it can be validated and what the expected result is. This is essential to find appropriate evaluators! You can then submit your 1) artifact abstract and 2) the PDF of your paper with 3) the Artifact Appendix attached via a dedicated MLSys AE website before March 8, 2026. Be sure to include a link to the GitHub (or similar public service) page in the Appendix so reviewers can access your code. Your submission will then be reviewed according to the guidelines in the Artifact Evaluation process.
AE is run by a separate committee that assesses how submitted artifacts support the work described in accepted papers, in accordance with the standard ACM Artifact Review and Badging policy. The reviewers will assign scores to artifacts to decide whether the requested badges (Availability, Functional, and Reproducible) may be awarded. In addition, MLSys’26 plans to confer a small number of Distinguished Artifact Awards to recognize exceptional artifacts that set a high standard for reproducibility and provide strong value to the community. If you have successfully passed AE with at least one of the three AE badges, you will need to add the badges and your artifact appendix to your camera-ready paper while removing all unnecessary or confidential information. This will help readers better understand what was evaluated and how.
We are looking for members of the Artifact Evaluation Committee (AEC), who will contribute to MLSys’26 Artifact Evaluation (AE) process by reviewing submitted artifacts. AEC membership is especially suitable for researchers early in their career, such as PhD students. Even as a first-year PhD student, you are welcome to join the AEC, provided you are working in a topic area covered by MLSys. For a given artifact, you will be asked to evaluate its public availability, functionality, and reproducibility following the instructions in the artifact appendix. You will be able to discuss with other AEC members and anonymously interact with the authors as necessary, for instance if you are unable to get the artifact to work as expected. Finally, you will provide a review for the artifact to give constructive feedback to its authors, discuss the artifact with fellow reviewers, and help award the paper artifact evaluation badges. Please ensure that you have sufficient time and availability for the AEC during the period from March 8 to April 8, 2026. If you are interested in becoming a part of the AEC, please complete the online self-nomination form before Feb 28, 2026.
Dates
- Submissions due Mar 8, 2026 20:00 UTC
- AE result notification: Apr 16, 2026
Links