Fri 25 Oct 2024 16:00 - 16:20 at IBR East - Testing Everything, Everywhere, All At Once Chair(s): Alex Potanin

Rust is an emerging systems programming language that aims to reduce the trade-off between safety and performance with a type system that constrains pointer operations, preventing bugs such as use-after-free. However, these constraints may be too strict for programming tasks such as implementing cyclic data structures, requiring the programmer to temporarily suspend the checks using the unsafe keyword. Rust libraries wrap unsafe code blocks and expose higher-level APIs; these APIs need to be extensively tested to prevent memory-safety bugs from being induced by unexpected API call sequences or inputs. While prior works have attempted to test Rust APIs, significant challenges remain: they fail to support common Rust features, such as polymorphism, traits, and higher-order functions, or require the analyst to pick a smaller subset of APIs.

We propose Crabtree, a testing tool for Rust APIs that employs automatic library test synthesis that natively supports Rust traits. Our tool improves upon the test synthesis algorithms of prior works by combining synthesis and fuzzing through a coverage-guided search algorithm that intelligently grows test programs and input corpus towards higher coverage. To the best of our knowledge, our tool is the first to generate well-typed tests for libraries that make use of higher-order trait functions. Evaluation of Crabtree on 30 libraries found four memory-safety bugs, all of which were accepted by the respective authors.

Fri 25 Oct

Displayed time zone: Pacific Time (US & Canada) change

16:00 - 17:40
Testing Everything, Everywhere, All At OnceOOPSLA 2024 at IBR East
Chair(s): Alex Potanin Australian National University
16:00
20m
Talk
Crabtree: Rust API Test Synthesis Guided by Coverage and Type
OOPSLA 2024
Yoshiki Takashima Carnegie Mellon University, Chanhee Cho Carnegie Mellon University, Ruben Martins Carnegie Mellon University, Limin Jia , Corina S. Păsăreanu Carnegie Mellon University; NASA Ames
DOI
16:20
20m
Talk
Drowzee: Metamorphic Testing for Fact-conflicting Hallucination Detection in Large Language Models
OOPSLA 2024
Ningke Li Huazhong University of Science and Technology, Yuekang Li UNSW, Yi Liu Nanyang Technological University, Ling Shi Nanyang Technological University, Kailong Wang Huazhong University of Science and Technology, Haoyu Wang Huazhong University of Science and Technology
DOI
16:40
20m
Talk
Reward Augmentation in Reinforcement Learning for Testing Distributed Systems
OOPSLA 2024
Andrea Borgarelli Max Planck Institute for Software Systems, Constantin Enea LIX, CNRS, Ecole Polytechnique, Rupak Majumdar MPI-SWS, Srinidhi Nagendra CNRS, Université Paris Cité, IRIF, Chennai Mathematical Institute
DOI
17:00
20m
Talk
Rustlantis: Randomized Differential Testing of the Rust Compiler
OOPSLA 2024
Qian (Andy) Wang ETH Zurich and Imperial College London, Ralf Jung ETH Zurich
DOI
17:20
20m
Talk
Statistical Testing of Quantum Programs via Fixed-Point Amplitude Amplification
OOPSLA 2024
Chan Gu Kang Korea University, Joonghoon Lee Korea University, Hakjoo Oh Korea University
DOI