Computer Science Colloquium

Prof. Dr. Walter Binder
University of Lugano

Workload Discovery and Benchmark Synthesis from Public Code Repositories

Thu 10.11.2016, 14:00, 60 minutes
JKU, Computer Science Building, room 218

Abstract

Researchers often rely on benchmarks to demonstrate feasibility or efficiency of their contributions. However, finding the right benchmark suite can be a daunting task - existing benchmark suites may be outdated, known to be flawed, or simply irrelevant for the proposed approach. Creating a proper benchmark suite is challenging, extremely time consuming, and also - unless it becomes widely popular - a thankless endeavor. This talk introduces AutoBench, a novel approach to help researchers find relevant workloads for their experimental evaluation needs. AutoBench relies on the huge number of open-source projects available in public repositories, and on unit testing having become best practice in software development. Using a repository crawler employing pluggable static and dynamic analyses for filtering and workload characterization, AutoBench allows users to automatically find projects with relevant workloads. In this talk, we illustrate AutoBench's approach to find, filter, and characterize real-world workloads from public open-source repositories, and show several motivating scenarios. Preliminary results towards automatic generation of benchmark suites are also presented, arguing that unit tests can provide a viable source of workloads, and that the combination of static and dynamic analysis improves the ability to identify relevant workloads that can serve as the basis for custom benchmark suites.

Bio

Walter Binder is an associate professor in the Faculty of Informatics, Università della Svizzera italiana (USI), Switzerland. He holds an MSc, a PhD, and a Venia Docendi from TU Wien. His main research interests are in the areas of program analysis, virtual machines, parallel programming, and cloud computing.

Invited by Prof. Dr. Hanspeter Mössenböck, Institut für Systemsoftware

---

The Computer Science Colloquium is organized by the Department of Coputer Science at JKU, the Österreichische Gesellschaft für Informatik (ÖGI) and the Österreichische Computergesellschaft (OCG).

List of all talks