Static analysis tools - benchmarking

A Comparison of Static Analysis Tools for Vulnerability Detection in C/C++ Code
- SYNASC 2017
- Andrei Arusoaie, Ștefan Ciobâcă, Vlad Crăciun, Dragoș Gavriluț, Dorel Lucanu
- https://profs.info.uaic.ro/~stefan.ciobaca/synasc2017.pdf
Analyzing the State of Static Analysis: A Large-Scale Evaluation in Open Source Software
- Software Analysis, Evolution, and Reengineering (SANER) 2016
- http://ieeexplore.ieee.org/document/7476667/
- https://www.slideshare.net/inventitech/analyzing-the-state-of-static-analysis-a-largescale-evaluation-in-open-source-software-60473247
- 2015 Master’s Thesis: https://repository.tudelft.nl/islandora/object/uuid:3d834130-8dd7-420a-9af9-6e77761cdad6/
Automated Customized Bug-Benchmark Generation
- IEEE SCAM 2019
- Vineeth Kashyap, Jason Ruchti, Lucja Kot, Emma Turetsky, Rebecca Swords, David Melski, Eric Schulte
- https://arxiv.org/abs/1901.02819
- https://www.dropbox.com/sh/f3p1wbomv93fz78/AABzXcLzqp5zkx75nQym6gJPa
Benchmark Software for Evaluation of Software Analysis Tools
- https://github.com/Toyota-ITC-SSD/Software-Analysis-Benchmark
- Static analysis benchmarks from Toyota ITC - https://github.com/regehr/itc-benchmarks
How Toyota Picks Software Tools
- EETimes, August 11, 2015; Bernard Cole
- https://www.eetimes.com/author.asp?section_id=36&doc_id=1327387
- http://web.archive.org/web/20150815120014/http://www.eetimes.com/author.asp?section_id=36&doc_id=1327387
Quantitative Evaluation of Static Analysis Tools
- Software Reliability Engineering Workshops (ISSREW) 2014
- Shinichi Shiraishi, Veena Mohan, Hemalatha Marimuthu
- https://dl.acm.org/citation.cfm?id=2706360
Software Assurance Reference Dataset (SARD)
- https://samate.nist.gov/SARD/
- Test Suites - https://samate.nist.gov/SRD/testsuite.php
Static Analysis Benchmarks
- https://blog.regehr.org/archives/1217
Test Suites for Benchmarks of Static Analysis Tools
- IEEE International Symposium on Software Reliability Engineering (ISSRE) 2015
- Shinichi Shiraishi, Veena Mohan, and Hemalatha Marimuthu
- https://www.researchgate.net/publication/283548090_Test_Suites_for_Benchmarks_of_Static_Analysis_Tools
Towards Automatically Generating a Sound and Complete Dataset for Evaluating Static Analysis Tools
- NDSS Workshop on Binary Analysis Research (BAR) 2019
- Aravind Machiry, Nilo Redini, Eric Gustafson, Hojjat Aghakhani, Christopher Kruegel, Giovanni Vigna
- https://ruoyuwang.me/bar2019/pdfs/bar2019-final90.pdf
- https://github.com/ucsb-seclab/autofacts
Toyota ITC Benchmark
- https://runtimeverification.com/match/1.0-SNAPSHOT/docs/benchmark/
Using Benchmarks to Assess Static Analysis Tools
- http://blogs.grammatech.com/using-benchmarks-to-assess-static-analysis-tools
Differentially testing soundness and precision of program analyzers
- International Symposium on Software Testing and Analysis 2019
- Christian Klinger, Maria Christakis, Valentin Wüstholz
- https://arxiv.org/abs/1812.05033
Quickchecking Static Analysis Properties
- Testing, Verification and Reliability 27(6) 2017
- Jan Midtgaard, Anders Møller
- http://janmidtgaard.dk/papers/Midtgaard-Moeller%3aSTVR17.pdf
Test suites for benchmarks of static analysis tools
- Software Reliability Engineering Workshops (ISSREW) 2015
- Shinichi Shiraishi, Veena Mohan, Hemalatha Marimuthu
- http://ieeexplore.ieee.org/document/7392027/
Testing Static Analyses for Precision and Soundness
- CGO 2020
- Jubi Taneja, Zhengyang Liu, and John Regehr
- https://doi.org/10.1145/3368826.3377927
Testing Static Analyzers with Randomly Generated Programs
- NFM 2012: NASA Formal Methods
- Pascal Cuoq, Benjamin Monate, Anne Pacalet, Virgile Prevosto, John Regehr, Boris Yakobowski, Xuejun Yang
- http://blog.frama-c.com/public/csmith.pdf
Towards Scalable Translation Validation of Static Analyzers
- ROSAEC Technical Report ROSAEC-2014-003, Nov 2014
- Jeehoon Kang, Sungkeun Cho, Joonwon Choi, Chung-Kil Hur, Kwangkeun Yi
- http://sf.snu.ac.kr/gil.hur/publications/ROSAEC_2014_003.pdf
Systematic Approaches for Increasing Soundness and Precision of Static Analyzers
- State Of the Art in Program Analysis (SOAP) 2017
- Esben Sparre Andreasen, Anders Møller, Benjamin Barslev Nielsen
- https://pldi17.sigplan.org/event/soap-2017-papers-systematic-approaches-for-increasing-soundness-and-precision-of-static-analyzers
- https://cs.au.dk/~amoeller/papers/tajsexperience/paper.pdf
Verasco, a formally verified C static analyzer
- A Formally-Verified C Static Analyzer
  - POPL 2015
  - Jacques-Henri Jourdan, Vincent Laporte, Sandrine Blazy, Xavier Leroy, David Pichardie
  - https://hal.inria.fr/hal-01078386
  - https://xavierleroy.org/publi/verasco-popl2015.pdf
- Microsoft Research 2016; Jacques Henri Jourdan
  - "the design and soundness proof of Verasco, a formally verified static analyzer for most of the ISO C99 language (excluding recursion and dynamic allocation), developed using the Coq proof assistant."
  - https://www.youtube.com/watch?v=0pUueg3Dslo
- Checking a Checker - Verasco: a Formally Verified C Static Analyzer
  - 2016 PhD Dissertation; Jacques-Henri Jourdan
  - https://jhjourdan.mketjh.fr/thesis_jhjourdan.pdf
Verified Translation Validation of Static Analyses
- Computer Security Foundations Symposium 2017
- Gilles Barthe, Sandrine Blazy, Vincent Laporte, David Pichardie, Alix Trieu
- https://hal.inria.fr/hal-01588422/

Tags: reading

Last modified 23 August 2025

Static analysis tools - benchmarking

A collection of readings around benchmarking and deciding between static analysis tools.