See also: compilers and language implementation

General

What is a secure programming language?

ACM SIGPLAN Conference on Certified Programs and Proofs (CPP) - http://dblp.org/db/conf/cpp/
Black-Box Equivalence Checking Across Compiler Transformations
- 2018 PhD thesis; Manjeet Dahiya
- https://www.cse.iitd.ac.in/~sbansal/pubs/manjeet_thesis.pdf
Calculating Correct Compilers
- Journal of Functional Programming, Volume 25, September 2015
- Patrick Bahr and Graham Hutton
- http://www.cs.nott.ac.uk/~pszgmh/bib.html#ccc
Calculating Correct Compilers II: Return of the Register Machines
- September 2019
- Patrick Bahr and Graham Hutton
- http://www.cs.nott.ac.uk/~pszgmh/bib.html#ccc2
Compiling with Proofs
- 1998 Ph.D. Thesis; George C. Necula
- https://people.eecs.berkeley.edu/~necula/Papers/thesis.pdf
Compilers and Termination Revisited
- https://blog.regehr.org/archives/161
Formal Approaches to Secure Compilation: A Survey of Fully Abstract Compilation and Related Work
- ACM Computing Surveys (CSUR) 51(6) 2019
- Marco Patrignani, Amal Ahmed, Dave Clarke
- https://doi.org/10.1145/3280984
- http://theory.stanford.edu/~mp/mp/Publications_files/a125-patrignani.pdf
- http://theory.stanford.edu/~mp/mp/Publications_files/main-full.pdf
How to prove a compiler correct - Daniel Patterson
- https://dbp.io/essays/2018-01-16-how-to-prove-a-compiler-correct.html
- https://github.com/dbp/howtoproveacompiler
How to prove a compiler fully abstract
- https://dbp.io/essays/2018-04-19-how-to-prove-a-compiler-fully-abstract.html
Operational Refinement for Compiler Correctness
- 2012 PhD Dissertation; Robert W. Dockins
- ftp://ftp.cs.princeton.edu/reports/2012/936.pdf
Some Goals for High-impact Verified Compiler Research - https://blog.regehr.org/archives/1565
The Next 700 Compiler Correctness Theorems (Functional Pearl)
- ICFP 2019
- Daniel Patterson, Amal Ahmed
- https://icfp19.sigplan.org/details/icfp-2019-papers/35/The-Next-700-Compiler-Correctness-Theorems-A-Functional-Pearl-
- https://dbp.io/pubs/2019/ccc/
- https://www.ccs.neu.edu/home/amal/papers/next700ccc.pdf
- https://www.youtube.com/watch?v=qrwzpo6ISCQ
What even is compiler correctness? - https://www.williamjbowman.com/blog/2017/03/24/what-even-is-compiler-correctness/
Write Your Compiler by Proving It Correct - http://liamoc.net/posts/2015-08-23-verified-compiler.html

Debugging

See also: Section 6.3 (Compiler Bug Debugging) in "A Survey of Compiler Testing"

Automatic Isolation of Compiler Errors
- ACM Transactions on Programming Languages and Systems (TOPLAS) 16(5) 1994
- David Whalley
- https://www.cs.fsu.edu/~whalley/papers/acmtoplas94.pdf
Bugfind: A Tool for Debugging Optimizing Compilers
- ACM SIGPLAN Notices 25, no. 1 (1990)
- Jacqueline M. Caron, Peter A. Darnell
- https://doi.org/10.1145/74105.74106
Compiler Bug Isolation via Effective Witness Test Program Generation
- ESEC/FSE 2019
- Junjie Chen, Jiaqi Han, Peiyi Sun, Lingming Zhang, Dan Hao, Lu Zhang
- https://dl.acm.org/citation.cfm?id=3338957
- https://personal.utdallas.edu/~lxz144130/publications/fse2019.pdf
- DiWi (Diversified Witnesses)
  - https://github.com/JunjieChen/DiWi
Debugging compilers with optimization fuel
- 2011
- Edward Z. Yang
- http://blog.ezyang.com/2011/06/debugging-compilers-with-optimization-fuel/
GCC Wiki: Finding miscompilations on large testcases
- https://gcc.gnu.org/wiki/Analysing_Large_Testcases
LLVM bugpoint
- LLVM bugpoint tool: design and usage
  - https://llvm.org/docs/Bugpoint.html
- Reduce Your Testcases with Bugpoint and Custom Scripts
  - http://blog.llvm.org/2015/11/reduce-your-testcases-with-bugpoint-and.html
Locating a compiler bug with git bisection
- 2020; William Woodruff
- https://blog.yossarian.net/2020/05/07/Locating-a-compiler-bug-with-git-bisection
Replay Compilation: Improving Debuggability of a Just-in-Time Compiler
- OOPSLA 2006
- Kazunori Ogata, Tamiya Onodera, Kiyokuni Kawachiya, Hideaki Komatsu, Toshio Nakatani
- https://doi.org/10.1145/1167473.1167493
- https://www.researchgate.net/publication/221321785_Replay_compilation_Improving_debuggability_of_a_just-in-time_compiler
Toward Automatic Debugging of Compilers
- International Joint Conference on Artificial Intelligence 1977
- Hanan Samet
- http://www.cs.umd.edu/~hjs/pubs/compilers/toward-automatic-debugging.pdf
Type-Based Verification of Assembly Language for Compiler Debugging
- ACM SIGPLAN Workshop on Types in Language Design and Implementation (TLDI) 2005
- Bor-Yuh Evan Chang, Adam Chlipala, George C. Necula, Robert R. Schneck
- http://adam.chlipala.net/papers/CoolaidTLDI05/
Using Mutants to Help Developers Distinguish and Debug (Compiler) Faults
- Journal of Software Testing, Verification, and Reliability (STVR) Volume 30, Issue 2 (2020)
- Josie Holmes and Alex Groce
- https://agroce.github.io/stvr20.pdf
- https://github.com/agroce/compilermutants

History

Advice on Structuring Compilers and Proving Them Correct
- Principles of Programming Languages (POPL) 1973
- F. Lockwood Morris
- https://dl.acm.org/citation.cfm?id=512941
Compiler Verification: A Bibliography
- ACM SIGSOFT Software Engineering Notes 28(6) 2003
- Maulik A. Dave
- https://dl.acm.org/citation.cfm?id=966235
- http://www.cs.utah.edu/~skchoe/research/p2-dave.pdf
- Compiler Verification: A Brief History - http://web.archive.org/web/20090807085152/http://www.geocities.com/compiler00/dave1.html
Correctness of a Compiler for Algol-like Programs
- Stanford Artificial Intelligence Memo No. 48 (1967)
- Donald M. Kaplan
- https://exhibits.stanford.edu/ai/catalog/hk625xv7120
Correctness of a Compiler for Arithmetic Expressions
- Mathematical Aspects of Computer Science (1) 1967
- John McCarthy and James A. Painter
- http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.76.7835
- http://www-formal.stanford.edu/jmc/mcpain.html
Formalising Meaning: a History of Programming Language Semantics
- 2019 PhD dissertation; Troy Kaighin Astarte
- http://homepages.cs.ncl.ac.uk/t.astarte/res/pdf/TK_Astarte_Formalising_Meaning_2019.pdf

Lectures

Compiler Verification: The Next Generation
- PurPL Fest 2019; Amal Ahmed
- https://www.youtube.com/watch?v=KcOxdyq1vs0
OPLSS (Oregon Programming Languages Summer School)
- 2019 - https://www.cs.uoregon.edu/research/summerschool/summer19/topics.php
  - Secure Compilation - Amal Ahmed
    - https://www.youtube.com/playlist?list=PL0DsGHMPLUWXXWjqfRuA20FNnNXOTj_Wh
- 2017 - https://www.cs.uoregon.edu/research/summerschool/summer17/topics.php
  - Correct and Secure Compilation for Multi-Language Software - Amal Ahmed
    - https://www.youtube.com/playlist?list=PL0DsGHMPLUWUSJ8_THYt6Jcu7Kgp2kjaP
- 2016 - https://www.cs.uoregon.edu/research/summerschool/summer16/curriculum.php
  - Logical relations/Compiler verification - Amal Ahmed
    - https://www.youtube.com/playlist?list=PLiHLLF-foEexzqkMlTqzbbX_7V45MAXyX
- 2015 - https://www.cs.uoregon.edu/research/summerschool/summer15/curriculum.html
  - Logical Relations - Amal Ahmed
    - https://www.youtube.com/playlist?list=PLiHLLF-foEex7BOvMbrbUFC9XgU7fZW66
- 2014 - https://www.cs.uoregon.edu/research/summerschool/summer14/curriculum.html
  - Software Verification - Andrew Appel
- 2013 - https://www.cs.uoregon.edu/research/summerschool/summer13/curriculum.html
  - Logical Relations - Amal Ahmed
  - Verifying LLVM Optimizations in Coq - Steve Zdancewic
- 2012 - https://www.cs.uoregon.edu/research/summerschool/summer12/curriculum.html
  - Logical Relations - Amal Ahmed
  - Compiler verification - Xavier Leroy

Testing

Readings

An empirical comparison of compiler testing techniques
- International Conference on Software Engineering (ICSE 2016)
- Junjie Chen, Wenxiang Hu, Dan Hao, Yingfei Xiong, Hongyu Zhang, Lu Zhang, Bing Xie
- http://sei.pku.edu.cn/~xiongyf04/papers/ICSE16.pdf
- http://emponcc.github.io/
- https://github.com/emponcc/emponcc.github.io/blob/master/CompilerTestingComparison.md
- https://dl.acm.org/citation.cfm?id=2884878
Automatic Test Case Reduction for OpenCL
- IWOCL 2016
- Moritz Pflanzer, Alastair F. Donaldson, Andrei Lascu
- https://dl.acm.org/citation.cfm?id=2909439
- paper: https://spiral.imperial.ac.uk/handle/10044/1/39576
- slides: http://www.iwocl.org/wp-content/uploads/iwocl-2016-automatic-test-case-reduction.pdf
Automated Testing of Graphics Shader Compilers
- SPLASH 2017 OOPSLA
- Alastair F. Donaldson, Hugues Evrard, Andrei Lascu, Paul Thomson
- https://dl.acm.org/citation.cfm?id=3152284.3133917
- https://www.doc.ic.ac.uk/~afd/homepages/papers/pdfs/2017/OOPSLA.pdf
- Overview of the GLFuzz transformations - https://medium.com/@afd_icl/overview-of-the-glfuzz-transformations-d530540a5a18
Automatic Testing of Symbolic Execution Engines via Program Generation and Differential Testing
- ASE 2017
- Timotej Kapus, Cristian Cadar
- https://srg.doc.ic.ac.uk/files/papers/symex-engine-tester-ase-17.pdf
Causal Distance-Metric-Based Assistance for Debugging After Compiler Fuzzing
- IEEE International Symposium on Software Reliability Engineering (ISSRE) 2018
- Josie Holmes and Alex Groce
- https://agroce.github.io/issre18.pdf
Checking Correctness of Code Generator Architecture Specifications
- Code Generation and Optimization (CGO) 2015
- N. Hasabnis, R. Qiao, R. Sekar
- http://www3.cs.stonybrook.edu/~nhasabni/papers/cgo15.pdf
- http://www3.cs.stonybrook.edu/~nhasabni/papers/cgo15_talk.pdf
Compiler fuzzing, part 1
- http://www.vegardno.net/2018/06/compiler-fuzzing.html
Compiler Fuzzing through Deep Learning
- International Symposium on Software Testing and Analysis (ISSTA) 2018
- Chris Cummins, Pavlos Petoumenos, Hugh Leather and Alastair Murray
- http://homepages.inf.ed.ac.uk/hleather/publications/2018_deepfuzzing_issta.pdf
- https://chriscummins.cc/deepsmith
Compiler Fuzzing: How Much Does It Matter?
- OOPSLA 2019
- Michael Marcozzi, Qiyi Tang, Alastair Donaldson, Cristian Cadar
- https://srg.doc.ic.ac.uk/projects/compiler-bugs/
- https://sites.google.com/view/michaelmarcozzi/software-tools/compiler-bugs-impact
- A Systematic Impact Study for Fuzzer-Found Compiler Bugs
  - 2019 arXiv (pre-print)
  - https://arxiv.org/abs/1902.09334
  - https://sites.google.com/view/michaelmarcozzi/compiler-bugs
- PapersWeLove London 2020
  - Michael Marcozzi
  - https://www.youtube.com/watch?v=CUBmXYahTb0
Coverage Prediction for Accelerating Compiler Testing
- IEEE Transactions on Software Engineering (2019)
- Junjie Chen, Guancheng Wang, Dan Hao, Yingfei Xiong, Hongyu Zhang, Lu Zhang, Bing Xie
- https://ieeexplore.ieee.org/abstract/document/8588375
CUDAsmith: A Fuzzer for CUDA Compilers
- Computers, Software and Applications Conference (COMPSAC) 2020
- Bo Jiang, Xiaoyan Wang, W. K. Chan, T. H. Tse, Na Li, Yongfeng Yin, Zhenyu Zhang
- https://www.cs.hku.hk/data/techreps/document/TR-2020-05.pdf
- https://github.com/gongbell/CUDAsmith
DATm: Diderot's Automated Testing Model.
- 39th International Conference on Software Engineering ICSE (12th International Workshop on Automation of Software Test AST) 2017
- C. Chiw, G. Kindlmann, J. Reppy
- https://www.researchgate.net/publication/317836930_DATm_Diderot%27s_Automated_Testing_Model
- https://www.dropbox.com/s/5twsrp12vg4or7t/datm_talk.key?dl=0
Deep Differential Testing of JVM Implementations
- ICSE 2019
- Yuting Chen, Ting Su, Zhendong Su
- https://tingsu.github.io/files/icse19-classming.pdf
DeepFuzz: Automatic Generation of Syntax Valid C Programs for Fuzz Testing
- AAAI Conference on Artificial Intelligence (AAAI) 2019
- Xiao Liu, Xiaoting Li, Rupesh Prajapati, Dinghao Wu
- https://faculty.ist.psu.edu/wu/papers/DeepFuzz.pdf
- https://github.com/s3team/DeepFuzz
Differential Testing for Software
- Digital Technical Journal 10, 1 (1998)
- William M. McKeeman
- http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.83.445
Effect-Driven QuickChecking of Compilers
- ICFP 2017
- Jan Midtgaard, Mathias Nygaard Justesen, Patrick Kasting, Flemming Nielson, Hanne Riis Nielson
- paper: http://janmidtgaard.dk/papers/Midtgaard-al%3AICFP17-full.pdf
- implementation: https://github.com/jmid/efftester
- talk
  - http://podcasts.ox.ac.uk/effect-driven-quickchecking-compilers
  - https://www.youtube.com/watch?v=_KrZzaShDew&list=PLnqUlCo055hW7kU-SBQEhC_87etA5Gqlq&index=15
Extending Equivalence Transformation Based Program Generator for Random Testing of C Compilers
- A-TEST 2018
- Shogo Takakura, Mitsuyoshi Iwatsuji, Nagisa Ishiura
- https://dl.acm.org/citation.cfm?doid=3278186.3278188
- https://ist.ksc.kwansei.ac.jp/~ishiura/publications/C2018-11a.pdf
Finding and Analyzing Compiler Warning Defects
- ICSE 2016
- Chengnian Sun, Vu Le, Zhendong Su
- http://ieeexplore.ieee.org/document/7886904/
- https://web.cs.ucdavis.edu/~su/publications/icse16-warning.pdf
Finding and Understanding Bugs in C Compilers
- PLDI 2011
- Xuejun Yang, Yang Chen, Eric Eide, John Regehr
- http://www.cs.utah.edu/~regehr/papers/pldi11-preprint.pdf
- https://www.flux.utah.edu/download?uid=114
- https://blog.regehr.org/archives/492
- http://lambda-the-ultimate.org/node/4241
Fuzzing with Grammars
- In "The Fuzzing Book" - https://www.fuzzingbook.org/
- Andreas Zeller, Rahul Gopinath, Marcel Böhme, Gordon Fraser, Christian Holler
- https://www.fuzzingbook.org/html/Grammars.html
Fuzzing the .NET JIT Compiler
- http://mattwarren.org/2018/08/28/Fuzzing-the-.NET-JIT-Compiler/
- Fuzzlyn: Fuzzer for the .NET toolchains
  - https://github.com/jakobbotsch/Fuzzlyn
History-Guided Configuration Diversification for Compiler Test-Program Generation
- Automated Software Engineering (ASE) 2019
- Junjie Chen, Guancheng Wang, Dan Hao, Yingfei Xiong, Hongyu Zhang, Lu Zhang
- https://xiongyingfei.github.io/publications.html#ASE19b
Improving the Utility of Compiler Fuzzers
- 2014 Ph.D. Dissertation; Yang Chen
- http://www.cs.utah.edu/~chenyang/papers/thesis_draft.pdf
- https://search.proquest.com/openview/4799de27b7f7d50c7d4f1d2335316065/1?pq-origsite=gscholar&cbl=18750&diss=y
K-CONFIG: Using Failing Test Cases to Generate Test Cases in GCC Compilers
- Automated Software Engineering (ASE 2019) Late Breaking Research-Track
- Md Rafiqul Islam Rabin, Mohammad Amin Alipour
- https://arxiv.org/abs/1908.10481
Learning to Accelerate Compiler Testing
- International Conference on Software Engineering (ICSE), Doctoral Symposium, 2018
- Junjie Chen
- https://www.icse2018.org/event/icse-2018-doctoral-symposium-learning-to-accelerate-compiler-testing
- Slides (2017): http://materials.dagstuhl.de/files/17/17502/17502.JunjieChen.Slides.pdf
Learning to Prioritize Test Programs for Compiler Testing
- International Conference on Software Engineering (ICSE 2017)
- Junjie Chen, Yanwei Bai, Dan Hao, Yingfei Xiong, Hongyu Zhang, Bing Xie
- http://sei.pku.edu.cn/~xiongyf04/papers/ICSE17b.pdf
Nautilus: Fishing for Deep Bugs with Grammars
- Network and Distributed System Security Symposium (NDSS) 2019
- Cornelius Aschermann, Tommaso Frassetto, Thorsten Holz, Patrick Jauernig, Ahmad-Reza Sadeghi, Daniel Teuchert
- https://www.syssec.ruhr-uni-bochum.de/research/publications/nautilus/
- https://github.com/RUB-SysSec/nautilus
- testing applications: ChakraCore (the JavaScript engine of Microsoft Edge), PHP, mruby, and Lua
RandIR: Differential Testing for Embedded Compilers
- SCALA 2016
- Georg Ofenbeck, Tiark Rompf, Markus Püschel
- https://www.cs.purdue.edu/homes/rompf/papers/ofenbeck-scala16.pdf
ReduKtor: How We Stopped Worrying About Bugs in Kotlin Compiler
- Automated Software Engineering (ASE) 2019
- Daniil Stepanov, Marat Akhin, Mikhail Belyaev
- https://arxiv.org/abs/1909.07331
System Under Test: LLVM - https://systemundertest.org/llvm/
Taming compiler fuzzers
- PLDI 2013
- Y. Chen, A. Groce, C. Zhang, W.-K. Wong, X. Fern, E. Eide, J. Regehr
- https://www.cs.utah.edu/~regehr/papers/pldi13.pdf
- Fuzzers Need Taming - https://blog.regehr.org/archives/925
Test-Case Reduction for C Compiler Bugs
- PLDI 2012
- John Regehr, Yang Chen, Pascal Cuoq, Eric Eide, Chucky Ellison, Xuejun Yang
- https://www.cs.utah.edu/~regehr/papers/pldi12-preprint.pdf
Testing LLVM - http://blog.regehr.org/archives/1450
Testing Static Analyses for Precision and Soundness
- Code Generation and Optimization (CGO) 2020
- Jubi Taneja, Zhengyang Liu, John Regehr
- http://www.cs.utah.edu/~regehr/cgo20.pdf
- https://github.com/jubitaneja/souper-cgo20-artifact
- Testing Dataflow Analyses for Precision and Soundness
  - https://blog.regehr.org/archives/1709
  - LLVM Dataflow Info Printer Pass
    - https://github.com/regehr/llvm-dataflow-info
The problem with differential testing is that at least one of the compilers must get it right
- http://blog.frama-c.com/index.php?post/2013/09/25/The-problem-with-differential-testing-is-that-at-least-one-of-the-compilers-must-get-it-right

Performance Optimization

Compiler Testing via a Theory of Sound Optimisations in the C11/C++11 Memory Model
- Programming Language Design and Implementation (PLDI) 2013
- Robin Morisset, Pankaj Pawan, Francesco Zappa Nardelli
- https://www.di.ens.fr/~zappa/readings/pldi13.pdf
Detecting Arithmetic Optimization Opportunities for C Compilers by Randomly Generated Equivalent Programs
- IPSJ Transactions on System LSI Design Methodology, vol. 9, 2016; A. Hashimoto and N. Ishiura
- https://www.jstage.jst.go.jp/article/ipsjtsldm/9/0/9_21/_article
Detecting Missed Arithmetic Optimization in C Compilers by Differential Random Testing
- The 20th Workshop on Synthesis And System Integration of Mixed Information technologies (SASIMI 2016)
- Mitsuyoshi Iwatsuji, Atsushi Hashimoto, Nagisa Ishiura
- http://ist.ksc.kwansei.ac.jp/~ishiura/publications/C2016-10a.pdf
Evaluating the Effects of Compiler Optimizations on Mutation Testing at the Compiler IR Level
Finding Missed Compiler Optimizations by Differential Testing
- Compiler Construction (CC) 2018
- Gergö Barany
- https://github.com/gergo-/missed-optimizations/raw/master/missed_optimizations_preprint.pdf
- Missed optimizations in C compilers: https://github.com/gergo-/missed-optimizations/
- https://hal.inria.fr/hal-01682683
Lost in translation: Exposing hidden compiler optimization opportunities
- 2019 arXiv
- Kyriakos Georgiou, Zbigniew Chamski, Andres Amaya Garcia, David May, Kerstin Eder
- https://arxiv.org/abs/1903.11397
- https://github.com/TrustworthySystemLab/LostInTranslation
Random Testing of Compilers’ Performance Based on Mixed Static and Dynamic Code Comparison
- A-TEST 2018
- Kota Kitaura, Nagisa Ishiura
- https://ist.ksc.kwansei.ac.jp/~ishiura/publications/C2018-11b.pdf
- https://dl.acm.org/citation.cfm?id=3278192
Reinforcing Random Testing of Arithmetic Optimization of C Compilers by Scaling up Size and Number of Expressions
- IPSJ Transactions on System LSI Design Methodology, vol. 7, 2014
- E. Nagai, A. Hashimoto, N. Ishiura
- https://www.jstage.jst.go.jp/article/ipsjtsldm/7/0/7_91/_article
Scaling up Size and Number of Expressions in Random Testing of Arithmetic Optimization of C Compilers
- SASIMI 2013
- E. Nagai, A. Hashimoto, N. Ishiura
- http://ist.ksc.kwansei.ac.jp/~ishiura/publications/C2013-10.pdf
The Correctness-Security Gap in Compiler Optimization
- LangSec 2015, IEEE SPW
- Vijay D'Silva, Mathias Payer, Dawn Song
- paper: https://research.google.com/pubs/pub43856.html
- slides: https://nebelwelt.net/publications/files/15LangSec-presentation.pdf
- talk: https://www.youtube.com/watch?v=g6LCtHz_MDc&list=PL0pRF4xvoD0kuECJuowraVIIHlT3pN1Cm&index=3

Software

CF3: Test suite for arithmetic optimization of C compilers
- https://ist.ksc.kwansei.ac.jp/~ishiura/pub/CF3/
- https://github.com/ishiura-compiler/CF3
Csmith, a random generator of C programs
C-Reduce, a C program reducer
- https://embed.cs.utah.edu/creduce/
- https://github.com/csmith-project/creduce
- https://github.com/zjturner/creduce-windows
- Design and Evolution of C-Reduce
  - Part 1: https://blog.regehr.org/archives/1678
  - Part 2: https://blog.regehr.org/archives/1679
Fuzzing LLVM libraries and tools - https://llvm.org/docs/FuzzingLLVM.html
- Adventures in Fuzzing Instruction Selection
  - 2017 EuroLLVM Developers’ Meeting; Justin Bogner
  - http://llvm.org/devmtg/2017-03/assets/slides/adventures_in_fuzzing_instruction_selection.pdf
  - https://www.youtube.com/watch?v=UBbQ_s6hNgg
- Structure-aware fuzzing for Clang and LLVM with libprotobuf-mutator
  - 2017 LLVM Developers’ Meeting; Kostya Serebryany, Vitaly Buka, Matt Morehouse
  - https://www.youtube.com/watch?v=U60hC16HEDY
gcc-for-llvm-testing: A modified GCC test suite suitable for testing non-GCC compilers
- https://github.com/embecosm/gcc-for-llvm-testing
- Using the GCC regression test suite for LLVM (and other compilers)
  - GNU Tools Cauldron 2018; Simon Cook
  - https://speakerdeck.com/simonpcook/using-the-gcc-regression-test-suite-for-llvm-and-other-compilers
- Repurposing GCC Regression for LLVM Based Tool Chains
  - 2018 LLVM Developers’ Meeting; Jeremy Bennett
  - https://www.youtube.com/watch?v=GV4PoWu0UZ0
GraphicsFuzz: A testing framework for automatically finding and simplifying bugs in graphics shader compilers.
- https://github.com/google/graphicsfuzz
- GraphicsFuzz: Metamorphic Testing for Graphics Shader Compilers
  - VF Conference 2019; Alastair Donaldson
  - https://www.youtube.com/watch?v=r2GHwhCbcKo
kscope
- a library which recursively generates randomized code while keeping it 100% equivalent to the original one
- http://ithare.com/c17-compiler-bug-hunt-very-first-results-12-bugs-reported-3-already-fixed/
- https://github.com/ITHare/kscope
lang_tester: Rust testing framework for compilers and VMs
- https://crates.io/crates/lang_tester
ldrgen: Liveness-driven random C code generator - https://github.com/gergo-/ldrgen
llvm-mutate – mutate LLVM IR - http://eschulte.github.io/llvm-mutate/
opt-fuzz: a simple implementation of bounded exhaustive testing for LLVM programs
- https://github.com/regehr/opt-fuzz
Orange3
- a tool to test C compilers with randomly generated programs; mainly targets arithmetic optimization such as constant folding.
- https://ist.ksc.kwansei.ac.jp/~ishiura/pub/orange3/
- https://github.com/ishiura-compiler/Orange3
Orange4
- a tool to test C compilers by randomly generated programs; based on equivalent transformations on C programs and can generate wider class of C test programs than Orange3.
- https://ist.ksc.kwansei.ac.jp/~ishiura/pub/orange4/
- https://github.com/ishiura-compiler/Orange4
OutputCheck: A tool for checking tool output inspired by LLVM's FileCheck
- https://github.com/stp/OutputCheck/
prog-fuzz: Compiler/source code fuzzing tool using AFL instrumentation
- https://github.com/vegard/prog-fuzz
Quest: A tool for testing C compilers - https://github.com/lindig/quest
shader-compiler-bugs: A collection of shader compiler bugs - https://github.com/mc-imperial/shader-compiler-bugs
yarpgen: Yet Another Random Program Generator
- a random C/C++ program generator, which produces correct runnable C/C++ programs
- specifically designed to trigger compiler optimization bugs and is intended for compiler testing
- https://github.com/intel/yarpgen

Talks

A Year of Experience with Broad Based Continuous Testing with GCC
Adventures in Fuzzing Instruction Selection
- EuroLLVM 2017; Justin Bogner
- https://www.youtube.com/watch?v=UBbQ_s6hNgg
- http://llvm.org/devmtg/2017-03//assets/slides/adventures_in_fuzzing_instruction_selection.pdf
Coverage-Directed Differential Testing of JVM Implementations
- PLDI 2016; Yuting Chen
- https://www.youtube.com/watch?v=2Reaqfp4v-g
- http://cse.sjtu.edu.cn/~zhao/pub/pdf/pldi2016.pdf
Exposing Difficult Compiler Bugs With Random Testing
- GCC Developers' Summit 2010; John Regehr, Xuejun Yang, Yang Chen, Eric Eide
- https://gcc.gnu.org/wiki/summit2010?action=AttachFile&do=get&target=regehr_gcc_summit_2010.pdf
FileCheck
- FileCheck Follies
  - 2016 LLVM Developers’ Meeting; Paul Robinson
  - https://www.youtube.com/watch?v=4rhW8knj0L8
  - http://www.llvm.org/devmtg/2016-11/Slides/Robinson-FilecheckFollies.pdf
- FileCheck: learning arithmetic
  - 2019 LLVM Developers’ Meeting; Thomas Preud'homme
  - https://www.youtube.com/watch?v=mcrQ5f-mASw
  - https://llvm.org/devmtg/2019-10/slides/Preudhomme-FileCheck.pdf
Finding Missed Optimizations in LLVM (and other compilers)
- EuroLLVM 2018; Gergö Barany
- https://www.youtube.com/watch?v=V6ug3e3jC54
- http://llvm.org/devmtg/2018-04/slides/Barany-Finding%20Missed%20Optimizations%20in%20LLVM.pdf
- https://github.com/gergo-/missed-optimizations/
Getting Started with the LLVM Testing Infrastructure
- 2019 LLVM Developers’ Meeting; Brian Homerding, Michael Kruse
- https://www.youtube.com/watch?v=isVQ8kYqaSA
- https://llvm.org/devmtg/2019-10/slides/Homerding-Kruse-LLVMTestingInfrastructureTutorial.pdf
Testing and Qualification of Optimizing Compilers for Functional Safety
- 2019 EuroLLVM Developers’ Meeting; José Luis March Cabrelles
- https://www.youtube.com/watch?v=nSfT4oND9dU
Testing Language Implementations
- Programming Language Implementation Summer School (PLISS) 2017; Alastair Donaldson
- https://www.youtube.com/watch?v=ZJUk8_k1HbY

Validation

Validation: Including translation validation, equivalence checking.

Black-Box Equivalence Checking Across Compiler Optimizations
- APLAS 2017
- Manjeet Dahiya, Sorav Bansal
- http://www.cse.iitd.ac.in/~sbansal/pubs/aplas17.pdf
Modeling undefined behaviour semantics for checking equivalence across compiler optimizations
- HVC 2017; Manjeet Dahiya, Sorav Bansal
- http://www.cse.iitd.ernet.in/~sbansal/pubs/hvc17.pdf
- http://www.cse.iitd.ac.in/~dahiya/hvc17.pdf
Evaluating value-graph translation validation for LLVM
- Programming and Language Design Implementation (PLDI) 2011
- Jean-Baptiste Tristan, Paul Govereau, Greg Morrisett
- https://dl.acm.org/citation.cfm?id=1993498.1993533
Formally Verified Compilation of Low-Level C code
- 2016 PhD Dissertation; Pierre Wilke
- https://tel.archives-ouvertes.fr/tel-01483676
Proving the correctness of heuristically optimized code
- CACM 1978; Hanan Samet
- http://www.cs.umd.edu/~hjs/pubs/compilers/proving-correctness.pdf
Semantic Program Alignment for Equivalence Checking
- PLDI 2019
- Berkeley Churchill, Oded Padon, Rahul Sharma, Alex Aiken
- https://www.youtube.com/watch?v=FJGvInzaiAQ
- https://raw.githubusercontent.com/bchurchill/pldi19-equivalence-checker/master/pldi2019.pdf
- https://pldi19.sigplan.org/details/pldi-2019-papers/14/Semantic-Program-Alignment-for-Equivalence-Checking
- Semantic Alignment Equivalence Checker
  - https://github.com/bchurchill/pldi19-equivalence-checker
Translation validation
- TACAS 1998
- Amir Pnueli, Michael Siegel, Eli Singerman
- https://dl.acm.org/citation.cfm?id=349314
- "We present the notion of translation validation as a new approach to the verification of translators (compilers, code generators). Rather than proving in advance that the compiler always produces a target code which correctly implements the source code (compiler verification), each individual translation (i.e. a run of the compiler) is followed by a validation phase which verifies that the target code produced on this run correctly implements the submitted source program."
Translation Validation: Automatically Proving the Correctness of Translations Involving Optimized Code
- Hanan Samet
- translation validation - http://www.cs.umd.edu/~hjs/hjscat.html#sectiontranslationvalidation
- compiler testing - http://www.cs.umd.edu/~hjs/hjscat.html#sectioncompilertesting
- http://www.cs.umd.edu/~hjs/pubs/compilers/CS-TR-75-498.pdf
- http://www.cs.umd.edu/~hjs/slides/translation-validation-slides.pdf
Translation Validation: From DC+ to C*
- FM-Trends 1998
- Amir Pnueli, Ofer Shtrichman, Michael Siegel
- https://dl.acm.org/citation.cfm?id=729871
- "Translation validation is an alternative to the verification of translators (compilers, code generators). Rather than proving in advance that the compiler always produces a target code which correctly implements the source code (compiler verification), each individual translation (i.e. a run of the compiler) is followed by a validation phase which verifies that the target code produced on this run correctly implements the submitted source program."
Translation validation for an optimizing compiler
- PLDI 2000
- George C. Necula
- https://dl.acm.org/citation.cfm?id=349314
Translation Validation for Verified, Efficient and Timely Operating Systems
- 2017 PhD Dissertation; Thomas Sewell
- http://handle.unsw.edu.au/1959.4/58861
- https://ts.data61.csiro.au/publications/papers/Sewell:phd
- https://ts.data61.csiro.au/projects/TS/compiler-correctness.pml
Translation Validation of Bounded Exhaustive Test Cases
- 2017; Nuno Lopes, John Regehr
- https://blog.regehr.org/archives/1510
- Translation Validation with Alive - https://github.com/nunoplopes/alive/tree/newsema/tv
Validating Optimizations of Concurrent C/C++ Programs
- CGO 2016
- Soham Chakraborty, Viktor Vafeiadis
- https://plv.mpi-sws.org/validc/paper.pdf

Verification

A Higher-Order Abstract Syntax Approach to the Verified Compilation of Functional Programs
- 2016 PhD thesis; Yuting Wang
- https://arxiv.org/abs/1702.03363
- http://www.cs.yale.edu/homes/wang-yuting/files/phd_thesis.pdf
A Verified Compiler for a Linear Function/Imperative Intermediate Language
- 2018 PhD Thesis; Sigurd Schneider
- https://www.ps.uni-saarland.de/Publications/documents/Schneider_2018_PhDThesis.pdf
- LVC - Linear Verified Compiler: https://www.ps.uni-saarland.de/~sdschn/LVC.html
A Verified Packrat Parser Interpreter for Parsing Expression Grammars
- Conference on Certified Programs and Proofs (CPP) 2020
- Clement Blaudeau, Natarajan Shankar
- https://arxiv.org/abs/2001.04457
- https://www.youtube.com/watch?v=hLjRcMAMuts
ALIVe: Automatic LLVM InstCombine Verifier
- https://github.com/nunoplopes/alive
- online: http://rise4fun.com/Alive
- blog post: http://blog.regehr.org/archives/1170
- slides: http://llvm.org/devmtg/2014-10/Slides/Menendez-Alive.pdf
- Alive-FP: Automated Verification of Floating Point Based Peephole Optimizations in LLVM
  - SAS 2016; David Menendez, Santosh Nagarakatte, Aarti Gupta
  - https://www.cs.rutgers.edu/research/technical_reports/report.php?series_id=1&report_id=723
- Alive-Loops: https://github.com/rutgers-apl/alive-loops
  - Termination checking for LLVM peephole optimizations
  - ICSE 2016; David Menendez, Santosh Nagarakatte
  - https://www.cs.rutgers.edu/~sn349/papers/icse2016-alive-loops.pdf
- Alive-NJ - https://github.com/rutgers-apl/alive-nj
- LifeJacket: Verifying precise floating-point optimizations in LLVM
  - SOAP 2016, EuroLLVM 2017; Andres Nötzli, Fraser Brown
  - http://export.arxiv.org/abs/1603.09290
  - https://github.com/4tXJ7f/alive
  - http://llvm.org/devmtg/2017-03/2017/02/20/accepted-sessions.html#25
- Practical Formal Techniques and Tools for Developing LLVM's Peephole Optimizations
  - 2018 PhD Thesis; David Menendez
  - https://www.cs.rutgers.edu/~santosh.nagarakatte/david-menendez-phd-thesis.pdf
- Precondition Inference for Peephole Optimizations in LLVM
  - PLDI 2017
  - David Menendez, Santosh Nagarakatte
  - http://export.arxiv.org/abs/1611.05980
  - https://www.cs.rutgers.edu/~santosh.nagarakatte/papers/pldi2017-alive-infer.pdf
  - PLDI 2017 talk, David Menendez - https://pldi17.sigplan.org/event/pldi-2017-papers-precondition-inference-for-peephole-optimizations-in-llvm
- Provably Correct Peephole Optimizations with Alive
  - PLDI 2015
  - Nuno P. Lopes, David Menendez, Santosh Nagarakatte, John Regehr
  - https://www.cs.utah.edu/~regehr/papers/pldi15.pdf
  - http://web.ist.utl.pt/nuno.lopes/pubs.php?id=alive-pldi15
- AliveInLean: A Verified LLVM Peephole Optimization Verifier
  - Computer-Aided Verification (CAV) 2019
  - Juneyoung Lee, Chung-Kil Hur, Nuno P. Lopes
  - https://sf.snu.ac.kr/aliveinlean/
  - https://github.com/Microsoft/AliveInLean
Alive2: Automatic verification of LLVM optimizations
- https://github.com/AliveToolkit/alive2
- Alive2: Verifying Existing Optimizations
  - 2019 LLVM Developers’ Meeting; Nuno Lopes, John Regehr
  - https://www.youtube.com/watch?v=paJhdBp_iA4
  - https://llvm.org/devmtg/2019-10/slides/Lopes-Regehr-Alive2.pdf
- Alive 2 Part 1: Introduction
  - https://blog.regehr.org/archives/1722
An Abstract Stack Based Approach to Verified Compositional Compilation to Machine Code
- POPL 2019
- Yuting Wang, Pierre Wilke, Zhong Shao
- https://popl19.sigplan.org/event/popl-2019-research-papers-an-abstract-stack-based-approach-to-verified-compositional-compilation-to-machine-code
Blackbox Equivalence Checking of Program Optimizations
- 2019 Ph.D. Dissertation; Berkeley Roshan Churchill
- https://theory.stanford.edu/~aiken/publications/theses/churchill.pdf
CakeML: A Verified Implementation of ML
- https://cakeml.org/
- https://github.com/CakeML/cakeml
- Verified Compilation of CakeML to Multiple Machine-Code Targets
  - Certified Programs and Proofs (CPP) 2017
  - Anthony Fox, Magnus O. Myreen, Yong Kiam Tan, Ramana Kumar.
  - http://www.cl.cam.ac.uk/~mom22/cpp17.pdf
  - http://www.cl.cam.ac.uk/~mom22/publications.html
- Verified Compilation on a Verified Processor
  - PLDI 2019
  - Andreas Lööw, Ramana Kumar, Yong Kiam Tan, Magnus O. Myreen, Michael Norrish, Oskar Abrahamsson, Anthony Fox
  - https://cakeml.org/pldi19.pdf
- The Verified CakeML Compiler Backend
  - JFP 2019
  - Yong Kiam Tan, Magnus O. Myreen, Ramana Kumar, Anthony Fox, Scott Owens, Michael Norrish
  - https://www.cs.cmu.edu/~yongkiat/files/cakeml-jfp.pdf
- Icing: Supporting Fast-math Style Optimizations in a Verified Compiler
  - Computer-Aided Verification (CAV) 2019
  - Heiko Becker, Eva Darulova, Magnus O. Myreen, Zachary Tatlock
  - https://cakeml.org/cav19.pdf
CompCert: formally-verified C compiler
- http://compcert.inria.fr/
- https://github.com/AbsInt/CompCert
- The formal verification of compilers - Xavier Leroy - DeepSpec Summer School 2017
  - https://deepspec.org/event/dsss17/lecture_leroy.html
  - http://gallium.inria.fr/~xleroy/courses/DSSS-2017/
- A Verified CompCert Front-End for a Memory Model Supporting Pointer Arithmetic and Uninitialised Data
  - Journal of Automated Reasoning 62(4) (2019)
  - Frédéric Besson, Sandrine Blazy, Pierre Wilke
  - https://doi.org/10.1007/s10817-017-9439-z
  - https://hal.inria.fr/hal-01656895
- An Abstract Stack Based Approach to Verified Compositional Compilation to Machine Code
  - POPL 2019
  - Yuting Wang, Pierre Wilke, Zhong Shao
  - https://www.youtube.com/watch?v=AK3wP1BK-K8
  - https://popl19.sigplan.org/event/popl-2019-research-papers-an-abstract-stack-based-approach-to-verified-compositional-compilation-to-machine-code
  - Stack-Aware CompCert and CompCert MC - https://certikos.github.io/compcertmc/
  - http://flint.cs.yale.edu/flint/publications/sacc.html
- Closing the Gap – The Formally Verified Optimizing Compiler CompCert
  - SSS'17: Safety-critical Systems Symposium 2017
  - https://hal.inria.fr/hal-01399482/
- CompCertM: CompCert with Lightweight Modular Verification and Multi-Language Linking
  - POPL 2020
  - Youngju Song, Minki Cho, Dongjoo Kim, Yonghyun Kim, Jeehoon Kang, Chung-Kil Hur
  - https://sf.snu.ac.kr/compcertm/
- CompCertS: A Memory-Aware Verified C Compiler Using a Pointer as Integer Semantics
  - Journal of Automated Reasoning 63(2) (2019)
  - Frédéric Besson, Sandrine Blazy, Pierre Wilke
  - https://doi.org/10.1007/s10817-018-9496-y
  - https://hal.inria.fr/hal-01656875
- Compositional CompCert
  - POPL 2015
  - Stewart, G., Beringer, L., Cuellar, S., Appel, A.W.
  - https://github.com/PrincetonUniversity/compcomp
- Formal Verification of a Constant-Time Preserving C Compiler
  - Principles of Programming Languages (POPL) 2020
  - Gilles Barthe, Sandrine Blazy, Benjamin Grégoire, Rémi Hutin, Vincent Laporte, David Pichardie, Alix Trieu
  - https://eprint.iacr.org/2019/926
  - http://cs.au.dk/%7Etrieu/POPL20/
  - Slides (Verified Software Workshop): https://vetss.org.uk/wp-content/uploads/sites/122/2019/10/Blazy-Formal-Verification-of-a-Constant-Time.pdf
  - https://vetss.org.uk/verified-software-workshop-programme/
  - https://doi.org/10.1145/3371075
  - A CompCert Compiler that Preserves Cryptographic Constant-time
    - Sandrine Blazy, Rémi Hutin, David Pichardie
    - PriSC 2020 Principles of Secure Compilation
    - https://popl20.sigplan.org/details/prisc-2020-papers/10/A-CompCert-Compiler-that-Preserves-Cryptographic-Constant-time
    - https://www.youtube.com/watch?v=eci48EC4v4o
- Lightweight Verification of Separate Compilation
  - POPL 2016
  - Jeehoon Kang, Yoonseung Kim, Chung-Kil Hur, Derek Dreyer, Viktor Vafeiadis
  - https://sf.snu.ac.kr/sepcompcert/
- Reconciling Low-Level Features of C with Compiler Optimizations
  - 2019 Ph.D. Dissertation; Jeehoon Kang
  - https://sf.snu.ac.kr/jeehoon.kang/thesis/
  - Chapter I, Section 2, Background: A Brief Tour of CompCert
  - Chapter III, Separate Compilation and Linking
  - Chapter IV, Cast between Integers and Pointers
- Verified Peephole Optimizations for CompCert
  - PLDI 2016
  - Eric Mullen, Daryl Zuniga, Zachary Tatlock, Dan Grossman
  - http://peek.uwplse.org/
  - https://conf.researchr.org/event/pldi-2016/pldi-2016-papers-verified-peephole-optimizations-for-compcert-
  - Peek: a verified peephole optimizer for CompCert - https://github.com/uwplse/peek
Compilation Using Correct-by-Construction Program Synthesis
- 2016 Master's Thesis; Clément Pit-Claudel
- http://pit-claudel.fr/clement/MSc/
- https://dspace.mit.edu/bitstream/handle/1721.1/107293/973557793-MIT.pdf?sequence=1
- http://pit-claudel.fr/clement/MSc/FiatToFacade_Pit-Claudel_2016.pdf
Compositional Verification of Compiler Optimisations on Relaxed Memory
- ESOP 2018
- Mike Dodds, Mark Batty, Alexey Gotsman
- https://arxiv.org/abs/1802.05918
Crellvm: Verified Credible Compilation for LLVM
- Programming Languages Design and Implementation (PLDI) 2018
- Jeehoon Kang, Yoonseung Kim, Youngju Song, Juneyoung Lee, Sanghoon Park, Mark Dongyeon Shin, Yonghyun Kim, Sungkeun Cho, Joonwon Choi,Chung-Kil Hur, Kwangkeun Yi
- a verified credible compilation (or equivalently, verified translation validation) framework for LLVM
- http://sf.snu.ac.kr/crellvm/
- http://sf.snu.ac.kr/gil.hur/publications/crellvm.pdf
- http://sf.snu.ac.kr/gil.hur/publications/crellvm.zip
- https://github.com/snu-sf/crellvm-tests-parallel
Mechanized Semantics and Verified Compilation for a Dataflow Synchronous Language with Reset
- POPL 2020
- Timothy Bourke, Lélio Brun, Marc Pouzet
- https://popl20.sigplan.org/details/POPL-2020-Research-Papers/20/Mechanized-Semantics-and-Verified-Compilation-for-a-Dataflow-Synchronous-Language-wit
- https://github.com/INRIA/velus
Pilsner: A Compositionally Verified Compiler for a Higher-Order Imperative Language
- International Conference on Functional Programming (ICFP) 2015
- Georg Neis, Chung-Kil Hur, Jan-Oliver Kaiser, Craig McLaughlin, Derek Dreyer, Viktor Vafeiadis
- https://people.mpi-sws.org/~dreyer/papers/pilsner/paper.pdf
- http://plv.mpi-sws.org/pils/
Pushing the Limits of Compiler Verification
- 2018 PhD Thesis; Eric Mullen
- https://homes.cs.washington.edu/~djg/theses/mullen_thesis.pdf
- Œuf: Verified Coq Extraction in Coq - https://github.com/uwplse/oeuf
- Peek: Verified Peephole Optimizations for CompCert - https://github.com/uwplse/peek
Self-compilation and self-verification
- 2017 Ph.D. Dissertation; Ramana Kumar
- http://www.sigplan.org/Awards/Dissertation/2017_kumar.pdf
- https://cakeml.org/
The Correctness of a Code Generator for a Functional Language
- Verification, Model Checking, and Abstract Interpretation (VMCAI) 2020
- Nathanaël Courant, Antoine Séré, Natarajan Shankar
- https://doi.org/10.1007/978-3-030-39322-9_4
Towards Formally Verified Just-In-Time Compilation
- CoqPL 2020
- Aurèle Barrière, Sandrine Blazy, David Pichardie
- https://www.youtube.com/watch?v=Be-CwSREJOI
- https://popl20.sigplan.org/details/CoqPL-2020-papers/4/Towards-Formally-Verified-Just-in-Time-compilation
Vale (Verified Assembly Language for Everest)
- USENIX Security Symposium 2017
- Barry Bond and Chris Hawblitzel, Manos Kapritsos, K. Rustan M. Leino, Jacob R. Lorch, Bryan Parno, Ashay Rane, Srinath Setty, Laure Thompson
- https://github.com/project-everest/vale
- https://www.microsoft.com/en-us/research/publication/vale-verifying-high-performance-cryptographic-assembly-code/
- https://www.usenix.org/conference/usenixsecurity17/technical-sessions/presentation/bond
Vellvm: Verifying the LLVM
- http://www.cis.upenn.edu/~stevez/vellvm/
- https://github.com/vellvm
- DeepSpec Summer School 2017; Steve Zdancewic - https://deepspec.org/event/dsss17/lecture_zdancewic.html
- Galois, Inc. Tech Talk 2018; Steve Zdancewic
  - https://galois.com/blog/2018/07/vellvm-verifying-the-llvm/
  - https://www.youtube.com/watch?v=qGpRKkP8gec
- Strange Loop 2018; Steve Zdancewic
  - https://www.youtube.com/watch?v=q6gSC3OxB_8
  - https://thestrangeloop.com/2018/vellvm---verifying-the-llvm.html
Verified Compilers for a Multi-Language World
- Summit on Advances in Programming Languages (SNAPL) 2015
- Amal Ahmed
- http://www.ccs.neu.edu/home/amal/papers/verifcomp.pdf

Tags: language reading native clr llvm

Last modified 06 April 2022

Compiler correctness reading

A collection of links and resources on compiler correctness, certification, validation, verification.