Debug your programs like they're closed source! (strace, ltrace)
Hidden in Plain Sight: ACM Queue 4(1) (2006); Bryan Cantrill
Multi-scale navigation of large trace data: A survey
- Concurrency and Computation: Practice and Experience, 29(10) 2017
- Ezzati-Jivan, N. & Dagenais, M. R
Survey and Analysis of Kernel and Userspace Tracers on Linux: Design, Implementation, and Overhead
- ACM Computing Surveys (CSUR) Volume 51 Issue 2, March 2018
- Mohamad Gebai, Michel R. Dagenais
SynchroTrace:
- http://vlsi.ece.drexel.edu/index.php?title=SynchroTrace
- https://sites.tufts.edu/tcal/current-research-projects/synchrotrace/
- Synchronization-Aware Architecture-Agnostic Traces for Lightweight Multicore Simulation of CMP and HPC Workloads
- 2018 ACM Transactions on Architecture and Code Optimization (TACO) 15(1)
- http://vlsi.ece.drexel.edu/images/b/b1/ST_2018.pdf
- SynchroTrace: Synchronization-aware Architecture-agnostic Traces for Light-Weight Multicore Simulation
- 2015 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS)
- http://dpac.ece.drexel.edu/wp-content/uploads/2013/04/SynchroTrace.pdf
Hardware Assistance / Processor Tracing
- Hardware-assisted instruction profiling and latency detection
- Hardware-assisted software event tracing
- Hardware trace reconstruction of runtime compiled code
- Hardware Tracing for Fast and Precise Performance Analysis
- In-Depth System Analysis Using Hardware-Assisted Tracing
- Low-Impact System Performance Analysis Using Hardware Assisted Tracing Techniques
- Low Overhead Hardware-Assisted Virtual Machine Analysis and Profiling
- libbts.c: minimal BTS tracing wrapper for Linux Perf
- Andi Kleen's Intel Processor Trace resources
- Intel Processor Trace
- Failure Sketches: A Better Way to Debug
- Failure Sketching: A Technique for Automated Root Cause Diagnosis of In-Production Failures
- Fuzzing
- Internals of Hongfuzz - Intel PT
- PTrix: Efficient Hardware-Assisted Fuzzing for COTS Binary
- WinAFL Intel PT mode
- Intel PT Micro Tutorial - MICRO-48 (2015)
- libipt - an Intel(R) Processor Trace decoder library
- Linux perf Documentation: Intel Processor Trace
- Inferring Fine-grained Control Flow Inside SGX Enclaves with Branch Shadowing
- POMP: Postmortem Program Analysis with Hardware-Enhanced Post-Crash Artifacts
- Processor-Tracing Guided Region Formation in Dynamic Binary Translation
- SATT Software Analyze Trace Tool
- Experimental Linux SW tool to trace, process and analyze full stack SW traces utilizing Intel HW tracing block Intel PT (Intel Processor Trace).
- https://github.com/01org/satt
- Towards Production-Run Heisenbugs Reproduction on Commercial Hardware
- WinIPT: Windows Library for Intel Process Trace
- Windows Intel PT Support
- This driver implements the Intel Processor Trace functionality in Intel Skylake architecture for Microsoft Windows.
- https://github.com/intelpt/WindowsIntelPT
- Harnessing Intel Processor Trace on Windows for Vulnerability Discovery
Software
- barectf: Generator of ANSI C tracers which output CTF
- Drltrace: library calls tracer for Windows and Linux applications
- DTrace
- minitrace: Simple C/C++ library for producing JSON traces suitable for Chrome's built-in trace viewer (about:tracing)
- Phosphor: High performance event tracing
- PIEtrace: Platform Independent Executable Trace
Software: Viewers
- Babeltrace
- FlameGraph: Stack trace visualize
- LTTng Scope
- rgat: An instruction trace visualisation tool for dynamic program analysis
- Trace Compass: an open-source (EPL-licensed) trace viewer and analyzer
Software: Linux
- Linux tracing systems & how they fit together
- KUtrace
- Low-overhead tracing of all Linux kernel-user transitions, for serious performance analysis. Includes kernel patches, loadable module, and post-processing software. Output is HTML/SVG per-CPU-core timeline that you can pan/zoom down to the nanosecond.
- https://github.com/dicksites/KUtrace
- KUTrace: Where have all the nanoseconds gone?, Richard Sites
- Benchmarking "Hello, World!"
- KUtrace 2020
- ltrace: Debugging program to track runtime library calls in dynamically linked programs
- StackTrack — Linux Call graph visualization and execution tracking
- uftrace: Function (graph) tracer for user-space
Software: Linux - BCC/BPF
Software: Linux - BCC/BPF - Readings
- Dive into BPF: a list of reading material
- The BSD Packet Filter
- BPF and XDP Reference Guide
- Brendan D. Gregg
- Ferris Ellis
- Linux Tracing Workshops Materials - Sasha Goldshtein
Software: Linux - BCC/BPF - Software
- BPFtrace: High-level tracing language for Linux eBPF
- BCC - Tools for BPF-based Linux IO analysis, networking, monitoring, and more
- uBPF: Userspace eBPF VM
- ply: Dynamic Tracing in Linux
Software: Linux - BCC/BPF - Talks
- Tools and mechanisms to debug BPF programs
Software: Linux - ftrace
- Ftrace - https://alex.dzyoba.com/blog/ftrace/
- Ftrace: The hidden light switch - https://lwn.net/Articles/608497/
- ftrace: Where modifying a running kernel all started
- Debugging realtime application with Ftrace
- Hooking Linux Kernel Functions
- KernelShark
- perf-tools: Performance analysis tools based on Linux perf_events (aka perf) and ftrace
- Tracing the Linux kernel with ftrace
- Understanding the Linux Kernel via Ftrace - Kernel Recipes 2017 - Steven Rostedt
- See what your computer is doing with Ftrace utilities
Software: Linux - LTTng
- LTTng: Linux Trace Toolkit Next Generation
- LTTng analyses: Analyses scripts for LTTng kernel and user-space traces
- LTTng live streaming example
Software: Linux - ptrace
Software: Linux - strace
- strace - the linux syscall tracer
- Strace little book
- Julia Evans
- How does strace work?
- My Favourite Secret Weapon – strace
- Strace -- The Sysadmin's Microscope
- strace cheat sheet
- Strace: The Lost Chapter
Software: Linux - strace - Talks
- strace talks
- Modern strace
- Postmodern strace
- strace --seccomp-bpf: a look under the hood
- Strace: Monitoring The Kernel-User-Space Conversation
- strace: new features
Software: Windows
- ATrace: a tool for tracing execution of binaries on Windows
- Branch Monitoring Project
- Branch Monitor is an alternative for runtime process monitoring on modern (Windows) systems. Our approach makes use of Branch Trace Store (BTS) from Intel's processors to implement a dynamic, transparent framework. The framework provide many analysis facilities, such as function call tracing and Control Flow Graph (CFG) reconstruction.
- https://github.com/marcusbotacin/BranchMonitoringProject
- DIMCT: Dirty Inter Module Calls Tracer
- libptrace: An event driven multi-core process debugging, tracing, and manipulation framework
- MemTrace: Memory Tracing Software
- tiny_tracer: A Pin Tool for tracing API calls and transition between sections of the traced module
- tracectory: a tool to analyze and visualize x86 instruction traces
- WinIPT: The Windows Library for Intel Process Trace
Software: Windows - ETW
- ETW Central
- etrace: Command-line tool for ETW tracing on files and real-time events
- EtwConsumerNT
- KrabsETW: a modern C++ wrapper and a .NET wrapper around the low-level ETW trace consumption functions
- LiveStacks: Collect, aggregate, and display live stack traces for ETW events, including CPU sampling, of native and .NET processes.
- Hidden Treasure: Detecting Intrusions with ETW
- UIforETW: User interface for recording and managing ETW traces
- wtrace: Command line tracing tool for Windows, based on ETW
Talks
2017
- Now You See Me Too: Visual Tooling for Advanced System Analysis
2016
- Designing Tracing Tools
- Give me 15 minutes and I'll change your view of Linux tracing
- Low-Level Tracing for Latency Analysis: From Baremetal to Hardware Tracing Blocks
- Using Linux tracing tools
2014
- What is That Process Doing?
Tags:
native
linux
windows
reading
Last modified 07 October 2024