Distribution (networking, distributed systems, etc) reading

Distributed programming book (Website): "Source repo for the book that I and my students in my course at Northeastern University, CS7680 Special Topics in Computing Systems: Programming Models for Distributed Computing, are writing on the topic of programming models for distributed systems."

This is a book about the programming constructs we use to build distributed systems. These range from the small, RPC, futures, actors, to the large; systems built up of these components like MapReduce and Spark. We explore issues and concerns central to distributed systems like consistency, availability, and fault tolerance, from the lens of the programming models and frameworks that the programmer uses to build these systems.

Notes on Distributed Systems for Young Bloods:

Distributed systems are different because they fail often; Implement backpressure throughout your system; Find ways to be partially available; Use percentiles, not averages; Learn to estimate your capacity; Feature flags are how infrastructure is rolled out; Choose id spaces wisely; Writing cached data back to persistent storage is bad; Extract services.

Ideas and theory

Fallacies of Distributed Computing - Peter Deutsch
Why Distributed Computing? - Jim Waldo
A Note on Distributed Computing - Waldo, Wollrath et al
Harvest, Yield and Scalable Tolerant Systems - Real world applications of CAP from Brewer et al
On Designing and Deploying Internet Scale Services - James Hamilton
The Perils of Good Abstractions - Building the perfect API/interface is difficult
Chaotic Perspectives - Large scale systems are everything developers dislike - unpredictable, unordered and parallel
Stevey's Google Platforms Rant - Yegge's SOA platform experience
Latency Exists, Cope! - Commentary on coping with latency and it's architectural impacts
Latency - the new web performance bottleneck - not at all new (see Patterson), but noteworthy
The Tail At Scale - the latency challenges inherent of dealing with latency in large scale systems
The Byzantine Generals Problem
Distributed Computing Economics - Jim Gray
Rules of Thumb in Data Engineering - Jim Gray and Prashant Shenoy
Impossibility of distributed consensus with one faulty process - also known as FLP
Unreliable Failure Detectors for Reliable Distributed Systems A method for handling the challenges of FLP
Lamport Clocks - How do you establish a global view of time when each computer's clock is independent
Lazy Replication: Exploiting the Semantics of Distributed Services
Scalable Agreement - Towards Ordering as a Service
Scalable Eventually Consistent Counters over Unreliable Networks - Scalable counting is tough in an unreliable world

Pat Helland's works

Data on the Outside versus Data on the Inside - Pat Helland
Memories, Guesses and Apologies - Pat Helland
SOA and Newton's Universe - Pat Helland
Building on Quicksand - Pat Helland
Life Beyond Distributed Transactions - Helland
If you have too much data, then 'good enough' is good enough - NoSQL, Future of data theory - Pat Helland

CAP Theorem

CAP Conjecture - Consistency, Availability, Parition Tolerance cannot all be satisfied at once
Consistency, Availability, and Convergence - Proves the upper bound for consistency possible in a typical system
CAP Twelve Years Later: How the "Rules" Have Changed - Eric Brewer expands on the original tradeoff description
You Can't Sacrifice Partition Tolerance - Additional CAP commentary

Vogels' works

Consistency and Availability - Vogels
Eventual Consistency - Vogels

Transactions and commits

Avoiding Two-Phase Commit - Two phase commit avoidance approaches
2PC or not 2PC, Wherefore Art Thou XA? - Two phase commit isn't a silver bullet
Starbucks doesn't do two phase commit - Asynchronous mechanisms at work
Optimistic Replication - Relaxed consistency approaches for data replication

Languages and Tools

Programming Distributed Erlang Applications: Pitfalls and Recipes - Building reliable distributed applications isn't "just" using Erlang/OTP.

Infrastructure

Principles of Robust Timing over the Internet - Managing clocks is essential for even basics such as debugging

Storage

Paxos Consensus and other consensus papers

Paxos Made Simple - Leslie Lamport - read first
The Part-Time Parliament - Leslie Lamport
Paxos Made Live - An Engineering Perspective - Chandra et al
Revisiting the Paxos Algorithm - Lynch et al
How to build a highly available system with consensus - Butler Lampson
Reconfiguring a State Machine - Lamport et al - changing cluster membership
Implementing Fault-Tolerant Services Using the State Machine Approach: a Tutorial - Fred Schneider
Mencius: Building Efficient Replicated State Machines for WANs - consensus algorithm for wide-area network
In Search of an Understandable Consensus Algorithm - The extended version of the RAFT paper, an alternative to PAXOS.

Gossip Protocols (Epidemic Behaviours)

P2P

Chord: A Scalable Peer-to-peer Lookup Protocol for Internet Applications
Kademlia: A Peer-to-peer Information System Based on the XOR Metric
Pastry: Scalable, decentralized object location and routing for large-scale peer-to-peer systems
PAST: A large-scale, persistent peer-to-peer storage utility - storage system atop Pastry
SCRIBE: A large-scale and decentralised application-level multicast infrastructure - wide area messaging atop Pastry

Amazon, Google

Reading--some tech, some culture

A Conversation with Werner Vogels - Coverage of Amazon's transition to a service-based architecture
Discipline and Focus - Additional coverage of Amazon's transition to a service-based architecture
Vogels on Scalability
SOA creates order out of chaos @ Amazon
MapReduce
Chubby Lock Manager
Google File System
BigTable
Data Management for Internet-Scale Single-Sign-On
Dremel: Interactive Analysis of Web-Scale Datasets
Large-scale Incremental Processing Using Distributed Transactions and Notifications
Megastore: Providing Scalable, Highly Available Storage for Interactive Services - Smart design for low latency Paxos implementation across datacentres.
Spanner - Google's scalable, multi-version, globally-distributed, and synchronously-replicated database.
Photon - Fault-tolerant and Scalable Joining of Continuous Data Streams. Joins are tough especially with time-skew, high availability and distribution.
Mesa: Geo-Replicated, Near Real-Time, Scalable Data Warehousing - Data warehousing system that stores critical measurement data related to Google's Internet advertising business.

Distributed Locking

For starters, don't do it. Even if you think you need to, don't.

"How to do distributed locking"

Some links on this page copied from A Distributed Systems Reading List

Major distributed system technologies

OMG CORBA

DCOM

Java RMI

ZeroC ICE

Tags: reading distribution list

Last modified 26 April 2022