Distributed systems are the means by which programs on one machine communicate with another. Differ from architectures in that while an architecture may have a distributed system at its center, it's not a requirement, and while distributed systems may incorporate an architecture as part of their implementation, that architecture usually isn't imposed on the user of the distributed system. Includes networking topics, for example.
Technology stacks/stack-related links:
Interesting links:
Interesting tools based on network APIs
- imap-backup: Backup GMail (or other IMAP) accounts to disk
Auth-n-Auth and SSO
Proprietary/Cloud
- Auth0
- FusionAuth: User authentication and session management framework Source
- Okta
Open Source
Reading and reading lists:
Distributed systems patterns
Networking (Fundamentals, et al)
- An Introduction to Computer Networks (HTML, PDF, Kindle)
- Bits, Signals, and Packets: An Introduction to Digital Communications and Networks
- Code Connected vol.1 (PDF) (book on ZeroMQ)
- Computer Networking : Principles, Protocols and Practice (HTML, ePub, PDF, Kindle)
- Computer Networks: A Systems Approach - Larry Peterson and Bruce Davie (HTML, epub, mobi, PDF)
- Distributed systems for fun and profit
- High-Performance Browser Networking - Ilya Grigorik
- IPv6 for IPv4 Experts - Yar Tikhiy (PDF)
- Kafka gentle introduction - Mitch Seymour
- Kafka, The definitive Guide - Neha Narkhede (PDF)
- Network Science - Albert-Laszló Barabási
- Securing Wireless Networks for the Home User Guide - Mohamed Adel (HTML)
- The TCP/IP Guide
- Understanding IP Addressing: Everything you ever wanted to know (PDF)
- ZeroMQ Guide
HTTP API Design/Implementation/Usage
Detail Pages:
- "Serverless" The serverless architecture, usually implying smaller atoms of functionality executing on cloud servers.
- 0MQ (Zero-MQ) An open-source universal messaging library.
- 8Base A Serverless GraphQL Backend-as-a-Service.
- ActiveMQ The most popular open source, multi-protocol, Java-based message broker.
- ActivityPub A decentralized social networking protocol based upon the ActivityStreams 2.0 data format; it provides a client to server API for creating, updating and deleting content, as well as a federated server to server API for delivering notifications and content.
- ActivityStreams Details a model for representing potential and completed activities using the JSON format; it is intended to be used with vocabularies that detail the structure of activities, and define specific types of activities.
- Aeron Efficient reliable UDP unicast, UDP multicast, and IPC message transport.
- Akka A toolkit and runtime for building highly concurrent, distributed, and fault tolerant event-driven applications.
- Akka.NET A port of the Akka framework to the CLR.
- Anytype A next-generation software that breaks down barriers between applications, gives back privacy and data ownership to users.
- Apache Isis Provides apps with a standard, automatically generated UI.
- Apache Pulsar An open-source distributed pub-sub messaging system.
- Apache Thrift The Apache Thrift software framework, for scalable cross-language services development, combines a software stack with a code generation engine to build services that work efficiently and seamlessly between C++, Java, Python, PHP, Ruby, Erlang, Perl, Haskell, C#, Cocoa, JavaScript, Node.js, Smalltalk, OCaml and Delphi and other languages.
- Armeria Microservice framework for building any type of microservice leveraging your favorite technologies (gRPC, Thrift, Kotlin, Retrofit, Reactive Streams, Spring Boot and Dropwizard).
- ASP.NET/ASP.NET Core Microsoft's HTML/HTTP server-side framework.
- Async API (standard) Open source tools to easily build and maintain your event-driven architecture, powered by the AsyncAPI specification, the industry standard for defining ascychronous APIs.
- Asynchronous Javascript And XML (AJAX) A term coined that describes an approach to using HTML or XHTML, CSS, JavaScript, DOM, XML, XSLT, and most importantly the XMLHttpRequest object to make quick, incremental updates to the UI without reloading the entire browser page.
- Aucoda A "meta-compiler" that can take Au source and transpile to a different application language/platform.
- Awesome (lists) (Project Awesome) A curated collection of lists.
- Azure Messaging A collection of Azure messaging-related links.
- Ballerina A service-oriented programming language.
- Battlesnake A multi-player programming game played by developers all over the world. All you need to play is a live web server that implements the Battlesnake API.
- Beehive An open source event and agent system, which allows you to create your own agents that perform automated tasks triggered by events and filters. It has modules to interface with, talk to, or retrieve information from Twitter, Tumblr, Email, IRC, Jabber, RSS, Jenkins, Hue etc.
- Beej's Guide to Network Programming Intro to C-level sockets (TCP/IP, UDP/IP) programming.
- Bolt protocol A highly efficient, lightweight client-server protocol designed for database applications.
- bore bore is a simple CLI tool for making tunnels to localhost.
- Cadence Fault-Tolerant Stateful Code Platform
- Cadl A language for describing cloud service APIs and generating other API description languages, client and service code, documentation, and other assets.
- CalDAV Defines extensions to the Web Distributed Authoring and Versioning (WebDAV) protocol to specify a standard way of accessing, managing, and sharing calendaring and scheduling information based on the iCalendar format.
- Cassandra An open source NoSQL distributed database trusted by thousands of companies for scalability and high availability without compromising performance. Linear scalability and proven fault-tolerance on commodity hardware or cloud infrastructure make it the perfect platform for mission-critical data.
- Castled Data synchronization framework focused on external apps.
- Cherrybomb A CLI tool that helps you avoid undefined user behaviour by validating your API specifications.
- Clemens Vasters' Messaging Repo Clemens' collection of messaging material and terminology.
- Clio A pure functional lazy-evaluated programming language targeting decentralized and distributed systems.
- Cloud Design Patterns A pattern catalog from Microsoft about how to architect cloud applications.
- Cloud Events A common information model for events.
- CloudMorph Decentralize and self-host cloud gaming/application.
- Coherence A scalable, fault-tolerant, cloud-ready, distributed platform for building grid-based applications and reliably storing data.
- CORBA (Common Object Request Broker Architecture) A Java/.NET interoperability-via-distribution platform.
- CORBA (Common Object Request Broker Architecture) A distributed-objects platform, architecture, and specifications popular in the late 90s and early 2000s.
- Core J2EE Patterns Patterns around enterprise Java circa 2000
- CUE (Configure Unify Execute) Language Validate, define, and use dynamic and text-based data.
- Dapr A portable, event-driven runtime that makes it easy for any developer to build resilient, stateless and stateful applications that run on the cloud and edge and embraces the diversity of languages and developer frameworks.
- Derecho An open-source C++ distributed computing toolkit that provides strong forms of distributed coordination and consistency at RDMA speeds.
- Distribution (networking, distributed systems, etc) reading Generalized reading topics about spreading code around the network.
- DoIt It's Git for Data.
- dotnetRDF An open-source CLR library for RDF.
- DropboxMQ Filesystem-based JMS implementation.
- E An object-oriented programming language for secure distributed computing.
- Ecstasy (XTC) An object-ish language designed to be cloud native from the ground up.
- edn (Extensible Data Notation) A format for serializing data.
- Effective Enterprise Java A collection of topics for effective enterprise systems.
- Enterprise Integration Patterns Hohpe/Woolf's classic messaging text.
- Event-Driven .NET An Event Driven Microservices Platform for .NET.
- Extensible Markup Language (XML) Data or presentation format inspired by HTML and SGML, used extensively in software development.
- FIDO Authentication "The answer to the world's password problem."
- Firebase A (pseudo)real-time NoSQL that also has platform capabilities.
- Flogo An open source, resource efficient, Go-based ecosystem for building event-driven apps. It is designed to abstract the event processing paradigm from event consumption enabling devs to build once, consume from anywhere and process using any of the supported actions.
- Fluid Framework A collection of client libraries for distributing and synchronizing shared state, allowing multiple clients to simultaneously create and operate on shared data structures using coding patterns similar to those used to work with local data.
- Free Programming Books A collection of free learning resources (books).
- FX Command-line tool and terminal JSON viewer.
- Giraffe A functional ASP.NET Core micro web framework for building rich web applications.
- GraphQL A query language APIs can support to enable SQL-like behavior across APIs; supports both query and mutation, and can define complex entity descriptions.
- GraphQL Mesh Allows use of GraphQL query language to access data in remote APIs that don't run GraphQL (and also ones that do run GraphQL); can be used as a gateway to other services, or run as a local GraphQL schema that aggregates data from remote APIs.
- Grouparoo Data synchronization framework.
- gRPC A high-performance, open source universal RPC framework
- Hasura Instant GraphQL on all your data.
- Hypertext Application Language (HAL) An open specification describing a generic structure for RESTful resources.
- HyperText Transmission Protocol Backbone protocol for the World Wide Web.
- Idempotence is not a Medical Condition Idempotence is an essential property for reliable systems; what is it, how do we do it, how can it go wrong, what to do about it.
- IPFS (InterPlanetary FileSystem) protocol A peer-to-peer hypermedia protocol designed to preserve and grow humanity's knowledge by making the web upgradeable, resilient, and more open.
- IRC (Internet Relay Chat) A chat protocol for the Internet.
- JavaEE (Enterprise Edition) A collection of specifications and implementations for the JavaEE platform.
- Java RMI Java's Remote Method Invocation distributed objects platform.
- Jetlang A high performance java threading library, based upon Retlang.
- Jini (Apache River) A programming model to build adaptive network systems that are scalable, evolvable, and flexible.
- Jolie A microservice/SOA programming language.
- JSON Web Token (JWT) An open, industry standard (RFC 7519) method for representing claims securely between two parties.
- Kubernetes An open source container orchestration engine for automating deployment, scaling, and management of containerized applications.
- Lawrence Livermore National Labs (LLNL) Software portal for projects built at LLNL.
- Message passing (reading) A mechanism of communication or control flow in which a block of data is explicitly constructed and communicated to the recipient.
- Metawidget A smart widget that populates itself, either statically or at runtime, with UI components to match the properties of your domain objects.
- Mistral An experimental language for distributed programming, with an emphasis on the specification and maintenance of a virtual overlay network.
- Mizu API traffic viewer for Kubernetes enabling you to view all API communication between microservices. Think TCPDump and Wireshark re-invented for Kubernetes
- MQTT A Client Server publish/subscribe messaging transport protocol.
- MQTTnet A high performance .NET library for MQTT based communication. It provides a MQTT client and a MQTT server (broker).
- N8N Node-based workflow automation tool for developers.
- Naked Objects Framework Turns a POCO domain model (that follows a few simple conventions) into a complete application.
- NATS A simple, secure and high performance open source messaging system for cloud native applications, IoT messaging, and microservices architectures.
- Nitric A framework for cloud and serverless apps.
- Nomad A simple and flexible scheduler and workload orchestrator to deploy and manage containers and non-containerized applications across on-prem and clouds at scale.
- Noms The versioned, forkable, syncable database.
- OASIS Advanced Message Queue Protocol (AMQP) The global standard enterprise messaging protocol.
- OAuth Specification for third-party authentication.
- OData OASIS standard for data over HTTP-based communication.
- Olin Olin is like the JVM for WebAssembly. It wraps WebAssembly with a set of functions to access the outside world and keeps track of things like how many instructions were used, how many syscalls were made and how much memory was used.
- Opa An attempt at an "all-in-one" presentation, middleware, and storage system.
- OpenAI Collection of AI services for natural language and images.
- Open API Specification (OAS) A standard, programming language-agnostic interface description for REST APIs, which allows both humans and computers to discover and understand the capabilities of a service without requiring access to source code, additional documentation, or inspection of network traffic.
- OpenCRUD A GraphQL CRUD API specification for databases.
- Open Grid Forum An open global community committed to driving the rapid evolution and adoption of modern advanced applied distributed computing, including cloud, grid and associated storage, networking and workflow methods. OGF is focused on developing and promoting innovative scalable techniques, applications and infrastructures to improve productivity in the enterprise and within the international research, science and business communities.
- Orbit Virtual actor framework for building distributed systems.
- Orleans Actors-based implementation for distributed systems implementation.
- Owin Open Web Interface for .NET.
- Patterns of Enterprise Application Architecture Martin Fowler's enterprise systems text.
- Pipedream Workflow automation and API integration platform.
- Polly A .NET resilience and transient-fault-handling library that allows developers to express policies such as Retry, Circuit Breaker, Timeout, Bulkhead Isolation, and Fallback in a fluent and thread-safe manner.
- Prefect "Build, run, and monitor data pipelines at scale." Use a flexible Python framework to easily combine tasks into workflows, then deploy, schedule, and monitor their execution through the Prefect UI or API.
- Present-RPC Simple, idiomatic RPCs for Java, Javascript, Android, iOS, and more
- Project Tye An experimental developer tool that makes developing, testing, and deploying microservices and distributed applications easier.
- Protocol Buffers a language-neutral, platform-neutral, extensible way of serializing structured data for use in communications protocols, data storage, and more.
- pSpaces Implementations of Tuple Spaces (aka object spaces) across several languages/platforms.
- PubNub Cloud platform for realtime message broadcast/consumption.
- Quick.Ref Cheat Sheets A collection of "cheat sheets"/quick-reference-guides for various tools and platforms and languages and ....
- Reactive Streams An initiative to provide a standard for asynchronous stream processing with non-blocking back pressure. This encompasses efforts aimed at runtime environments (JVM and JavaScript) as well as network protocols.
- Really Simple Syndication (RSS) XML-based format for syndicated sites.
- RealWorld Exemplary fullstack Medium.com clone powered by React, Angular, Node, Django, ... multiple front-ends, multiple back-ends, all operating off of the same API specification.
- Reaqtor Reaqtor is a framework for reliable, stateful, distributed, and scalable event processing based on Rx.
- Reflow A language and runtime for distributed, incremental data processing in the cloud.
- Replicache An in-browser persistent key-value store that is git-like under the hood.
- Representational State Transfer (REST) Collection of links and reading and notes on Fielding's dissertation.
- Resource Description Framework (RDF) A data format for Semantic Web involving triplets of data.
- Rest# (Rest-Sharp) HTTP client library for the CLR.
- REST Assured Java DSL for easy testing of REST services
- Restful Objects A generic way to expose a domain model through a REST (or more precisely, hypermedia) API.
- RevoltChat Open-source user-first chat service.
- RIFE2 A full-stack, no-declaration, framework to quickly and effortlessly create web applications with modern Java.
- RocketMQ A unified messaging engine, lightweight data processing platform.
- SAFE Stack Full-stack web development based on F# throughout each layer.
- Scapy Python-based interactive packet manipulation program & library.
- Scuttlebutt A decentralized platform.
- Scuttlebutt (protocol) How Scuttlebutt (a decentralized peer-to-peer protocol) peers find and talk to each other.
- Semantic Web A model of the web based more around its founder's original intent (hyperlinks and full-state "documents").
- Service Design Patterns Collection of patterns specific to services.
- Silk Markdown-based document-driven RESTful API testing.
- SimpleActors Wraps any Javascript object as an actor.
- Simple Binary Encoding (SBE) Binary codec for several languages.
- SOA Design Patterns A collection of patterns specific to service-orientation.
- SPARQL A query language designed for semantic web data retrieval and update.
- Spring Rod Johnson's "everything but EJB" application server based on dependency injection.
- Starcounter "Fused" ACID memory centric database engine and C# VM.
- Steeltoe An open source project that provides a collection of libraries that helps users build production-grade cloud-native applications using externalized configuration, service discovery, distributed tracing, application management, and more.
- Suave A simple web development F# library providing a lightweight web server and a set of combinators to manipulate route flow and task composition.
- Swarm JavaScript replicated model (M of MVC) library.
- Teach Yourself Computer Science A collection of links for learning CS for those who didn't study it at school.
- Temporal Workflows as code platform.
- The Grand Unified Programming Theory: The Pure Function Pipeline Data Flow with Principle-based Warehouse/Workshop Model The Pure Function Pipeline Data Flow with Principle-based Warehouse/Workshop Model.
- The Reactive Manifesto "We believe that a coherent approach to systems architecture is needed, and we believe that all necessary aspects are already recognised individually: we want systems that are Responsive, Resilient, Elastic and Message Driven. We call these Reactive Systems."
- Tiny Actor RunTime (tart) Tiny actor runtime.
- Tiny Actor RunTime in Javascript (tartjs) JavaScript implementation of Tiny Actor Run-Time.
- Titan A scalable graph database optimized for storing and querying graphs containing hundreds of billions of vertices and edges distributed across a multi-machine cluster.
- Tuple Space An implementation of the associative memory paradigm for parallel/distributed computing. It provides a repository of tuples that can be accessed concurrently; they may be thought of as "distributed shared memory".
- tyk Open source Enterprise API Gateway, supporting REST, GraphQL, TCP and gRPC protocols.
- UCI (Universal Chess Interface) An open communication protocol that enables chess engines to communicate with user interfaces.
- Ur (Ur/Web) A programming language in the tradition of ML and Haskell, but featuring a significantly richer type system. Ur is functional, pure, statically typed, and strict. Ur supports a powerful kind of metaprogramming based on row types.
- VSync A new option for cloud computing that can enable reliable, secure replication of data even in the highly elastic first-tier of the cloud.
- wasmCloud A distributed platform for writing portable business logic in Rust and compiled to WebAssembly.
- Weaver A scalable, fast, consistent graph store.
- WebDAV A open protocol for Distributed Authoring and Versioning over the Web (HTTP).
- WebIDL An interface description language (IDL) format for describing application programming interfaces (APIs) that are intended to be implemented in web browsers.
- WebSockets A computer communications protocol, providing full-duplex communication channels over a single TCP connection.
- World Wide Web Consortium (W3C) An international community that develops open standards to ensure the long-term growth of the Web.
- XMLHttpRequest Enables Web browsers to issue HTTP requests from within a page's lifecycle and consume the result.
- XOOM DDD naked objects-like platform for low-code to full-code reactive, event-driven apps and microservices.
- Yaade An open-source, self-hosted, collaborative API development environment.
- ZeroC Comprehensive RPC framework with support for C++, C#, Java, JavaScript, Python and more.
Last modified 14 April 2023