Using Metaphor to Express Architectural Constraints
You can get a lot of benefits from a consistent architectural style but it is difficult to ensure that the architectural style is not violated when the programming language does not enforce it. I'm hoping that a vivid system metaphor will help explain the style to people reading the code and making architectural violations feel wrong because they break the metaphor.
Communicating Sequential Processes
My latest project has some awkward concurrency issues. We're using a proprietary analytics library that is not thread safe, receive high-volume market data feeds through vendor APIs that call into our code from their own threads, and have to look up reference data from slow, remote services without delaying the processing of unrelated market data events, which must all be processed on one thread because that analytics library is not thread safe.
This has led us to structure the system as multiple event-driven, concurrent threads. The market data threads feed events to the core processing thread, which is the only thread allowed to call into the analytics library. The core processing thread does not perform any blocking I/O: if it needs more data to process an event, it emits an event requesting that data; other threads react to that event by loading the data and sending it to to the core thread in another event.
This style of program can be pretty confusing if you don't carefully keep track of the various state machines and the communication that can go on between them. To help manage these issues, we've adopted an architectural style inspired by the Conic programming language.
Conic
In the Conic language a program is organised into lightweight processes (known as task modules) that communicate by asynchronous message passing. In contrast to the Actor model, adopted by Erlang, processes do not address one another directly. A Conic process has exitports through which it sends messages and entryports through which it receives messages. At runtime, a processes exitports are connected to the entryports of other processes. The interface of a component is defined as a set of named, typed entry- and exitports.
Conic modules are composed into group module types that instantiate submodules, connect them together and expose some of their ports at the interface of the group module. From outside, there's no way to know if a module is a primitive task module or a composite group module.
A program is a group module that has no unsatisfied inputs or outputs.
Conic strictly separates behaviour and structure. A group module has no behaviour of its own. Its behaviour is only defined by the composition of component modules within it and the ports of its component modules that it exposes at its interface.
These constraints lead to an architecture that acts as a helpful conceptual framework in which to decompose complex concurrent behaviour into event-driven parts, or, conversely, to compose those parts into systems with sophisticated behaviour. Those parts and compositions of parts are easy to test (or even model check) because they are context independent with clearly defined inputs and outputs. It is easy to understand the system's architecture from the configuration code that defines the system. In fact, you can generate visualisations of the architecture, like the diagrams above, that have a one-to-one correspondence with the code. And the small set of primitives (modules, inputs, outputs & links) makes for a very simple meta-model that makes it easy to write reflective code that manipulates any module or system in a generic way: for example, to insert interceptors for management & monitoring or unit testing.
A Circuit Metaphor
In Conic this constraint is enforced by the language. However, our system is written in Java. There's nothing in the language to stop programmers adding behaviour to group modules and violating the model. We must adhere to the architectural style through convention.
The constraints imposed by the style are not really obvious from the terminology used by Conic: "task module", "group module", "entryport", "exitport", etc. When I was introduced to Conic, the style was described to me as "software integrated circuits", so I've used a circuit metaphor in the Java implementation to better express the important constraints. We define "components", rather than "modules", and compose components into "circuits" rather than "group modules". Components have inputs and outputs and a circuit "connects", rather than "links", outputs to inputs.
The circuit metaphor has guided the naming of other parts of the system. For example, we have JMX MBeans that install "wiretaps" between components to let us remotely monitor message flow.