Humane assessment is a versatile and generic method. That is both a strength and a weakness. The strength is that it can be applied in many situations. The weakness is that it is perceived to be too distant from concrete scenarios that development teams face.
To bridge the perception gap, I have started to look since a couple of years for classes of problems that are concrete enough to be directly applicable. The first one is that of steering agile architecture for which I designed a 2-day course.
The concept of architecture started to regain traction recently. For example, two new industrial conferences were launched recently on the subject [1] [2]. That’s a good thing.
However, there are still quite some open questions as to how architecture fits within agile development. Of course, the main focus still tends to be placed on how to create architecture, and less about how to deal with the existing one. Humane assessment fills the latter space quite well. Specifically, the daily assessment process helps teams to make architecture concerns explicit and keep track of what goes on in the system. That is why the course is not about good or bad architecture. It’s about knowing the real architecture of your software system, and choosing to how to steer based on that reality.
If you are interested in working with me, please drop me an email.
Here is a teaser video.
This slideshow offers a visual argument, based on the System Attraction visualization, of how software systems have emergent structures.
This slideshow strengthens the argument I made previously that the evolution of software architecture is a complex game, and it offers the motivation for why the daily assessment process is needed in practice.
Architecture is an emergent property, and like any respectable emergent property it is hard to communicate.
Visualization is the best medium I know for this purpose. However, I could not find a visualization that serves the purpose properly. There are many ways to visualize a software system, and there are many tools that do it. A common way of representing software systems is by displaying it as a graph. The graph is typically laid out to exhibit a certain desired structure such as inheritance or containment. These are great tools when looking at those aspects.
However, in my experiment, I wanted to show structure as an emerging property rather than impose a certain view on the system. To this end, I asked myself "where does code want to be?". Extrapolating object-oriented principles, I wanted to let the code go both where it is needed and next to what it needs.
Designing a visualization with this purpose in mind turned out to be rather straightforward. I simply displayed all classes, methods and attributes as nodes and inheritance, containment, invocations and accesses as edges, and I used a force based layout to arrange the graph. The result is a visualization as can be seen below. I named it the System Attraction and the implementation can be found in Moose.
What can you see here?
Classes are black. Methods are red. Attributes are blue. Method containment, attribute containment, and class inheritance are gray. Invocations are red. And accesses are blue.
While things might appear random at first, at a deeper look there is a certain order that emerges from all the entangling. You might notice that colors are not uniformly distributed throughout the space. At the center, there appears to be much agglomeration of red surrounded by blue. This seems to indicate that there exist a consolidated behavior at the center of the system. There are groups of black spots (i.e., classes) that want to be close together. Also, the blue backbone on the bottom right seems to indicate a significant aggregation of related data.
To get a better feeling for how systems can be different, let's compare the above example with the one below. The two systems are comparable in size, yet, they exhibit much different patterns. The second system is much stronger on state. In fact, it looks as if it's the data that drives the system, and methods are just adjacent. Furthermore, while the first system seems to have at least one strong agglomeration of behavior, the second one has multiple small ones.
These picture exhibit macro emergent patterns that are determined by forces in the code. For example, a method is indeed attached to its classes, but if there are many other external interactions, the method will tend to go away from its class. The visualization lets the sum of all these forces decide the shape of the system.
Few visualizations offer a meaningful outcome both when applied in the large and when used in the small. System Attraction is an experiment that aims to offer value at several granularity levels.
For example, look at the picture below showing a visualization applied on one single class.
This class has essentially an equal amount of attributes and methods. What's more, one method seems to be particularly lonely. It's as if, it does not want to be a method at all. Guess what. Here is the code:
public void actionPerformed(ActionEvent ae) {
// ignored
}Let's compare this class with another one. This class has a more classic structure with multiple methods and few attributes. The methods seem to collaborate with one another and the attribute usage is reasonably distributed throughout the methods. It looks like a tidy class with proper structure in place.
If you investigate the class through the lenses of a classic visualization, such as a class blueprint, you can indeed see a rather clean structure.
Let's add to our scene a collaborator class next to the original one.
The class changes shape a bit. Let's add another one. The shape changes a bit more.
Now, let's see what happens when we visualize this class together with all its provider and client classes.
Ahh, where is our class now? The tiny tidy shape cannot be seen anymore. Let us highlight our class.
When placed next to its collaborators, the class changes shape dramatically. Regularity is secondary to distinct interests. For example, a group of four methods wants to be the center of everyone's attention, while some other methods want to mingle with other classes in the north.
Software architecture is an emergent property. Clearly, this shape of the system does not reflect any traditional view on architecture. There are no model-view-controller or layer patterns to be found. Nevertheless, patters do emerge. And what is architecture if not the structure that defines the system?
Indeed, when we look at the code only through a diagram that imposes a predefined structure, such as a UML diagram, we will tend to see only the defined regularities. These visualizations are not bad. They have their place precisely when it is those regularities we want to reason about. But, only being exposed to rigid shapes encourages a type of thinking that leads to the illusion of complete controllability.
System Attraction is an exercise that offers a radically different view on a software system. By not imposing a specific structure, it enables hidden patterns to emerge.
A common point of view says that architecture is only to be seen in the large. Yet, the forces embedded in the code influence the shape of the system in the small as well. Even if it is more comfortable to see a class as having a regular and controllable shape, when seen in its usage environment, the shape can morph in unexpected ways.
In our exercise we only looked at simple code elements such as classes and methods and standard interactions such as inheritance and invocations. We completely ignored technology specific details. And we completely ignored the social behavior around the software system. Even so, the experiment shows that software architecture is driven by interaction and it never is single minded. Only by acknowledging the underlying forces in a software system can you handle them appropriately.
Is this the right view for acting on a software system? Likely not. But, the goal of the exercise is to help communicate the emergent nature of software architecture. Why is it important to communicate it? Because we need to move from a controlling model towards a distributed one in which everyone is empowered to contribute.
If you pay attention to complexity theorists you might notice that they start by emphasizing the difference between various classes of problems. For example, the Cynefin framework distinguishes between simple, complicated, complex, chaotic (there is also disorder, but let's leave it for another time).
The typical issues in software development oscillate between the complicated and complex domains. On the one hand, a complicated problem is a deterministic one. In a complicated domain, you detect problems, devise analyses for them and extract precise answers that you apply directly.
One the other hand, a complex problem is one that is not necessarily repeatable, and it is the result of multiple agents interacting with each other. In a complex domain, it is essentially impossible to predict outcomes precisely. Instead you start from a hypothesis that seems to make sense, you check it to build an empirically verified theory, and then you react.
Complex is not necessarily worse than complicated. Neither is the other way around. However, it is important to distinguish between them because otherwise you risk of approaching a complex problem through a complicated mindset.
Controlling software architecture has long been approached as a command and control type of problem: the architect lays out a master plan at the beginning, and development follows it in every detail. In the best case, some adjustments are allowed during development, but they are always subject of architect's approval. According to this view, architecture is a deterministic artifact and it approached from the point of view of a complicated mindset.
A different point of view says that architecture is not a picture, but the structure defined in the code. The current structure of the code is the result of multiple forces. Multiple developers work at the same time in different corners of the system in order to build new functionalities. Developers work together with product owners, testers, designers and other stakeholders and the system is shaped as a result of this interaction - often relying on social behavior rather than on rigorous planning. Furthermore, the actual technical solutions are not built in vacuum, but they are also subject to the constraints posed by the existing state of the system, and by the underlying technologies and frameworks used in the system.
In other words, architecture is an emergent property produced by multiple agents interacting constantly with each other. This places architecture in the complexity realm.
This suggests that approaching architecture through a prescribed ordered (or even worse, simple) lenses can have a limited impact. More suitable is to rely on setting boundaries and using attractors. For an entertaining brief explanation of how to steer emergent behavior, listen to Dave Snowden's comparison of the different ways of organizing a children's party.
A key element of this approach is that neither the boundaries nor the attractors are static. Instead, they are to be reshaped constantly in order to achieve desired emergent properties.
Daily assessment offers a game for steering the architecture. The game combines social interactions with hard checks against the actual data. Architecture is constrained through an explicit contract between all players. This contract changes over time through social interactions and common decisions. No single person is in control, yet the situation is controllable.
An interesting effect of this approach is that if you want to control the quality of the architecture, you focus less on what exists in the system at any given time, and more on the types of problems that are being explicitly looked at. For example, as long as developers only look for granular issues related to low level technological issues (such as memory leaks), you should not be too confident on the overall architecture. Once high level constraints are placed on dependencies, or on domain-specific patterns, your confidence level should raise.
In this game, the current state is less important. The level of the game matters more.
Contextual constraints are hard to predict. It is tempting to think that a couple of generic rules will suffice to handle all future programming cases, but in practice they often fall short.
For example, programming languages come with various mechanisms to manage code visibility. Smalltalk makes it easy: state is protected, methods are public. Java takes "the more the better" approach and offers an enhanced set of language constraints: private, public, protected, and package.
Are these enough? Sometimes yes, but most often not. For example, in a Java system we wanted to ensure that a certain part of the model remains immutable to most of the system, but not to a selected set of factory classes. In particular, we needed to ensure that a set of classes, all inheriting from the ImmutableConcept, does not get instantiated outside of those factories.
One possibility would have been to make the constructors have package visibility and place all factories in the same package. This would have solved the immediate technical problem, but it did not match our constraints given that we needed the factories to be packaged in the plugins, not in the model. Thus, we had to let the constructors be public, and we documented that our ImmutableConcept classes should only be instantiated by dedicated factory classes that were annotated with @ImmutableConceptFactory.
The documentation even provided an example of what is allowed. The example looked somewhat like this:
@ImmutableConceptFactory
public class AConceptFactory {
...
ImmutableConcept concept = new ConcreteImmutableConcept();
}But because we could not sleep at night with a rule that only lives in the documentation, we had to craft a checker for it:
| immutables targetStructures |
immutables := (model allTypes detect: [:each | 'ImmutableConcept' = each name ]) withSubclassHierarchy.
targetVariables := immutables flatCollect: #structuresWithDeclaredType.
(targetVariables flatCollect: #incomingAccesses) select: [:each |
(each typeScope isAnnotatedWith: 'ImmutableConceptFactory') not ]
Essentially, the checker had to identify all write accesses to variables that were declared of type ImmutableConcept, and ensure that they come from within classes that were annotated explicitly with @ImmutableConceptFactory.
Once the checker crafted, it was integrated in the daily assessment process and we ensured that people do not get astray in the future.
Documentation is great for letting know people why the system looks the way it does. But, documentation alone will not ensure the cohesiveness of the system regardless of how disciplined you want to be.
