Humane assessment

Traditionally, the activity of assessment is dominated by either code reading, or by the use of standard reporting tools. Unfortunately, none of these work well in practice.

Software assessment needs rethinking.

The humane solution: Custom analyses for custom problems

The assessment process must be centered on our human needs and must match our human capabilities: We need to take the context of the problem into account, but we require automatic tools to scale.

The humane solution: craft custom tools to meet custom needs. This is the essence of humane assessment.

Humane.png

Software development is a multi-dimensional effort that makes it close to impossible to predict the exact problems that will occur in a particular system. We can reasonably predict the classes of problems, but not the actual problems.

Thus, tools that solely provide automatic recipes to be followed without any know-how are not useful. Instead, we need tool infrastructures that enable engineers to consider the context and program the actual tools.

This requires skill and dedicated know-how.

The humane assessment method

Through an assessment effort we aim to find answers that lead to decisions. To be able to find an answer, we first need to ask the question. The question reveals the problem. To be able to interpret the answer, we first need to formulate a hypothesis. The original hypothesis is the driving force behind the assessment effort.

Method.png

Once a hypothesis is made explicit, we want to analyze the data to ensure whether it is correct or not. Software systems are complex and they present plenty of context-specific variables. Even if a question looks standard, it most often will contain some contextual variation. Thus, it is likely that there will be no direct analysis that will provide directly the answer.

A custom problem requires a custom solution. To be effective, it is critical to craft an analysis tool for it. This is an activity that must be captured explicitly as part of the development process, and must be backed up by the skill of putting it in practice.

Regardless how smart an analysis is, it still only produces yet another set of data. It is the human that must interpret the results to decide what to do next. It is in this phase that the original hypothesis plays the crucial role: when the problem is clearly stated, the results can be interpreted in the context of this problem.

Assessment is rarely black and white. Most often it is about building confidence that the system is like we think it is so that we can make a decision. If the confidence is not there, perhaps the hypothesis needs to be refined, or perhaps a further analysis needs to be employed. Iterations and correlations are a must.

The end goal of assessment is decision making. If it does not lead to a decision, it is not useful. The process must end with a definite path of action.