Blog

Software assessment lecture at the University of Bern (2014)

On October 15, I gave a lecture at the University of Bern on software assessment. The lecture was split into three parts:

  • an introduction on why we need to pay serious attention to software assessment,
  • a Moose Jazz session showing multiple live case studies, and
  • a concluding session on why custom tools are essential in software engineering.

The most exciting part, at least for me, was the second one. Instead of offering recounts of case studies, I showed these cases live. The cases ranged from:

  • multiple queries on a Java system,
  • parsing and analyzing a proprietary scripting language,
  • spotting performance problems by analyzing log files,
  • reasoning about development issues by querying issue tracker, and
  • inspecting the file system.

All of these tools were developed using Moose and they were produced in a short amount of time.

On the one hand, it’s nice to be able to show solutions to real problems live using a mature tool produced mainly by researchers. On the other hand, the very fact that all of these problems were solved with one uniform set of mechanisms attest the fact that assessment is indeed a generic skill that once learnt can be utilized in many different contexts.

At the end, I remarked that assessment is not deemed an interesting enough subject of conversation to capture the attention of engineers even though it is the most expensive and pervasive activity in software development.

So, I asked the audience if they found the demos exciting. I did not ask about interesting, I asked if they were exciting. And they all said yes.

You see, assessment is exciting. It only takes us to look at it properly.

Posted by Tudor Girba at 26 October 2014, 8:50 pm with tags assessment, presentation, moose link
|

Glamorous Toolkit developments: endless GTPlayground memory, remote sharing, closeable panes and others

We, Alex Syrel, Andrei and I, are actively working on the Glamorous Toolkit.

First, together with the integration in Pharo 4, we have received a significant amount of feedback that we took into account. Second, we continued building new functionality.

Improvements

The contextual menu now contains multiple actions explicitly (including cut/copy/paste).

Cmd+o changed to become Cmd+g, and the action is now called Go. The reason for the change is that in Pharo, Cmd+o is already mapped on global composite shortcuts for opening various tools. There is still an ongoing discussion to change it again to simply be triggered via the Inspect when inside the Playground or Inspector.

The default object inspection presentation changed its title from State to Raw. It seems that State got people confused, and we found that Raw better denotes that the presentation is showing the raw object structure.

Close-able inspector panes

A feature that people asked for is the ability of closing panes in the inspector. Now, the Pager user interface offers the possibility of closing the last pane. To make it feel comfortable, we also added an animation of sliding the rest of the panes to the right when one closes. For this feature to happen, Glamour now supports one presentation specific action that is rendered in the title bar.

One question that might arise is why make this action available only on the last pane? First, we do not want to allow closing arbitrary panes. This would be a reasonable request if the panes would not be causally connected. But, the inspector keeps track of the exact inspection session and removing arbitrary steps would break this contract. Second, we could have the action of closing a pane from the middle of a session to mean removing all panes to the right. This might conceptually work, but it would be a surprising effect for a newcomer. As a consequence, we offer the possibility of closing the only last pane.

Shrinking tab labels

One concern about the design of the inspector is that the tabs rendering does not scale if we have too many presentations. To address this problem, Alex Syrel worked along two directions. On the one hand, tab labels shrink when there is not enough space.

On the other hand, shrunk tab labels expand on mouse hover so that you can still read the contents.

Of course, the same mechanism also works on the object title. There are still things to improve. For example, when a label is even larger than the available tab space, you will not see the complete. A solution here would be to have the printout be rendered as a popup right on top of the tab. Nevertheless, the current solution is already pretty workable.

Instant caching

Remembering code changes when the text changes. In the previous version, the current code was remembered only when pressing Cmd+g. However, the real goal is to not lose anything that is typed, not just executed. So, now, the playground remembers the code on every text change.

To support this feature, Glamour now offers the possibility of hooking a custom action when a port changes. A simple example looks like this:

GLMCompositePresentation new
   with: [ :composite |
        composite rubricText
          onChangeOfPort: #text act: [ :textPresentation |
             Transcript cr; cr; show: textPresentation text ] ];
   openOn: 'Type something and check the Transcript'

On disk playground caching

Still related to remembering code, up to now, the playground remembered the code in memory. This was nice, but as soon as you moved to another image, the code was not available anymore. Furthermore, if the image crashes (it typically does not, but once in a while it does) or you forget to save, you again loose the contents in the playground.

To solve the problem, the playground now saves/loads the content to/from disk. By default it works with a folder named ''play-cache'', but one can choose a custom one via a setting.

Playground remote sharing

A not so seldom use case is that of prototyping a code snippet and then sending it around via pastebin or a similar service. Best would be to service this use case directly in the Playground. *Sven created a shared workspace>http://ws.stfx.eu* a while ago, and we shamelessly replicated the client side behavior in the Playground. Triggering the ''save to cloud'' icon publishes the contents of the workspace and then conveniently copies the url in the clipboard.

These are a couple of new features that we worked on recently. More will come soon. Stay tuned. Actually, don’t stay. Better use the tools and give us feedback.

Posted by Tudor Girba at 24 October 2014, 11:50 pm with tags moose, pharo, tools, analysis link
|

Careful software

Apple creates cool products. Regardless of whether you use them or not, you just cannot ignore them.

Since quite a while, I made a hobby out of following the Apple public appearances (be them keynotes, interviews or otherwise) as I am interested in understanding what happens behind their thick curtains. In a recent interview, Jonathan Ive, Apple's chief designer, provided some hints about their design process. Among others, he talks about the carefulness in design. Specifically, he notices how we have this innate ability of detecting when a product was carefully designed and he goes on to say that in many cases the products on the market do not have this trace.

This concept is similar to what Christopher Alexander called the quality without the name. I love the concept and I am taken by its power.

On another track, since a couple of years, I noticed that Apple is exposing this carefulness in a peculiar way. Ever since the introduction of the MacBook Air, beside showing the how the hardware and software work together, the Apple keynotes almost invariably also show the inner makings of their hardware. This is particularly strange given that regular users have no chance of ever seeing that interior. Yet, they spend a significant part of their marketing effort in precisely showing the unseen by stressing the order and beauty of the mechanisms. It took me a while to figure it out, but I am not convinced that it is precisely the love for carefulness that makes this strange behavior meaningful.

They are certainly not alone in taking this approach. For example, in a beautiful TED talk, Richard Seymour tells a story of his grandfather that went to a watchmaker to have some work done on his watch. As the watchmaker opened the watch, the grandfather noticed that there is something engraved on the inner side of the watch. He prompted the watchmaker on the fact and asked him why he engraved it there given that nobody will ever see it. The watchmaker answered: God sees it!

The carefulness Mr. Ive is talking about brings with it a beautiful alignment between form and function. It’s as if a law of beauty make them come together harmoniously. This law can be a great guide during the design process. One can see they are proud of it. And so should they be.

But, this carefulness is not straightforward. It’s hard to do and it requires focus and dedication.

As I was pondering on this, something hit me: Why is it that Apple never shows the inner mechanisms of the software? Is software not a significant part of what makes Apple product valuable? Of course it is. Then, why?

Interesting enough, in the same interview Mr. Ive mentioned how they use the same kind of tables as in the shop to see how the products fit. Tools matter. So much so, that in another interview, he built live a custom made badge for the interviewer and he took the time to explain the beauty of the machine and of the process of crafting the object out of one metal piece.

Tools matter. Processes matter. And they have to be beautiful as well.

Should software not be subject to the same intensive care? I think it should be and I think it's mainly a matter of culture and of preoccupation.

How should we do it?

There is lots to say about the how. I will pick just one: the tools that are used for software development have to change radically. Just think of the interface. It’s primarily text. That is great when you want to write or read something, but it is terrible when you need to digest lots of data. And lots of data we have when we talk about millions of lines of code developed continuously by hundreds or thousands of developers. Exposing software as text does not benefit anyone because reading does not scale. It might be a simple solution, but it is not a careful one. We have to do better.

This is not a little thing either as developers spend statistically more than half of their time trying to understand existing systems. It’s essentially the single most significant software development activity. Yet, everyone, including Apple, is treating it in the same way we have done since decades: like a shapeless mass of necessary code that is unworthy of care.

This view is unsustainable. As we, as a society, rely more on more on software, we owe it to ourselves and to the future generations to make software sustainable. I argue that the problem is of environmental proportions because it affects the entire industry and we are drowning more and more into amorphous masses of code that is hard to understand and evolve. Like with any environmental problem, the solution is to go to the root and shift the system. We need to become as proud of the software sustainability and inner beauty as Apple designers are about the hardware. And just like those same Apple keynotes boasts about being environmentally friendly, we should look after the software environmental checklist as well.

We need to care.

Posted by Tudor Girba at 22 October 2014, 11:08 pm with tags assessment, environmentalism link
|

Creating custom browsers out of GTInspector extensions

The GTInspector empowers objects to display themselves through dedicated presentations. These are then wired together through the pager interface of the inspector. This provides a powerful way to experience and manipulate objects, but this is not the end of the story. Together with these dedicated presentations another opportunity opens up.

While the inspector is nice, it might still not always be desirable to use all the presentations. Let’s consider browsing the file system. Of course, we can use the default inspector, but if we are only interested in browsing files the relevant tabs are the one showing the items in a directory and the one showing the contents of a file. We can simply reuse the existing presentations and wire them up in a custom pager browser:

GLMPager new with: [ :pager |
     pager show: [ :composite :file |
          composite title: file basename.
          file gtInspectorItemsIn: composite.
          file gtInspectorContentsIn: composite ] ];
     openOn: FileSystem disk workingDirectory

The script simply delegates to the gtInspectorItemsIn: and gtInspectorContentsIn: methods already defined in the FileReference class. The result is a browser that looks like this:

Simple-file-pager.png

Quite simple. Now, let’s imagine a more complicated scenario in which you want to work on a Pillar book. As announced recently, the inspector offers dedicated presentations for Pillar.

Once you know that you will spend quite some time working with Pillar projects, you might prefer a dedicated browser that only exposes the presentations that you care about. We can extend the above browser to take these presentations into account.

GLMPager new with: [ :pager |
     pager show: [ :composite :file |
          composite title: file basename.
          file gtInspectorItemsIn: composite.
          file gtInspectorPillarConfigurationIn: composite.
          file gtInspectorPillarIn: composite.
          file gtInspectorPillarProjectIn: composite.
          file gtInspectorPngIn: composite.
          file gtInspectorGifIn: composite.
          file gtInspectorContentsIn: composite ] ];
     openOn: FileSystem disk workingDirectory

Once we were at it, we added also the PNG preview for good measure. What we get is a browser that looks like this.

Pillar-pager-pillar.png

Pillar-pager-png.png

Essentially, with the above, we have created a scoped GTInspector. However, we are not constrained to wire the presentations in a Pager. We can actually use any other Glamour browser we want. Let’s try another variation in which the file tree is shown to the left and the preview is shown to the right.

GLMTabulator new 
     with: [ :t |
          t title: 'File Browser'.
          t column: #directories; column: #preview.
          t transmit to: #directories; andShow: [ :a :root |
               a tree
                    title: root fullName;
                    display: { root };
                    format: #basename;
                    children: [:each | each isFile ifTrue: [{} ] ifFalse: [each children]]].
          t transmit from: #directories; to: #preview; andShow: [ :a :file |
               file gtInspectorPillarConfigurationIn: a.
               file gtInspectorPillarIn: a.
               file gtInspectorPillarProjectIn: a.
               file gtInspectorPngIn: a.
               file gtInspectorItemsIn: a.
               file gtInspectorContentsIn: a ].
          t transmit from: #preview port: #strongSelection; to: #directories port: #selection ];
     openOn: FileSystem disk workingDirectory

The script produces a browser that looks like in the pictures below. Most of the script is straightforward. In the first column we show the file system tree, and in the second column we show the contents of what is selected. Perhaps less obvious is the line at the very end that connects the strong selection port of the preview with the selection port of the directories. This is a little bonus to make it possible to double click on items displayed to the right and have them selected in the tree.

Tree-pillar-items.png

Tree-pillar-pillar.png

Tree-pillar-png.png

The GTInspector implements the philosophy of a moldable development environment in which tools should be customizable easily and extensively. On the one hand, the GTInspector makes it easy to create and expose multiple presentations for an object. On the other hand, these presentations are not captive in the inspector, but can be used in other contexts as well with little effort.

With the right infrastructure, investing in various custom presentations has more benefits than meets the eye at first sight. The interesting thing is that we can now easily imagine creating rather sophisticated tools with little code and that we only use for a very short time (e.g., minutes). This has a rather high disruption potential given that no other environment I know of allows for something like this.

Posted by Tudor Girba at 14 October 2014, 11:19 pm with tags tooling, moose, pharo, analysis link
|

Writing Pillar books with the GTInspector

Documentation is regarded as an important activity in the Pharo community. Besides commenting code, a significant documentation effort is being invested around books. For example, one of the ongoing such project is the Pharo for the Enterprise.

The book is written in Pillar, a wiki-like syntax extracted by Damien Cassou from the Pier content management system. The Pillar project offers transformers into several target formats such as LaTeX, html or markdown.

It’s quite a nice package. The only problem is that it offers no tool support for editing book contents, and as a consequence, people prefer using external editors that offer a minimal source highlighting. That is a no go for an environment that touts itself as being live.

I teamed up with Andrei and Jan to change this.

We needed several things to get this to work. First, we needed a text widget that can handle custom syntax highlighting. To this end, Andrei extended the Rubric text editor (authored by Alain Plantec) and its integration in Glamour to make it more suitable for the task.

Second, we needed the logic of producing the syntax highlighting. For this purpose we needed a parser. Pillar does come with its own hand-written parser, but the parser was only conceived for generating other formats and it does not come with any support for mapping parsed constructs to the original source. This is where the work on island parsing of Jan comes in. With an island parser we can match subsets of the input without needing to understand the whole input. This is particularly valuable when building a syntax highlighter. It’s a bit like regular expressions, only more powerful.

The end result looks like this:

Pillar-syntax.png

We get as you type highlighting for headers, links, embedded files, notes, code etc directly in the GTInspector for each file that has a pier or pillar extension. Adding new constructs is as easy as extending the parser. For example, the embedding rule is specified as:

GTPillarParser>>embed
     ^ '+' asParser , file token , '+' asParser ==> [:res |
          GTPillarFileHighlightingCommand new
               token: res second]

Granted, the parser is not fast. In fact it is more than an order of magnitude slower than the hand-made parser that comes with Pillar. It would ideal for the Pillar parser to keep track of tokens as well, but until then, the island parser is cheap and it works reasonably given that the highlighting happens in the background.

Perhaps this does not look like much, but this is the first time when we can have a cheap way to manage custom highlighter in Pharo. Yet, the highlighting was actually not the end goal of this exercise. Let’s put it in perspective for a second. Highlighting is a necessary step to bring the effort of writing Pillar-based documentation inside Pharo, but once this is achieved, the interesting things happen when we start to take advantage of the live environment.

For example, the syntax highlighting of Pharo code is done by simply delegating to the code highlighter (i.e., Shout). This is easily possible exactly because we are inside Pharo.

Other opportunities open up as well. For example, a Pillar book is made of multiple files (one per chapter) placed in subfolders of the root book folder. These must then be linked from a pillar.conf file that is placed in the root folder. Given that the logic is entirely in Pharo, when we inspect the root directory of a Pillar book, the inspector can offer us a Pillar specific index. For example, the picture below shows the folder of The Moose Book that I started to rewrite using Pillar. Selecting a file from the index previews the highlighted content to the right.

Pillar-book-index.png

Essentially, we get a cheap book editor. And when we are ready with editing, we can simply trigger the generation of the output directory from the inspector (see the play button on the top right).

One trouble when handling such books written in a language that is not visual is to manage the links to external pictures. To show the potential of a tight integration, the current solution comes with a simple tool support that provides a link for every embedded file (in our example, we have pictures/glamour-presentations-details.png). If the file cannot be found on the harddisk, the text becomes red. If the file is found on the harddisk, the text becomes a magenta link. Clicking on the link spawns the file object to the right offering a quick preview possibility.

Pillar-book-picture.png

To get this behavior, load the following code in a Moose 5.0 image, and inspect the directory of a Pillar book.

Gofer new
     smalltalkhubUser: 'Pier' project: 'Pillar';
     configuration;
     loadStable.
Gofer new
     smalltalkhubUser: 'JanKurs' project: 'PetitParser';
     configurationOf: #PetitParserIslands;
     load.
#ConfigurationOfPetitParserIslands asClass loadDevelopment.
Gofer new
     smalltalkhubUser: 'Moose' project: 'GToolkit';
     package: 'GT-InspectorExtensions-Pillar';
     load.

The current solution is an exercise that opens up new possibilities. An easy application is that of enhancing the rendering of class comments. For example, when loading the aforementioned packages, you also get the ability of rendering class comments with Pillar syntax.

Pillar-comment.png

But, we can envisage bolder directions as well. One clear direction is to go towards more live book writing. For example, if we stay in the image, we can also manage all code snippets in real methods instead of having them copied as text in a book. Pillar could simply provide a way to embed a method like +method:Object>>#printString+ and this would embed the method code while still allowing the writer to preview the code. Yet another thing is to move towards eliminating static pictures. For example, most pictures in a technical book are some sort of screenshots. These screenshots could easily be produced with snippets that could be generated. Thus, the book would only include a way to embed the result of a piece of code like +object:aScriptThatGeneratesAGraphicalForm+.

Yet another direction is to bring the books inside the image and link them to the code they talk about. Imagine that every packages comes with a dedicated chapter that offers scenarios of usages, and each scenario links explicitly to methods. In this case, when browsing the methods in question, you can get relevant scenarios to read about.

I am certain there are many other ideas that will flourish by bringing liveliness to documentation. We just have to be bold. I invite you to join us.

Posted by Tudor Girba at 7 September 2014, 3:27 pm with tags tools, moose, pharo, analysis link
|
<< 1 2 3 4 5 6 7 8 9 10 >>