Blog

Rethinking the Print It action in Pharo

In Smalltalk, "Print it" denotes the action of obtaining the textual representation of an object. It’s a great feature that can be used from any code editor to obtain a quick feedback.

The way it works goes like this. You select the text, then invoke print it (in Pharo, it works through Cmd+p), and you get the result as shown below.

Old-42.png

This feature exists since decades, and it is implemented exactly like this in all Smalltalk-related system I know. Andrei and I considered it's time to rethink it.

Let's take a closer look at it. The main problem with the user interface solution is that the result of the action influences the content of the code editor. Imagine you have the following script:

aNumber := 40 + 1.
theNumber := aNumber + 1.

Old-twoinstructions.png

Printing the first line, inserts 41 in the code editor. As a result, the editor is marked as dirty, and the second line becomes red because the newly introduced 41 changes the semantics of the code.

Old-twoinstructions-prinout.png

Actually, I have never encountered a case in which I wanted to get the printed text in the same code editor. Granted, the 41 text gets selected and this makes it easy to press Backspace and erase the text. However, if you press an arrow key instead, the selection gets lost and you are left with the tedious job of manually selecting the printout text.

To improve this situation, a different solution is be to not insert it in the same editor, but to pop it up in a separate morph.

Popper-twoinstructions-prinout.png

The original editor is left untouched. The popper morph takes the focus. When the printout is too large, the size of it can be contained and scrolled if needed. The popper is not modal. It goes away if you click anywhere else, or if you press Esc. And just to preserve the behavior that everyone else knows, it also goes away if you press Backspace as well.

Popper-longer-prinout.png

While solving the initial problem, the popper also comes with a couple of interesting side effects. First, if you want to select something, you can either select everything through Cmd+a or select parts of it like in any text readonly editor.

Popper-longer-prinout-selected.png

But, there is a more interesting feature. Every once in a while you get to a piece of code that takes long to run. Let’s say it's kind of like this:

[ (Delay forSeconds: 10) wait. 42 ] value

What you really want to do is inspect the result, you do it, wait for the 10 seconds, only afterwards to realize you pressed Cmd+p instead. You just do not want to wait again just to inspect the result. What you want is to just press Cmd+i and inspect the object you already obtained. The popper let's you do exactly that either through the keybinding or by pressing the associated inspect icon.

The popper implementation ships with the latest version of the Glamorous Toolkit.

Inserting the printout in the same editor probably made sense at the time of the implementation, but it does not have to be the solution anymore. This little feature might sound trivial, and in a sense it is. But, to me it exhibits the opportunities that await at any corner in our environment. We just have to look it again.

Posted by Tudor Girba at 28 July 2014, 1:19 am with tags assessment, moose, pharo, analysis, tooling link
|

Managing Morphic keybindings with GTInspector

Pharo offers a Keymapping framework (developed originally by Guillermo Polito for managing keybindings throughout the Morphic interface.

For example, defining an action bound to a keybinding is as elegant as:

Morph new
     bindKeyCombination: $, command toAction: [ self inform: 'Action!' ];
     openInWindow;
     takeKeyboardFocus

Pressing Cmd+, after executing the above code will trigger the action from the block.

This is a simple sample of how to add anonymous actions directly to a morph. But, the framework handles significantly more than that:

  • It allows you to specify shortcuts that are platform dependent,
  • It can handle global keybindings,
  • It deals with the propagation of keybindings, and
  • It lets you group your keybindings into categories.

The framework is now used everywhere in the Pharo environment. In my current image, doing:

KMKeymap allInstances size

... reveals that there are some 400 such registered keys. This means that when developing a Morphic interface, managing keybindings can be a problematic issue due to the interferences that can occur when defining the same keybinding for different actions. The GTInspector can help you in several ways.

There are at least two ways you want to look at how keybindings are associated to morphs:

  • understanding what keybindings exists overall and to what they are associated, and
  • understanding what keybindings exist for a specific morph.

Let’s look at all statically defined keybindings. They are held in the KMRepository default. Inspecting it offers us an overview of all the bindings. The presentation presents them as a table with shortcut, description, action and category columns:

Kmrepository.png

Given that there exist many such keybindings, to find your specific one you might want to search for it. There are two ways to filter the list. First, clicking on the category tag to the right shows only the keybindings from the respective category. Second, the input field at the bottom provides a custom search functionality. For example, below you can see the result of searching for Ctrl + D.

Kmrepository-filtering.png

A recurrent issue is to relate an installed keybinding to its definition in source code. For example, when selecting the Ctrl + D keybinding from RubSmalltalkEditor we get a KMKeymap that offers a presentation with the definition of the source code as well.

Keymap-source.png

The other way to relate to keybindings is from the point of view of a specific morph. For example, let’s look at a simple morph that defines a keybinding as in the first snippet from this post. Inspecting the morph reveals that the Cmd + , key is installed:

Morph-keys.png

The ability of inspecting the keys associated to a morph gets particularly relevant when reasoning about hierarchies of morphs. For example, when inspecting a RubScrolledTextMorph, we can find most of the relevant text editing keybindings associated with the RubEditingArea submorph.

Submorphs-keys.png

Keymapping is a nice framework, and now it is easily manageable from within the most basic development tool: the inspector.

Posted by Tudor Girba at 20 July 2014, 11:52 pm with tags assessment, moose, pharo, analysis, tooling link
|

Beacon: a slim announcement-based logging engine for Pharo

Logging is a pervasive analysis tool, but most logging systems focus on text. There is a historical reason for this, but we can certainly do better in this century. Particularly in Pharo, where everything is an object, it should follow that logs should be made of objects and that scrolling endlessly through large text files should make room for more sensible engineering approaches.

So, how would an object-oriented logging system look like?

The SystemLogger proposes a Pharo solution that focuses on objects. The central concept is the Log object which represents a single logging event. The event can be specialized via subclassing with various types of events. Similar to Toothpick, it features a central object that collects log objects, and has several concrete loggers that consume the log objects through various bindings such as the standard output or a database. The size is also rather tiny, the core containing some 535 lines of code.

However, there are still some lesser objects involved in the engine. Like in other logging frameworks, you can specify both levels of severity and tags that can be used for filtering. However, just like in other frameworks, these levels are numbers (even if encapsulated), and the tags are just strings.

At the same time, Sven described his thoughts about how a simple logging system could look like in Pharo. He talks about using event mechanisms that already exists in Pharo such as: Notifications and Announcements. Indeed, this makes plenty of sense. Of these two, I like Announcements more because they are meant to announce events from objects to objects.

So, I set myself to implement an engine based solely on Announcements. The goal was to get to pragmatic solution that should be usable in practice, yet be elegantly slim at the same time. After several iterations and a couple of feedback sessions with Sven and Andrei, the Beacon project was born:

Gofer new
     smalltalkhubUser: 'girba' project: 'Beacon';
     configuration;
     loadDevelopment

I know. Yet another logging framework! Just bare with me for a while. At the very least, it’s an exercise. And just to entice you: it has 200 lines of code. Let’s go.

The basic structure is similar to the SystemLogger. Objects log BeaconSignals that are captured by a receiving announcer. To make things easy, there exits a global announcer in the Beacon class. The beacon announcer does not do anything by itself. To actually produce an effect either on some sort of a stream like the Transcript, or on some sort of a recording support, it needs a dedicated BoundedBeacon.

Beacon.png

Take a look at some simple examples. In the first one, we record signal objects that are logged in between the starting and stopping of the recorder:

RecordingBeacon start.
CurrentStackSignal log.
RecordingBeacon stop.

In the the second one, we simply output to the transcript the textual representation of the signals logged within the scope of a

TranscriptBeacon runDuring: [
     StringSignal log: 'This is a message'. ]

Basic signals

Beacon works with Announcements. However, for logging purposes, we typically need a timestamp. To this end, BeaconSignal offers a handy Announcement that stores the current timestamp during the initialization.

The code also ships with three concrete signals for convenience:

  • StringSignal is used for simple string messages. This is used mostly for example purposes, and it is actually not meant to be used much in practice. Why not? Because it’s a string.
  • WrapperSignal is useful to log an arbitrary object without necessarily creating a class for it. Again, it is not meant to be used extensively, but can prove useful for quick prototypes.
  • CurrentStackSignal is a utility useful for capturing the current stack as objects (how cool is that?!).

Other than that, each application has to define its own, much like how it is supposed to define Exceptions and Announcements.

O yes. Another thing to remember is that each Signal is triggered using a log instance side message. That is it.

Basic bindings

To achieve anything useful, we need a BoundedBeacon to bind the general Beacon to a concrete medium, such as a stream. The core engine comes with a couple of concrete bounded beacons.

As seen in the previous examples, we have seen a RecordingBeacon that records the signals in an ordered collection, and a TranscriptBeacon that outputs the textual representation to the Transcript. Similarly, we can also output a textual representation to any custom string stream:

String streamContents: [ :stream |
     (CustomStringStreamBeacon with: stream)
          runDuring: [
               StringSignal log: 'This is a message' ] ]

An example of a custom binding to Fuel

A funny way of binding can be found in the FuelBeacon package from the same repository:

Gofer new
     smalltalkhubUser: 'girba' project: 'Beacon';
     package: 'FuelBeacon';
     load

As signals are objects, and as Fuel can store any such objects, it follows that we can simply combine the two and store a fuelized version of each signal.

FuelBeacon new runDuring: [
     StringSignal log: 'This is a message'.
     StringSignal log: 'This is another message’.
     StackSignal log ].

Each signal gets stored as a fuel file in a dedicated fuelbeacon folder. This is not particularly efficient and it should be used with caution because of two reasons:

  1. the creation of each file is not particularly fast, and
  2. serializing an object with Fuel will take with it all referenced objects, so each file size depends on the shape of your Signal. Thus, this can influence both the size and the speed of the logging.

Nevertheless, it does provide an example of how to create a custom BoundedBeacon.

No fancy text formatting

One common logging concept that did not make it in the engine is that of a formatter. This is typically an essential piece in text focused logging engines.

Perhaps there are cases in which it can still be useful for having special formatting on top of Beacon. If needed, most common of the formatting use cases can be achieved through concrete bounded beacons. For example, the StringStreamBeacon uses polymorphism to delegate the printing to the Signal hierarchy. This is enough.

Given that serialization should focus on objects, formatting text is neither essential nor general enough to warrant more prominent modeling. Once we have objects, we can approach presenting them through smarter tools like it can be seen below.

Inspecting beacon objects in the GTInspector

The Beacon comes with a dedicated set of extensions for GTInspector. In the picture below, you can see an inspector on an instance of RecordingBeacon. The list updates every time a new signal is recorded. By default, the latest one is shown on top, but the table is sortable. As you can see, in this example, we simulated 100k + 1 signals.

Recording.png

The CurrentStackSignal also comes with a dedicated presentation that shows the stack.

Stack.png

Filtering signal objects

Logging can quickly spawn a ton of data. Filtering is essential in this situation.

But, Beacon has no tags and no levels. Why not? Because these filtering methods pose a rather arbitrary constraint as they require us to know in advance at what level we want a certain logging event to be. They might work well in some situations, but in others they are more of a hindrance as often when using logging for debugging purposes, we mainly want to see only the events we nominally want and not others.

A approach for this problem is to take advantage of objects. Indeed, the Announcement framework allow us to register our interest for a set of announcement types of our choice. For example, the snippet below:

RecordingBeacon new
     runFor: StringSignal, StackSignal
     during: [
          StringSignal log: 'This should be recorded.'.
          StackSignal log.
          DummySignal new log ]

will only log the first two signals, but not the dummy one. For each application, we can define various combinations of arbitrary signals that should be logged together without having an extra need of levels and tags.

Of course, applications can still create their own specialized BoundedBeacons and introduce another level of filtering if it makes sense within that application. However, please note that the default filtering mechanism offers developers an incentive to design fine grained logging classes that can be easily picked up at a later time. That can only be for the better.

Filtering signal data

Another filtering issue is related to the data associated with a signal object. Let’s take a concrete example. The goal of StackSignal is to store the entire current stack. This in turn can lead to including an extensive object graph that can produce large files when stored using Fuel.

To limit the size, we use two mechanisms. First, the StackSignal does not store the actual context objects. Instead it only stores the references (i.e., Ring definitions) to the methods being present in the stack. Indeed, this is not quite a stack, but it is a reasonable approximation for the point of view of logging. A benefit of doing so, is that the logged objects become constant and thus they are better suited for longer term inspection (see also the thoughts of Sven).

Second, the FuelBeacon itself ensures that only a limited amount of stack items are being serialized. This decision is essentially dependent both on the FuelBeacon serialization strategy and on the StackSignal. To handle it, we need a double dispatch. For example, in our case, the FuelBeacon delegates to the Signal hierarchy the job of actual serialization via extensions:

Signal>>serializeInDirectory: aDirectoryReference
     ...
     FLSerializer
          serialize: self objectToSerialize
          toFileNamed: (aDirectoryReference / oneLinerAsFileName , 'fuel') fullName

And the StackSignal can specify the acceptable copy to be serialized:

StackSignal>>objectToSerialize
     | trimmedCopy |
     trimmedCopy := self copy.
     trimmedCopy
          instVarNamed: #stack
          put: (self stack first: 100).
     ^ trimmedCopy

This is but an example of a pattern, and the solution described above can be used in other cases as well.

Beyond global bindings

Up to now, we have mostly looked at global signals that are captured by a global Beacon and passed to other global BoundedBeacon objects. These globals make it cheap to add a new log entry and have it captured in a central place. Yet, the engine does not limit the usage to these globals. The last example from above shows that we can equally well collect the results through an instance of a BoundedBeacon (e.g., RecordingBeacon new).

This becomes interesting when we want to debug selectively only a dedicated scenario without having to dig through the entire log that might contain many irrelevant details. For example, while working with the Rubric text morph, we noticed that when the syntax highlighting is activated, it gets slow. At the same time, the same syntax highlighting engine does not cause a problem in the commonly used PluggableTextMorph. Thus, we wanted to understand what execution paths lead to the SHTextStyler>>style:. To this end, we added:

SHTextStyler>>style:
     StackSignal log.
     ...

And, after selecting a Rubric morph, we simulated the troubled scenario like this:

RecordingBeacon new runDuring: 
     [ ActiveHand simulateKeyStrokes: 'aa.' ]

Inspecting the resulting RecordingBeacon revealed that indeed, the style method is being used at every character change, while in the case of the PluggableTextMorph, it is triggered much more seldom. For this type of analysis it is critical to be able to collect only the signals we care about and only for the duration we care about.

Logging any announcements from any announcer

Not only can we use multiple instances for bindings, but we do not necessarily need the Beacon either. By using Announcements as the transmission engine, we can also log the transmissions of announcements from any announcer. To achieve this, we simply have to point a bounded beacon to the announcer. For example:

announcer := Announcer new.
b := RecordingBeacon for: announcer.
b 
     runFor: Announcement
     during: [ announcer announce: Announcement new ].

This feature is particularly interesting given that the Announcements framework is becoming the primary means of communication between decoupled objects in Pharo. As announcements tend to be hard to debug, logging the activity of an announcer inexpensively can save a lot of trouble.

Closing remarks

I was surprised by how little code I needed to write the engine. At first sight, the implementation can appear too simple. Indeed, it is based on a small amount of concepts. However, I believe that the examples above show that by combining these concepts, we can handle a wide range of classic requirements we might have for a logging system.

There is certainly room for improvements. One direction of development is to create more bindings to support various backends. Another direction is to support filtering based on the instance of a signal, and not only based on its type. Yet another one, is to work on how to effectively trim the stored data.

Some things in Beacon are happening in a radically different manner than what is considered the norm, but I think this is what makes it interesting. Filtering is simpler and more flexible without imposing restrictions deep in the framework. Textual formatters do not exist at all. Add to that the announcement debugging option and we get a different bread of an engine that is more versatile at less cost.

Posted by Tudor Girba at 15 June 2014, 2:48 pm with tags assessment, moose, pharo, analysis, tooling link
|

A beautifully odd method

Some pieces of code are just beautiful. One of my favorite methods from the past year is part of CodeCity:

CCCuboid>>verticesFor: anObject
     | xSize ySize zSize p1 p2 p3 p4 p5 p6 p7 p8 |
     xSize := self widthFor: anObject.
     ySize := self heightFor: anObject.
     zSize := self depthFor: anObject.
     p1 := CCPoint x: 0 y: 0 z: zSize.
     p2 := CCPoint x: xSize y: 0 z: zSize.
     p3 := CCPoint x: 0 y: ySize z: zSize.
     p4 := CCPoint x: xSize y: ySize z: zSize.
     p5 := CCPoint x: 0 y: 0 z: 0.
     p6 := CCPoint x: xSize y: 0 z: 0.
     p7 := CCPoint x: 0 y: ySize z: 0.
     p8 := CCPoint x: xSize y: ySize z: 0.
     ^ OrderedCollection new
          add: p1;
          add: p2;
          add: p3;
          add: p4;
          add: p5;
          add: p6;
          add: p7;
          add: p8;
          yourself 

No. It’s not a joke. Yes, I know that it has p1 ... p8 in there. And, I know it is difficult to figure out what it does exactly. Please indulge me for a little while and let’s take a closer look at it.

The method seems to build some points in a three dimensional space (due to the x:y:z: selector) that are related to a cuboid object. The only more meaningful names we have there are =xSize=, =ySize=, =zSize= which relate to dimensions, and these variables are used for constructing the p1 ... p8 points. Given that we have 8 points, and that a cuboid shape is made of 8 points, it likely means that we deal with a factory method that constructs the cuboid. The only problem is: which point corresponds to which point?

One approach to address this question is to name the variables like frontTopLeft and backBottomRight. That’s what the book would tell you. But, there is another way. The reason I like this method, is due to the comment that I intentionally omitted above. Take a look again.

CCCuboid>>verticesFor: anObject
“
        7 - - - 8
      / |     / |
     3 - - - 4  |
     |  5- - |- 6
     | /     | /
     1 - - - 2
“
     | xSize ySize zSize p1 p2 p3 p4 p5 p6 p7 p8 |
     xSize := self widthFor: anObject.
     ySize := self heightFor: anObject.
     zSize := self depthFor: anObject.
     p1 := CCPoint x: 0 y: 0 z: zSize.
     p2 := CCPoint x: xSize y: 0 z: zSize.
     p3 := CCPoint x: 0 y: ySize z: zSize.
     p4 := CCPoint x: xSize y: ySize z: zSize.
     p5 := CCPoint x: 0 y: 0 z: 0.
     p6 := CCPoint x: xSize y: 0 z: 0.
     p7 := CCPoint x: 0 y: ySize z: 0.
     p8 := CCPoint x: xSize y: ySize z: 0.
     ^ OrderedCollection new
          add: p1;
          add: p2;
          add: p3;
          add: p4;
          add: p5;
          add: p6;
          add: p7;
          add: p8;
          yourself

All of a sudden, the cryptic numbers have a clear meaning, don’t they? This is beautiful.

Working code is for the machine. Understandable code is for humans. The main responsibility of a developer is for the humans that consume that code, not for the machine. Achieving this requires imagination and creativity.

Posted by Tudor Girba at 27 May 2014, 7:38 pm with tags assessment link
|

Communicating the changes in Pharo 3.0

Pharo 3.0 is out.

This is the most significant release of Pharo so far. The momentum is growing and Pharo is improving on many fronts. To exhibit this, we decided to augment the textual announcement with a visualization.

Pharo-city.png

We wanted a visualization that is not only beautiful but that is also built with Pharo. The above picture is built with the most recent version of CodeCity built by Richard Wettel (if you want to try the visualization, you can download the image) that runs on top of Moose 5.0 and Pharo 3.0.

The goal of the visualization is to show the amount and spread of changes throughout the system. Specifically, every city district is a package, every building is a class, and the red bricks represent the modified methods in Pharo 3.0.

I was asked several times how I built it. Here is a short description that covers both the technical details and some design decisions. Let’s proceed.

First, we need to obtain all the changes that happened in Pharo 3.0. This has two sides.

Given that CodeCity adds extra packages that we do not want to take into account, we need to identify the packages that are only in Pharo 3.0. To this end, we use a heuristic and identify all packages that have associated the Pharo 3.0 Monticello repository.

packages := RPackageOrganizer default packages select: [ :package |
     MCWorkingCopy allManagers
          detect: [ :each | each packageName = package name ]
          ifOne: [ :mc | mc repositoryGroup repositories anySatisfy: [ :each | each location = 'http://smalltalkhub.com/mc/Pharo/Pharo30/main' ] ]
          ifNone: [ false ] ].

Next, we need to identify all methods defined in the above packages that have been modified after the start of Pharo 3.0. For this, we use another heuristic (I took this heuristic from Sven Van Caekenberghe) and simply compare the method modification timestamp with the date at which Pharo 3.0 was started.

all := IdentityDictionary new.
packages do: [ :package |
     package methodReferences do: [ :m |
          all at: m compiledMethod put:
               ([ '2013-03-18T00:00:00' asDateAndTime <=
                  (DateAndTime fromMethodTimeStamp: m compiledMethod timeStamp) ]
                         on: Error
                         do: [ false ]) ] ].

Given that comparing timestamps is time consuming and that this information will be used at rendering time, we store it in a dictionary to speed the visualization up.

Like any visualization engine in Moose, CodeCity offers a fluent API that is suitable for scripting. Using this API we can build nodes, specify shapes and define nesting (see below).

Talking about nesting, CodeCity is at its best when we get deep nesting. However, in Pharo, packages have almost no nesting. To improve the rendering, we can use another heuristic and split the package names by -. For example, something like AST-Interpreter-Core would produce 3 pseudo packages: AST, AST-Interpreter and AST-Interpreter-Core.

allPackageStrings := packages collectAsSet: #name.
allPackageStrings copy do: [ :each |
     dash := each lastIndexOf: $-.
     dash > 0 ifTrue: [
          allPackageStrings add: (each first: dash - 1) ] ].

We are now ready to tackle the actual visualization. The script below builds nodes for each packages, inside each of them, it build the nodes for classes, and inside each class, it builds the nodes for all its methods. Afterwards, it defines the nesting of packages based on string matching.

builder := CCBuilder new.
builder shape platform color: Color white.
builder
     nodes: allPackageStrings
     forEach: [ :eachPackageName |
          | package |
          package := RPackageOrganizer default packageNamed: eachPackageName 
                                               ifAbsent: [ RPackage new ].
          builder shape platform
               color: Color lightGray.
          builder nodes: package definedClasses forEach: [ :class |
               builder shape box
                    color: [:m |
                         (all at: m ifAbsent: [ false ])
                              ifTrue: [ Color red ]
                              ifFalse: [ Color lightGray ]].
               builder nodes: (class methods sorted: [:a :b |
                                     (all at: b ifAbsent: [false]) ]).
               builder wallLayout ].
          builder packingLayout innerGap: 20 asCCPoint ].
builder nest: allPackageStrings node: #yourself in: [ :each |
     dash := each lastIndexOf: $-.
     dash > 0 ifTrue: [ each first: dash - 1 ] ].
builder packingLayout innerGap: 40 asCCPoint.
builder

Another complication of the visualization is associated with positioning the changed methods. Because we want to convey the amount of changes that are touching the system, we need the changed methods to appear on top of the other ones so that the red is easily spottable when we look over the city.

The interesting side effect of this decision is that it works beautifully also when we use the cool feature of CodeCity to observe the city from on top (which in effect turns the 3D visualization into a treemap).

Pharo-city-top.png

Job done. Pharo 3.0 has changed a lot.

This is indeed a large script. However, if you look closely, you will see that most of the code is related to building data structures that are more suitable for the visualization (detecting packages, querying changed methods, manufacturing nesting) than it is about specifying the visualization. Furthermore, the goal here is not to produce a reusable piece of code, but to produce a visualization that can have impact.

The exercise took not more than 1 hour and it was not linear. For something like this, the interactivity offered by the GTPlayground is of critical importance. Due to the very short feedback loop, I managed to produce several intermediate visualizations, and focus most of my time tweaking the parameters until I could find a suitable balance between the accuracy of the visualization and the goal of it.

Playground-pharo-city.png

Posted by Tudor Girba at 18 May 2014, 9:18 am with tags assessment, moose, pharo, analysis, tooling, story, spike link
|
<< 1 2 3 4 5 6 7 8 9 10 >>