Blog

Writing Pillar books with the GTInspector

Documentation is regarded as an important activity in the Pharo community. Besides commenting code, a significant documentation effort is being invested around books. For example, one of the ongoing such project is the Pharo for the Enterprise.

The book is written in Pillar, a wiki-like syntax extracted by Damien Cassou from the Pier content management system. The Pillar project offers transformers into several target formats such as LaTeX, html or markdown.

It’s quite a nice package. The only problem is that it offers no tool support for editing book contents, and as a consequence, people prefer using external editors that offer a minimal source highlighting. That is a no go for an environment that touts itself as being live.

I teamed up with Andrei and Jan to change this.

We needed several things to get this to work. First, we needed a text widget that can handle custom syntax highlighting. To this end, Andrei extended the Rubric text editor (authored by Alain Plantec) and its integration in Glamour to make it more suitable for the task.

Second, we needed the logic of producing the syntax highlighting. For this purpose we needed a parser. Pillar does come with its own hand-written parser, but the parser was only conceived for generating other formats and it does not come with any support for mapping parsed constructs to the original source. This is where the work on island parsing of Jan comes in. With an island parser we can match subsets of the input without needing to understand the whole input. This is particularly valuable when building a syntax highlighter. It’s a bit like regular expressions, only more powerful.

The end result looks like this:

Pillar-syntax.png

We get as you type highlighting for headers, links, embedded files, notes, code etc directly in the GTInspector for each file that has a pier or pillar extension. Adding new constructs is as easy as extending the parser. For example, the embedding rule is specified as:

GTPillarParser>>embed
     ^ '+' asParser , file token , '+' asParser ==> [:res |
          GTPillarFileHighlightingCommand new
               token: res second]

Granted, the parser is not fast. In fact it is more than an order of magnitude slower than the hand-made parser that comes with Pillar. It would ideal for the Pillar parser to keep track of tokens as well, but until then, the island parser is cheap and it works reasonably given that the highlighting happens in the background.

Perhaps this does not look like much, but this is the first time when we can have a cheap way to manage custom highlighter in Pharo. Yet, the highlighting was actually not the end goal of this exercise. Let’s put it in perspective for a second. Highlighting is a necessary step to bring the effort of writing Pillar-based documentation inside Pharo, but once this is achieved, the interesting things happen when we start to take advantage of the live environment.

For example, the syntax highlighting of Pharo code is done by simply delegating to the code highlighter (i.e., Shout). This is easily possible exactly because we are inside Pharo.

Other opportunities open up as well. For example, a Pillar book is made of multiple files (one per chapter) placed in subfolders of the root book folder. These must then be linked from a pillar.conf file that is placed in the root folder. Given that the logic is entirely in Pharo, when we inspect the root directory of a Pillar book, the inspector can offer us a Pillar specific index. For example, the picture below shows the folder of The Moose Book that I started to rewrite using Pillar. Selecting a file from the index previews the highlighted content to the right.

Pillar-book-index.png

Essentially, we get a cheap book editor. And when we are ready with editing, we can simply trigger the generation of the output directory from the inspector (see the play button on the top right).

One trouble when handling such books written in a language that is not visual is to manage the links to external pictures. To show the potential of a tight integration, the current solution comes with a simple tool support that provides a link for every embedded file (in our example, we have pictures/glamour-presentations-details.png). If the file cannot be found on the harddisk, the text becomes red. If the file is found on the harddisk, the text becomes a magenta link. Clicking on the link spawns the file object to the right offering a quick preview possibility.

Pillar-book-picture.png

To get this behavior, load the following code in a Moose 5.0 image, and inspect the directory of a Pillar book.

Gofer new
     smalltalkhubUser: 'Pier' project: 'Pillar';
     configuration;
     loadStable.
Gofer new
     smalltalkhubUser: 'JanKurs' project: 'PetitParser';
     configurationOf: #PetitParserIslands;
     load.
#ConfigurationOfPetitParserIslands asClass loadDevelopment.
Gofer new
     smalltalkhubUser: 'Moose' project: 'GToolkit';
     package: 'GT-InspectorExtensions-Pillar';
     load.

The current solution is an exercise that opens up new possibilities. An easy application is that of enhancing the rendering of class comments. For example, when loading the aforementioned packages, you also get the ability of rendering class comments with Pillar syntax.

Pillar-comment.png

But, we can envisage bolder directions as well. One clear direction is to go towards more live book writing. For example, if we stay in the image, we can also manage all code snippets in real methods instead of having them copied as text in a book. Pillar could simply provide a way to embed a method like +method:Object>>#printString+ and this would embed the method code while still allowing the writer to preview the code. Yet another thing is to move towards eliminating static pictures. For example, most pictures in a technical book are some sort of screenshots. These screenshots could easily be produced with snippets that could be generated. Thus, the book would only include a way to embed the result of a piece of code like +object:aScriptThatGeneratesAGraphicalForm+.

Yet another direction is to bring the books inside the image and link them to the code they talk about. Imagine that every packages comes with a dedicated chapter that offers scenarios of usages, and each scenario links explicitly to methods. In this case, when browsing the methods in question, you can get relevant scenarios to read about.

I am certain there are many other ideas that will flourish by bringing liveliness to documentation. We just have to be bold. I invite you to join us.

Posted by Tudor Girba at 7 September 2014, 3:27 pm with tags tools, moose, pharo, analysis link
|

GTinspector received the 1st prize at the ESUG Innovation Awards

The GTInspector, part of the Glamorous Toolkit authored by Andrei, Alex Syrel and myself, received the 1st prize at the ESUG 2014 Innovation Awards.

These awards work like this: on the first day we get a 2-3 hours slot to demo in the open hall, and at the end of it the participants vote. It was great fun to demo the inspector. I alone gave some dozens of demos both on that day and afterwards. I can now do it with my eyes closed.

The work was met with quite some enthusiasm. Already during the conference several people extended the inspector for their own contexts, and we even had a couple of presenters showing their work through inspector extensions created during the week of the conference.

This is highly encouraging, especially keeping in mind that we just got started.

And yes, the trailer did help spreading the word:

Posted by Tudor Girba at 24 August 2014, 11:27 pm with tags moose, pharo, tools, analysis link
|

Moose: how to solve real problems without reading code (video from ESUG 2014)

I gave the above talk at ESUG 2014. The abstract went as follows.

Moose is a platform for software and data analysis (http://moosetechnology.org). It runs on Pharo and it can help you figure out problems around software systems.

In this talk, I show several real-life examples of how custom tools built on top of Moose helped solve concrete problems. The examples vary both in scope and in the kind of problems. For example, we talk about how we fixed a caching problem in a Java system by analyzing logs, or how we fixed a Morphic problem by means of visualization and interaction. Even if these problems are so different, all of them were solvable with one uniform set of programmable tools.

That is the power of Moose, and it is now at the fingertips of any Pharo programmer.

Posted by Tudor Girba at 23 August 2014, 9:24 pm with tags moose, pharo, tools, analysis link
|

gt.moosetechnology.org & GTInspector trailer

As part of the submission for the ESUG 2014 Innovation Awards, we launched a webpage dedicated to the Glamorous Toolkit for Pharo at gt.moosetechnology.org.

And, to make it more exciting, we put together a trailer to exhibit the power of the inspector and the playground.

Posted by Tudor Girba at 15 August 2014, 1:34 pm with tags moose, pharo, tooling, assessment link
|

My NDC 2014 talk on Pharo: Playing with Live Objects is now available (video)

The talk I gave at NDC on "Pharo: Playing with Live Objects" is now publicly available.

On the one hand, the talk shows how programming with Pharo can be both serious and fun at the same time. On the other hand, the talk exhibits the liveliness of Pharo through demos of the GTInspector.

Posted by Tudor Girba at 13 August 2014, 7:11 pm with tags presentation, moose, pharo, analysis link
|
<< 1 2 3 4 5 6 7 8 9 10 >>