Dear World,
The time has come for Pharo 5.0!
Pharo 5.0 is our most significant release yet. Here are some highlights:
- The PharoVM is now based on Spur, the new memory management, and it brings with it a 35% speedup!
- A new unified foreign function interface (UFFI) replaced NativeBoost to provide a strong Spur-compatible framework for interfacing with the outside world.
- The Glamorous Toolkit now includes the GTDebugger to offer a moldable infrastructure that allows the developer to customise the debugger deeply.
- The underlying Reflectivity mechanism has reached maturity with multiple pieces coming together to empower developers to instrument their own systems. For example, we now have breakpoints implemented as just a simple extension of this mechanism.
- QualityAssistant is now part of the image to provide live feedback during development.
These are just the more prominent highlights, but the details are just as important. We have closed 2446 issues in Pharo 5.0. Take a moment to go through a more detailed recount of the progress.
While the technical improvements are significant, just think of getting 30% faster out-of-the-box, still the most impressive fact is that the new code that got in the main Pharo 5.0 image was contributed by 100 people. Together we have touched 43% of the classes, and 20% of the methods. The following visualisation rendered with Roassal in Pharo 5.0 is dedicated to this effort. The picture shows the touched classes and packages in gray, the authors and the links to the changed classes in red, and, using an automatically generated visual id, you can spot authors that have worked on similar projects.
Pharo is more than code. It is an exciting project involving energetic people. We thank all the contributors of this release:
Abdelghani Alidra, Clara Allende, David Allouche, Nicolas Anquetil, Thibault Arloing, Jean Baptiste Arnaud, Mangesh Bendre, Clement Bera, Alexandre Bergel, Torsten Bergmann, Usman Bhatti, Vincent Blondeau, Johan Brichau, Camillo Bruni, Miguel Campusano, Damien Cassou, Nicolas Cellier, Danny Chan, Andrei Chis, Christopher Coat, Ben Coman, Bernardo Contreras, Gabriel Omar Cotelli, Tommaso Dal Sasso, Paul De Bruicker, Sean De Nigris, Christophe Demarey, Simon Denier, Marcus Denker, Martin Dias, John Dougan, Stephane Ducasse, Stephan Eggermont, Johan Fabry, Sergio Fedi, Cyril Ferlicot, Holger Hans Peter Freyther, Joshua Gargus, Tudor Girba, Thierry Goubier, Kris Gybels, Norbert Hartl, Thomas Heniart, Dale Henrichs, Nicolai Hess, Alejandro Infante, Henrik Johansen, Goran Krampe, Pavel Krivanek, Juraj Kubelka, Denis Kudriashov, Matthieu Lacaton, Laurent Laffont, Kevin Lanvin, Jannik Laval, Alexander Lazarević, Skip Lentz, Max Leske, Dave Lewis, Esteban Lorenzano, Sheridan Mahoney, Mariano Martinez Peck, Max Mattone, John McIntosh, Rene Meusel, Eliot Miranda, Henrik Nergaard, Marion Noirbent, Merwan Ouddane, Nick Papoulias, Nicolas Passerini, Alain Plantec, Guillermo Polito, Damien Pollet, Baptiste Quide, Andreas Raab (RIP), Alain Rastoul, Stefan Reichhart, Lukas Renggli, Mark Rizun, Michael Rueger, Valentin Ryckewaert, Ronie Salgado, Udo Schneider, Boris Spasojevic, Igor Stasenko, Roger Stebler, Serge Stinckwich, Aliaksei Syrel, Camille Teruel, Pablo Tesone, Yuriy Tymchuk, Peter Uhnak, Masashi Umezawa, Dion Stewart, Sven Van Caekenberghe, Jan Van De Sandt, Benjamin Van Ryseghem, Toon Verwaest, Franck Warlouzet.
(If you contributed with Pharo 5.0 development in any way and we missed your name, please send us a mail and we will add you).
Enjoy!
The Pharo Team
Yesterday, I paired with Andrei and joined the hunt for leftover weak announcements that tend to accumulate in Pharo images since recently. This is a problem because it leads to an overall slowdown of the system. There were already several valuable points made in the long thread and there was even a fix readily available provided by Max Leske.
The main issue seemed to be that the weak announcements were not garbage collected because of a VM problem even if their subscribers were nil. However, there also seemed to be indications that the code related to the Glamorous Toolkit might have been responsible for this problem, so we wanted to understand the issue to see if there was anything we could do to help, especially given that the very problems in the inspector seemed to turn the inspector not so useful for identifying the problem. This turned out to be an exciting 30 minutes assessment session. Let me get you through it.
In a Pharo 50662, we started by inspecting:
SystemAnnouncer uniqueInstance
The inspector readily showed us the 2602 subscriptions. From the start we saw that there were many for which the subscribers are nil, a point also raised by Peter:
Another thing we noticed is that there seemed to be a repetition of announcements that appeared with nil subscribers. To check this, we executed:
grouped := (SystemAnnouncer uniqueInstance
subscriptions glmSubscriptions select: [ :each |
each subscriber isNil ])
groupedBy: [:each | each announcementClass name].Indeed, there were only 4 announcements that were affected by the problem:
More interestingly, we noticed that we had an equal amount of subscriptions for each announcement type (i.e, 640). This suggested that these registrations came from the same place. To browse this possibility, we looked for all methods that reference all these classes:
grouped keys allButFirst
inject: (SystemNavigation new
allReferencesTo: grouped keys first asClass binding)
into: [ :result :each |
result intersection: (SystemNavigation new
allReferencesTo: each asClass binding) ].We got 12 such methods:
We browsed a bit, and we saw that some of these methods registered other system announcements as well, such as ClassCommented, but those did not appear in our problem list:
Thus, this route did not seem to be the best way to understand the source of the problem. We turned a bit around and took another route. As, each subscription provides us the selector that is to be sent when the announcement gets triggered, we queried the system to give us those selectors:
groupedSelectors := grouped associations collect: [ :assoc |
assoc value collect: [:each | each action selector ] as: Set].In this case, too, we got for each distinct announcement exactly one selector.
So, we looked at all methods that send all of these selectors:
selectors := groupedSelectors flatCollect: #value.
selectors allButFirst
inject: (selectors first senders)
into: [ :result :each |
result intersection: each senders ].And we got exactly one method: PragmaCollector>>installSystemNotifications.
Essentially, this means that the responsible for creating the global subscriptions that should have been garbage collected was the PragmaCollector. Certainly, the problem was not related to the PragmaCollector, but we now knew that no code in the Glamorous Toolkit prevented these subscriptions to become garbage collected.
We manually applied the patch to finalize and eliminate these subscriptions:
(SystemAnnouncer uniqueInstance subscriptions glmSubscriptions copy select: [ :sub |
sub class = WeakAnnouncementSubscription and: [
sub subscriber isNil ]])
do: #finalize.
Smalltalk garbageCollect.We were not yet quite done because executing:
WeakAnnouncementSubscription allInstances select: [:sub | sub subscriber isNil]
told us that we have more than 2000 instances left to deal with. The first question we asked was where did all these come from. There needed to be some global variables that were keeping those around but we did not know which one given that SystemAnnouncer uniqueInstance was cleaned up. We applied the same type of analysis.
announcements := (WeakAnnouncementSubscription allInstances select: [ :each |
each subscriber isNil ])
asOrderedCollection groupedBy: [:each | each announcementClass ].Again, we found that all of the announcements are global announcers about the state of the system, and that each of them appeared in equal quantities.
Like in the previous case, we again looked at the place where all these announcements are sent from:
announcements keys allButFirst
inject: (SystemNavigation new
allReferencesTo: announcements keys first binding)
into: [ :result :each |
result intersection: (SystemNavigation new
allReferencesTo: each binding) ].We found 3 cases. The first two were using SystemAnnouncer uniqueInstance weak, so these could have not been the root. In the third one, we discovered a new announcer: SystemAnnouncer uniqueInstance private.
In the end, this reported problem was not related to GT. There were other problems related to the management of announcements, but those were other cases.
This session showed again how the quick connection between the runtime and code can dramatically speedup the amount and the depth of the questions we can ask our system. And this is critical for modern software engineering.
I will again be present at ArchConf. The spring edition is organized in San Diego during April 4-7. I will start with a 1 day tutorial on Steering agile architecture, and in the last day I will follow with four 90-minute talks on:
When I was at ArchConf 2015 in Fort Lauderdale, I had a conversation with Michael Carducci around the issue of how to solve real problems without reading code. As usual, I argued that code reading is the single most expensive activity in software development and that we ought to pay more attention to it.
This conversation appeared recently on the No Fluff Just Stuff podcast. I had a funny hoarse voice because I had a cold and the recording took place late in the evening after having given four long talks during that day.
On the Pharo mailing list there was a discussion about extracting sprites from the larger PNG files. The LPC Character Generation website offers an example of such a larger set of sprites organized in one large PNG. I take this opportunity to describe how using the GTInspector makes us quickly find the solution to our problem through continuous visual feedback.
Let’s start. First, we download and inspect a sprites.png file.
The preview tells us we have the correct file. Next, we load a form from the file.
self binaryReadStreamDo: [ :stream | PNGReadWriter formFromStream: stream ]
Now that we got the form loaded in memory, we want to split this form in smaller forms associated to individual sprites. To this end, we first need to learn the size of each sprite. One way to do that would be to count the amount of rows and columns in the matrix, but that is too boring. Instead, we explore a bit. Likely, the sprite has a power of 2 dimension. We try with 32x32.
self copy: (0@0 corner: 32@32)
Nah, that is not it. What about:
self copy: (0@0 corner: 64@64)
This looks much better. We verify that the height and width are multiple of 64:
self width / 64 “==>13"
self height / 64 “==>21"
Great. Now we can proceed with splitting the form:
sprites := OrderedCollection new.
0 to: (self width - 64) by: 64 do: [ :x |
0 to: (self height - 64) by: 64 do: [ :y |
sprites add: (self copy: (x@y corner: (x+64)@(y+64)))]].
sprites
And we are done. You can try this scenario with the latest Pharo.
