Blog

Introducing PetitParser2

I am happy to announce the availability of PetitParser2, a redesign of the original PetitParser developed by Jan Kurš. The version is already present in the latest Moose 6.1 development image.

The main goals of the redesign is performance and flexibility. The speedup ranges between 2-5x. This is achieved through the idea of a kind of a parser compiler that performs a static analysis of the parse tree to identify patterns that can be optimized and then changes the actual parser execution.

The original implementation of the compiler was actually implemented in PetitParser. However, the problem was that the resulting parser was not easy to understand. This was the main reason that prompted the redesign of PetitParser. In PetitParser2 the concept of a Parser is split in two:

  • Nodes: These essentially provides only the AST of the parser, but without actual logic inside. They mirror the previous PPParser classes.
  • Visitors. These provide the actual parsing behavior. Optimizations are implemented in this hierarchy.

For a typical external user, PetitParser2 has the same structure as PetitParser. The main difference is that asParser is changed to asPParser. For example, the following original snippet:

#letter asParser, #any asParser star

Becomes:

#letter asPParser, #any asPParser star

In this way, the two versions of PetitParser can co-exist in the same image to ease the transition. This transition is needed for parsers that rely on deeper constructs from the original PetitParser, such as concrete names of classes.

Another new feature is the availability of streams. The original PetitParser required the whole string to be present in memory to allow for indefinite backtracking. PetitParser2 offers stream utilities that apply parsers on a stream by keeping a buffer in memory for backtracking purposes.

Of course, the new version comes with all development tools that we got used with PetitParser. The nice thing is that the tools work even when we are using the optimized version of a parser.

For example, the picture below shows the PetitParser browser opened on the Smalltalk parser.

Pp2-default.png

The one addition is a larger run button which performs the optimized parsing. Triggering it for our case renders the result as shown below. The only difference is that in the above version, there were 907 individual parsers involved, while the optimized version only used 258 of them.

Pp2-optimized.png

To get a better feeling of how the magic happens, let’s look at a small sequence parser.

#any asPParser star

Inspecting the parser allows us to play with the sampler.

Pp2-simple-default.png

The unoptimized version of the parser triggers independent parsers for each of the letters, and then concatenates the results in another collection. However, triggering the optimized version only involves one parser that knows how to consume the entire sequence in one collection.

Pp2-optimized-default.png

All in all, PetitParser2 is an exciting development. Take a look at let us know what you think.

Posted by Tudor Girba at 8 November 2016, 5:07 pm with tags moose, tooling link
|

Glamorous Toolkit at ESUG 2016 (video)

At ESUG, I gave a talk about the Glamorous Toolkit, and the talk is now online. We have announced the Glamorous Toolkit project two years ago at the same conference. On that occasion I argued that IDEs should have different properties than what typical IDEs have to offer today. Over the last two years, we introduced 4 significant tools inside Pharo, namely Inspector, Playground, Spotter, and Debugger, and in this talk I focused on how these tools can affect the way you think about programs and programming.

Please take a look, and let us know what you think.

Posted by Tudor Girba at 28 September 2016, 7:57 am with tags gt, tooling, presentation link
|

Lam Research evaluates Pharo

We are very happy to make the following announcement:

Lam Research, a leading supplier of wafer fabrication equipment and services to the global semiconductor industry, is an experienced user of the Smalltalk programming language. Smalltalk is a key component in Lam's software control system for a broad range of the equipment it manufactures. Tudor Girba is a leading member of the tools and environment development effort in Pharo, having architected the Glamorous Toolkit for live programming. Eliot Miranda is author of the Cog virtual machine that underlies Pharo and other Smalltalk dialects.

Lam has engaged Tudor and Eliot to explore potential enhancements in Lam's use of Smalltalk. These enhancements range from running highly optimized Smalltalk on low cost, single board computers, to enhancing Lam's Smalltalk development practices with state-of-the-art live programming. During the engagement, Tudor and Eliot successfully moved a key communication component of the control system to Pharo. It was a challenging task aimed at extending the reach of Lam’s system to the Pharo world including the option of executing on ARM processors.

Cheers,
Tudor Girba, Eliot Miranda and Chris Thorgrimsson

Posted by Tudor Girba at 25 August 2016, 11:16 am link
|

Moose 6.0

We are happy to announce version 6.0 of the Moose Suite, the platform for software and data analysis built in Pharo:
http://moosetechnology.org/#install

Moose-6-0.png

Description

The key highlights are:

  • It is based on Pharo 5.0 including the latest version of the Glamorous Toolkit.
  • It includes the SmaCC parsing framework together with parsers and abstract syntax trees for Java, JavaScript and Swift.
  • Roassal2 comes with several enhancements.
  • Famix features a new and generic query API engine.
  • Moose Finder and GTInspector come with more custom presentations and visualizations.
  • SmaCC comes with a dedicated debugger.
  • The debuggers for Glamour, PetitParser and Announcements received a new update.
  • DeepTraverser is an order of magnitude faster.

Extra highlights:

Installation

The Moose Suite 6.0 comes for each platform as a separate bundle:

The Moose Suite 6.0 can also be loaded in a Pharo 5.0 image either from the Configuration Browser, or by executing the following script:

Metacello new
 smalltalkhubUser: 'Moose' project: 'Moose';
 configuration: 'Moose';
 version: #stable;
 load.

Enjoy,
The Moose team

Posted by Tudor Girba at 15 August 2016, 9:10 am with tags moose, tooling link
|

jdt2famix 1.0.1

Over the last couple of weeks I worked on a new importer for Java code that can be used for Moose, and I am happy to announce that I released version 1.0.1.

jdt2famix is an open-source project, and the repository can be on Github. It is implemented in Java as a standalone project, and it is based on JDT Core (developed as part of the Eclipse project and available under the EPL2 license) and Fame for Java (originally developed by Adrian Kuhn and is available under the LGPL license).

The current implementation has an extensive coverage of entities that it can import, namely:

  • classes, parameterizable classes, parameterized types, enums, anonymous classes, inner classes
  • annotation types, annotation type attributes
  • annotation instances, annotation instance attributes
  • methods, constructors, class initializers
  • parameters, local variables
  • attributes, enum values
  • accesses
  • invocations, constructor invocations, class instantiations
  • thrown exceptions, caught exceptions, declared exceptions
  • comments associated with named entities
  • source anchors for named entities

The simplest way to use it is through the command line:

  1. Download and unzip the self contained binary 1.0.1 release.
  2. Go to the root folder where you have the sources and dependency libraries of a Java system (e.g., mysystem)
  3. Execute /path/to/jdt2famix.sh

The result will be an MSE file having the same name as the name of the folder from which you executed the command (e.g., mysystem.mse).

Let’s look at a concrete example. For this purpose, we pick the Maven project. We first download the 3.3.9 sources. These are only the plain sources, but to be able to import the complete model, we also need to have the dependencies available in the same folder.

As Maven is a Maven project (no pun intended), we can use the configuration to download the dependencies locally:

mvn dependency:copy-dependencies -DoutputDirectory=dependencies -DoverWriteSnapshots=true -DoverWriteReleases=false

(If you do not have a Maven project, you have to adapt this step and obtain the dependencies depending on the specifics of your project)

Then we simply execute:

/path/to/jdt2famix.sh

And we get a maven-3.3.9.mse file that we can load into Moose.

Maven-3.3.9.png

Posted by Tudor Girba at 3 August 2016, 4:08 pm with tags tooling, moose link
|
<< 1 2 3 4 5 6 7 8 9 10 >>