Porting Blog

A chronological account of the work done with the Smalltalk Transform project.

The beginning was documented for the Values port to Squeak.

Now the real project starts and will be recorded here.

Tuesday, 21. June 2022

This version includes fileOuts for Squeak and Pharo. All load without fatal errors.

Squeak: 608 run, 317 pass, 63 fail, 228 error

Pharo: 608 run, 285 pass, 54 fail, 269 error

After the first raw fileOut, here is the result of the first porting changes.

Fortunately, I had nice coding sessions with Marcel Taeumel (Squeak) and Sean DiNigris (Pharo). We talked about porting, explored the image and tests and fixed things on the way without much of a plan. That was fun! Now, I followed up on all the issues, fixed them and added the transformations. The other issues reported on the list and on GitHub were fixed a well. Thank you, Christoph Thiede for the detailed and constructive list of issues. Thank you, Thorsten Bergmann for the discussion about names and new tests for Values.

Since Squeak and Pharo are still very similar, I fixed the issues in both Smalltalks. The new fileOut improves the test statistics quite nicely:

  • +11 tests, +169 pass, +44 fail, -202 error (Squeak)
  • +11 tests, +137 pass, +35 fail, -161 error (Pharo)

Details at the state of the port. PostScript is now almost ripe to pick. BTW, if anybody has use for PostScript, this package can be used stand-alone.

Now the list of issues. Maybe too detailed for most of you, but maybe you find an issue to comment on?

  • harvested 11 tests from the independent Values port of Thorsten Bergmann
  • fixed UTF16 encoding (using Zinc in Pharo - I don't how yet how to load prerequisites. Zinc seems to be in the base though)
  • added Character class»#illegalCode
  • replaced #decodeASCII85Group:. The problem is a difference in #first: (of lists): in VW it never fails (that's why I like and use it alot), because the parameter is forced into range. In Squeak and Pharo, it raises an error when the parameter is negative or the list has not enough elements. Here, the checks are done by the caller.
  • #doWithIndex: instead of #withIndexDo:
  • replaced #tokensBasedOn:
  • WideString instead of TwoByteString - I am not sure about this…
  • Squeak: replaced MD5 hash. Added a Monticello preload
  • #signal(:) instead of #raiseSignal(:)
  • replaced #isSignalledException by #isKindOf:
  • #theNonMetaClass instead of #instanceBehavior
  • Integer class»#readFrom:base: instead of Number class»#readIntegerFrom:radix:
  • Mutex instead of RecursionLock
  • added Pragma class»#allInMethod:
  • Pharo: Pragma»#methodSelector instead of #selector
  • Pharo: WebBrowser instead of ExternalWebBrowser
  • Pharo: added Valuemap»#valuesDo:

Saturday, 11. June 2022

This version includes fileOuts for VAST 10.0 and VAST 11.0. All load without fatal errors.

Download the fileout and file it into a fresh VAST image.

585 run, 143 pass, 14 fail, 428 error

VA Smalltalk is a bit stricter than other Smalltalks when it comes to compiling code. A fileIn aborts

  • when you define a class with an undeclared superclass,
  • when you extend an undeclared class or
  • when a method contains a reference to an undeclared class.

Syntax errors are shown on the Transcript and methods with syntax errors are not added to the system, but the fileIn proceeds.

Usually, for the initial fileOut, I add all the affected classes to the ignore list of the package change. This is not very practical, because much code is not written to the fileOut and references break.

But, hey, we do Smalltalk, right? Why not adding an error handler to the compilation and define the missing classes on the fly? The new class would subclasses Object and is useless otherwise. But now the class references stay valid and code browsing is much more meaningful. With some inlining and simplyfication I ended up with a handful methods without state - they could be attached to any class. I put them on the class side of the application class of a mini application.

This works very nicely. The freshly created classes are put into a subapplication with “_Undeclared” attached to the name. The goal for the port is to empty and remove that subapplication by modifying the code, so that the undeclared classes are not needed anymore.

I love Smalltalk!

But a problem remains. Now I have code for VAST to read the fileOut. But I want to have everything automatic. I don't want to ask the user to install whatever other software.

But, hey, we do Smalltalk, right? Why not adding the code of the AugmentingReader, as I call it, at the beginning of the fileOut and then use it to read the rest of the file? Wait, how can code which was just read from a file stream take over reading from that very stream? Ok, this is dark magic which you should not do at work :-). In Smalltalk we can ask for the current method invocation(!). Usually, this is accessed through the pseudo variable thisContext. VA Smalltalk does not have thisContext, but you can get a stack frame by using Processor currentProcess contextAtFrame: <Integer>. This is good enough. Now, I pick a frame a couple of sends before where the stream is an argument to that method. With the stream I continue reading with my AugmentingReader. When everything has been read, the original fileIn concludes normally. Of course, one could delete the AugmentingReader at the end of the fileIn automatically :-), but that may be too much dark magic.

I love Smalltalk!

Wednesday, 23. May 2022

This version includes fileOuts for Pharo 6.1, 7.0, 8.0, 9.0 and 10.0. All load without errors.

Download the fileout and file it into a fresh Pharo image.

597 Tests: 148 pass, 19 fail, 430 error

597 Tests: 174 pass, 23 fail, 400 error (Pharo 6.1)

Like Squeak, Pharo does not like methods with more than 15 arguments. That is OK. Unfortunately, the error is swallowed somewhere and only a little popup informs about the problem without the chance to open a debugger.

There is no progress feedback for fileIn. Since the sources are large, the image seems to hang: no reaction to user actions, no update of the Transcript and the text cursor stops blinking. Not even the mouse indicates a busy state. After 7 minutes on my machine, Pharo wakes up and everything is fine. So, hold on and keep waiting…

Like Squeak 5.3, Pharo 8.0 and earlier does not like methods with more than 255 literals.

Navigation is more difficult than in Squeak, because the class categories are sorted alphabetically. Squeak sorts them in fileIn order.

Pharo 6.1 has the same encoding (macintosh) problem as Squeak 5.3. Interestingly, when the fileIn is triggered by pressing the play button of the playground, after fileIn GT will raise an encoding error when trying to display the beginning of the file.

Wednesday, 18. May 2022

This new version also loads in Squeak 5.3.

Download the fileout and file it into a fresh Squeak 5.3 image.

The file should load without errors.

597 Tests: 174 pass, 23 fail, 400 error

The macintosh encoding does not support several characters used in the library. The characters are either in comments copied from the PDF specification (like $×) or they are used in encoding tests. My temporary solution is to replace all unencodable characters with a $? when writing the final result to the file. The problem will come back when the then failing encoding tests need to be fixed.

Another problem: Squeak 5.3 does not like methods with more than 255 literals. Fortunately, there is only one such data method in PDFtalk. The solution is the split the method containing the data into two and replace one and add the other for Squeak 5.3.

Monday, 16. May 2022

Version 2 of the Smalltalk Transform project is ready for action. Most notable, it allows the renaming of namespaces classes. All, well almost all, references to the renamed classes are replaced by the new names. Also, it is now possible to specify class definition changes, which allows to redefine specific aspects of a class like it's instance variables, pool dictionaries or even the superclass.

The new engine has been taken for a first run on PDFtalk.

FileOut from VW with

Squeak60 fileOutPDFtalk.

The fileout includes Values, PDFtalk, Values tests, PDFtalk tests and the PDFtalk Demonstrations. The file PDFtalk.Squeak60.st is about 6 MB in size.

Download the fileout and file it into a fresh Squeak 6.0 image.

The file should load without errors. This takes about 8 minutes on my machine.

Although the fileout is very raw, some tests are already running.

597 Tests: 174 pass, 23 fail, 400 error

Methods in Squeak cannot have more than 15 parameters. This was a bit surprising. This caused problems with two value classes with more than 15 instance variables. This was fixed by adding a few transformations to establish an alternative implementation.

Squeak 5.3 chokes on loading over a character not available in the macintosh encoding. The character is used in the PDF specification and I would like to not fix this in the source… I will think of something.

	name: #PDFtalk
	source: ((OrderedCollection new)
		add: (Package name: #Values);
		add: (Bundle name: #PDFtalk);
		add: (Package name: #'Values Testing');
		add: (Bundle name: #'PDFtalk Testing');
		add: (Package name: #'PDFtalk Demonstrations');
	changes: self pdftalkPackageChanges
	nameMapping: (NameMapping
		keep: ((OrderedCollection new)
			add: #{Smalltalk.PDF};
			add: #{PostScript.PSDictionary};
			add: #{PDFtalk.PDFObject};
			add: #{PDFtalk.PDFArray};
			add: #{PDFtalk.PDFDictionary};
			add: #{PDFtalk.PDFStream};
			add: #{PDFtalk.PDFString};
			add: #{PDFtalk.PDFDate};
			add: #{PDFtalk.PDFTypeDefinition};
			add: #{PDFtalk.PDFEncoder};
		classToNames: (Valuemap
			with: #{SubscriptOutOfBoundsError} -> #Error
			with: #{NonIntegerIndexError} -> #Error
			with: #{NotFoundError} -> #KeyNotFound)
		namespaceToPrefixes: ((Valuemap new)
			add: #{Smalltalk.PostScript} -> 'PS';
			add: #{Smalltalk.PDFtalk} -> 'Pt';
			add: #{PDFtalk.Fonts} -> 'PtF';
			add: #{PDFtalk.Fonts.CFF} -> 'CFF';
			add: #{PDFtalk.Fonts.OpenType} -> 'PtOT';
  • (meta) find a Squeak buddy to pair on this
  • fix the load warnings on the Transcript
  • resolve all remaining undeclareds
  • make the tests run
  • portingblog.txt
  • Last modified: 2022/06/22 10:57
  • by christian