Values porting log

Here, I try to record my steps for porting the Values package from VisualWorks to Squeak.

In your VisualWorks 8.3 image

have {Smalltalk Transform Project} (1.4.0.3,chaider) loaded
have [Squeak Fileout PDFtalk] (3.0.0.2,chaider) loaded, with empty transformations:

ValuesTransform
	^PackageChange new

If you don't have VisualWorks and cannot create the fileOut yourself, all versions are on GitHub.

The fileOut of this configuration is here.

File out from VW with

Squeak fileOutValues

. Start squeak, open the Transcript and file in Values.Squeak.st with

FileStream fileIn: 'C:\Users\Christian\Documents\image\Values.Squeak.st'

it loads 1 / 4 of the file before a syntax error occurs:

localSpecification
	<constant: #name class: #"Invalid literal character ->"{Printvalue}>
	<constant: #value class: #{Printvalue}>

The Transcript shows:

UndefinedObject»DoIt (PackageManifest is Undeclared) Attempt to create ManifestValues as a subclass of nil. Possibly a class is being loaded before its superclass.

VisualWorks has a special syntax for specifying a class in a namespace as literal; e.g. #{PDFtalk.Name}.

Fixed by adding add a transformation for the class method #localSpecification in any Value subclass. This translates #{Integer} to #(#Integer) and #{PDFtalk.Name} to #(#PDFtalk #Name).

ValuesTransform
	^PackageChange hierarchyChanges: (Array with: (ClassChange
		classReference: #{Smalltalk.Value}
		classChanges: (Array with: (Rewrite method: #localSpecification rule: #replaceLiteralBindingWithPath))))

Published as [Squeak Fileout PDFtalk] (3.0.0.3,chaider).

The Transcript warning complains that the class ManifestValues cannot be created as subclass of PackageManifest which is unknown.

Changed ChunkStream»#writeProperties:in:parents: to

use Object as superclass instead of PackageManifest
renamed class to About<Package>

Published as {Smalltalk Transform Project} (1.4.0.4,chaider).

FileOut from VW, then fileIn Values.Squeak.st into a fresh Squeak image.

Loads about 80% until it hits the error that ColorValue is not defined.

The Transcript shows:

Class>>nameRelativeTo: (Root is Undeclared) 
Class>>isInScope: (NameScope is Undeclared) 
UndefinedObject>>DoIt (ColorValue is Undeclared) 
UndefinedObject>>fromBytesRed:green:blue: (ColorValue is Undeclared) 
UndefinedObject>>fromBytesRed:green:blue: (ColorValue is Undeclared) 
UndefinedObject>>DoIt (ColorValue is Undeclared) 
UndefinedObject>>DoIt (ColorValue is Undeclared) 
UndefinedObject>>DoIt (ColorValue is Undeclared) 
UndefinedObject>>DoIt (ColorValue is Undeclared) 
UndefinedObject>>DoIt (ColorValue is Undeclared)

The system class ColorValue of VW is roughly equivalent to Color in Squeak. But instead of renaming all ColorValue references to Color, I define ColorValue as subclass of Color. With this, the target system is not polluted with compatibility methods for ColorValue.

This is expressed in the transformation with #newSuperclasses. The new transform looks like:

ValuesTransform
	^PackageChange
		newSuperclasses: (Valuemap with: #{Smalltalk.ColorValue} -> #Color)
		hierarchyChanges: (Array with: (ClassChange
			classReference: #{Smalltalk.Value}
			classChanges: (Array with: (Rewrite method: #localSpecification rule: #replaceLiteralBindingWithPath))))

The Transcript shows undeclared class references in two methods of class class.

Class>>nameRelativeTo: (Root is Undeclared) 
Class>>isInScope: (NameScope is Undeclared)

To handle this, we add local changes to the transformation. Local changes refer to classes in the source system.

ValuesTransform
	^PackageChange
		newSuperclasses: (Valuemap with: #{Smalltalk.ColorValue} -> #Color)
		hierarchyChanges: (Array with: (ClassChange
			classReference: #{Smalltalk.Value}
			classChanges: (Array with: (Rewrite method: #localSpecification rule: #replaceLiteralBindingWithPath))))
		localChanges: self valuesLocalTransform
 
valuesLocalTransform
	^Array with: (ClassChange
		classReference: #{Class}
		instanceChanges: (Array
			with: (Replace method: #nameRelativeTo: code: #_sq_nameRelativeTo:)
			with: (Replace method: #isInScope: code: #_sq_isInScope:)))

These transformations replace the body of the existing methods of class Class with the body of the #code methods.

Published as {Smalltalk Transform Project} (1.4.0.5,chaider) with minor changes (updated comments and corrected spelling).

Published as [Squeak Fileout PDFtalk] (3.0.0.4,chaider) with the updated transformations.

FileOut from VW, then fileIn Values.Squeak.st into a fresh Squeak image.

Loads about 85% until it hits the error that GeneralBindingReference is not defined.

The Transcript shows:

UndefinedObject>>DoIt (GeneralBindingReference is Undeclared) 
UndefinedObject>>DoIt (GeneralBindingReference is Undeclared) 
UndefinedObject>>DoIt (GeneralBindingReference is Undeclared) 
UndefinedObject>>DoIt (GeneralBindingReference is Undeclared)

Since GeneralBindingReference is specific to namespaces in VW, we don't need to port the class extension methods. For this we add the class to the #unusedClasses in our #ValuesTransform PackageChange:

ValuesTransform
	^PackageChange
		unusedClasses: #(#{Smalltalk.GeneralBindingReference})
		newSuperclasses: (Valuemap with: #{Smalltalk.ColorValue} -> #Color)
		hierarchyChanges: (Array with: (ClassChange
			classReference: #{Smalltalk.Value}
			classChanges: (Array with: (Rewrite method: #localSpecification rule: #replaceLiteralBindingWithPath))))
		localChanges: self valuesLocalTransform

Published as [Squeak Fileout PDFtalk] (3.0.0.5,chaider) with the updated transformation.

FileOut from VW, then fileIn Values.Squeak.st into a fresh Squeak image.

Loads about 97% until it hits the error that Timestamp is not defined.

The Transcript shows:

UndefinedObject>>DoIt (Timestamp is Undeclared) 
UndefinedObject>>DoIt (Timestamp is Undeclared) 
UndefinedObject>>DoIt (Timestamp is Undeclared) 
UndefinedObject>>DoIt (Timestamp is Undeclared) 
UndefinedObject>>DoIt (Timestamp is Undeclared)

We do the same trick as in FileIn x and set DateAndTime as superclass of Timestamp. The new Transformation is:

ValuesTransform
	^PackageChange
		unusedClasses: #(#{Smalltalk.GeneralBindingReference})
		newSuperclasses: (Valuemap
			with: #{Smalltalk.ColorValue} -> #Color
			with: #{Timestamp} -> #DateAndTime)
		hierarchyChanges: (Array with: (ClassChange
			classReference: #{Smalltalk.Value}
			classChanges: (Array with: (Rewrite method: #localSpecification rule: #replaceLiteralBindingWithPath))))
		localChanges: self valuesLocalTransform

Published as [Squeak Fileout PDFtalk] (3.0.0.6,chaider) with the updated transformation.

FileOut from VW, then fileIn Values.Squeak.st into a fresh Squeak image.

Loads without errors or warnings!.

First milestone: Clean fileIn without errors or warnings!

Next step: fileIn the tests.

Added Squeak class method #fileOutValuesTesting analog to #fileOutValues:

fileOutValuesTesting
	self
		write: ('.' asFilename construct: 'ValuesTesting.Squeak.st')
		pundles: (Array with: (Store.Registry packageNamed: #'Values Testing'))
		package: #'Values-Testing'
		prerequisites: nil
		packageChanges: (Valuemap with: 'Values Testing' -> self ValuesTestingTransform)

with the standard transformation for Value classes:

ValuesTestingTransform
	^PackageChange hierarchyChanges: (Array with: (ClassChange
		classReference: #{Smalltalk.Value}
		classChanges: (Array with: (Rewrite method: #localSpecification rule: #replaceLiteralBindingWithPath))))

FileOut from VW with:

Squeak fileOutValuesTesting

then fileIn ValuesTesting.Squeak.st into a Squeak image with Values loaded.

Loads without errors.

The Transcript shows:

ValuemapTests>>testIndexedAccess (SubscriptOutOfBoundsError is Undeclared) 
ValuemapTests>>testIndexedAccess (SubscriptOutOfBoundsError is Undeclared) 
ValuemapTests>>testIndexedAccess (NonIntegerIndexError is Undeclared) 
ValuemapTests>>testIndexedAccess (NonIntegerIndexError is Undeclared) 
ValuemapTests>>testKeyedAccess (NotFoundError is Undeclared) 
ValuemapTests>>testRemoving (NotFoundError is Undeclared) 
ValuemapTests>>testRemoving (NotFoundError is Undeclared) 
ValuemapTests>>testRemoving (NotFoundError is Undeclared) 
ValuemapTests>>testRemoving (NotFoundError is Undeclared)

The VW error classes are not known in Squeak. We define a rewrite of the three test methods:

ValuesTestingTransform
	^PackageChange
		hierarchyChanges: (Array with: (ClassChange
			classReference: #{Smalltalk.Value}
			classChanges: (Array with: (Rewrite method: #localSpecification rule: #replaceLiteralBindingWithPath))))
		localChanges: (Array with: (ClassChange
			classReference: #{Smalltalk.ValuemapTests}
			instanceChanges: ((OrderedCollection new)
				add: (Rewrite method: #testIndexedAccess rule: #replaceSqueakErrorClasses);
				add: (Rewrite method: #testKeyedAccess rule: #replaceSqueakErrorClasses);
				add: (Rewrite method: #testRemoving rule: #replaceSqueakErrorClasses);
				yourself)))

The rule #replaceSqueakErrorClasses in class ParseTreeRewriter (analog to #replaceGemstoneErrorClasses) replaces the Error classes by string replacement:

replaceSqueakErrorClasses
	^(self new)
		replace: SubscriptOutOfBoundsError name asString with: #Error asString;
		replace: NonIntegerIndexError name asString with: #Error asString;
		replace: NotFoundError name asString with: #NotFound asString;
		yourself

Published as [Squeak Fileout PDFtalk] (3.0.0.7,chaider) with the updated transformation.

FileOut [Values] and [Values Testing], then fileIn into a fresh Squeak image.

Loads without errors or warnings!

2. milestone: Clean fileIn of the tests without errors or warnings!

Test results: 25 run in 0:00:00:00, 0 passes, 0 expected failures, 1 failures, 24 errors, 0 unexpected passes

This Squeak fileOut (2022-02-01) contains the current Values.Squeak.st and ValuesTesting.Squeak.st.

The next step must be done in Squeak: fix the implementation to make the tests pass and modify the transformation so that the next fileOut contains the right code. If you don't have access to VW, please send a change file.