When VAST Platform 2021 (10.0.0) was released earlier this year, the Seaside version it shipped with was v3.2.0 (from 2016).
It was time for an update.
For our next release, VAST Platform 2022 (11.0.0), we decided it was a good time to port the latest Seaside version (v3.4.4 as of this post), and to create an easier and more frequent plan for future Seaside updates.
The previous Seaside port to VAST was done using the MonticelloImporter tool and required several manual steps. This time, we wanted to automate and script as much as possible so that future ports would be easier to do. Our new Tonel support tools were the logical choice for such a project. Additionally, we wanted to avoid patching canonical Seaside code to make it run in VAST, so we submitted all the VAST-specific changes directly to the Seaside upstream repository for acceptance and integration.
This was quite a challenging project, so we wanted to share some insights and the steps we took so that others doing similar ports or migrations have a head start.
The first step was to fork the canonical Seaside repository into a new Seaside repository for Instantiations, so we had a copy we could modify directly.
While Seaside is a framework used in most Smalltalk dialects, one problem we faced is that Seaside is not using Tonel as its default repository format. Instead, it uses an older format called FileTree, so we needed to first convert Seaside to Tonel format. We did this using the conversion tool provided by Iceberg on Pharo, and published the results in a new branch called tonel-dev in our forked repository.
Seaside is a very large project, but it depends on a smaller one called Grease to avoid having platform dependent code in its codebase. Therefore, the most logical flow was to start migrating Grease and then move onto Seaside itself.
Like Seaside, Grease also uses FileTree as its default format, so we had to move it to Tonel.
Once we had our forked repositories in Tonel format, we were ready to import the code into VAST using the following script:
"Tonel Setup"
repoPath := (CfsPath named: 'z:\Common\Development\git\Grease\').
reader := TonelReader new
readFrom: repoPath
filtering: [:packageName |
#('pharo' 'gemstone' 'squeak') noneSatisfy: [:pattern |
('*', pattern, '*') match: packageName ]
].
loader := TonelLoader on: reader.
loader beUnattended.
loader doNotCreateVersions.
loader mapTagsToSubapplications.
loader namingStrategy appSuffix: 'App'.
loader createsHookMethods: true.
loader doNotUseBaseEditions.
loader useApplicationPrerequisitesTable.
loader usePackageDependencyTable.
"Grease Core"
loader prerequisitesStrategy
at: 'Grease-Core' put: #('Kernel' ).
loader loadApplicationForPackageNamed: 'Grease-Core'.
#( #GreaseVASTCoreApp )
do: [ :anAppName | EmImageBuilder loadApplications: (Array with: (Application shadowsFor: anAppName ) first ) ].
"Grease Core Tests"
loader prerequisitesStrategy
at: 'Grease-Tests-Core' put: #( 'SUnit' 'GreaseVASTCoreApp').
loader packageDependencyStrategy
at: 'Grease-Tests-Core' put: #( 'Grease-Core' ).
loader loadApplicationsForPackagesNamed: #(
'Grease-Tests-Core'
).
#( #GreaseTestsVASTCoreApp )
do: [ :anAppName | EmImageBuilder loadApplications: (Array with: (Application shadowsFor: anAppName ) first ) ].Line 2 is the directory where we cloned our forked repository and checked out the branch tonel-dev.
Lines 3 to 8 (readFrom:filtering:) are just an optimization to avoid spending time parsing apps that are specific to other Smalltalk dialects.
Line 10 (beUnattended) is to tell Tonel we want to run in unattended mode. In other words, we don't want any GUI interaction.
Line 11 (doNotCreateVersions) tells Tonel to not version the loaded apps but instead leave them as open editions.
Line 12 (mapTagsToSubapplications) declares that package tags (available in other dialects like Pharo) should be mapped to VAST subapplications.
Line 14 (createsHookMethods:) enables the creation of the methods loaded, removing and failedRemove in the created applications. This step is important if you rely on the class side initialize method.
Line 15: By default, Tonel tools look for a "base edition" of a given app to compare and apply the changes you are loading. If there are no changes, it does nothing. For this use case where we are porting a project, it would be easier if Tonel always assumes there is no base edition, like we're starting from scratch. This is what line 15 (doNotUseBaseEditions) does.
Line 16: (useApplicationPrerequisitesTable) We explicitly declare the prerequisites (dependencies) of each loaded app. This is done by filling the table prerequisitesStrategy. Look at the senders in lines 20 and 27. Note that this is only for dependencies with apps that are not part of the apps being loaded. For dependencies between the apps being loaded, see the next paragraph about line 17.
Line 17 (usePackageDependencyTable) is similar to line 16 (useApplicationPrerequisitesTable) except that this defines the dependencies between the packages that are being loaded, and the load order of all the packages. This is done by filling the table packageDependencyStrategy as you can see in the sender in line 29.
As mentioned before, with prerequisitesStrategy and packageDependencyStrategy we can define all the dependencies of the applications and packages. The problem is: How do we know what are the actual dependencies of a given package?
Seaside has a convention, using extension methods to GRPackage, to define the dependencies for each package. However, we found out that these were not up to date as most dialects just rely on Metacello BaselineOf for dependency management. Therefore, we manually extracted this information from BaselineOf and applied it to our script.
Once the Tonel setup was done, the next steps were to start loading the packages and organizing the apps into Configuration Maps. The package loading happens in lines 22 and 31 (loadApplicationForPackageNamed:).
There isn't any strict methodology for organizing apps within Configuration Maps. It's all up to the user to decide how to do it. In this example with Grease, we have decided to create config maps called Grease Core and Grease Core Tests. However, note that the Tonel loader itself does not create any configuration map, and is something we have to manually do. The only thing we'll do with the Tonel loader is to define which config maps we are going to create and which apps it will include.
Something interesting to note is how we load the VAST specific apps during this process. See the senders of loadApplications: in lines 24 and 36. This is necessary because the VAST specific apps may include classes that are needed for the next apps to be loaded.
Once we had Grease ported and loaded, it was time to port Seaside itself. The process is exactly the same as with Grease except that there are many more packages, dependencies, configuration maps, etc. But, the logic is the same:
| repoPath reader loader |
repoPath := CfsPath named: 'z:\Common\Development\git\Seaside\'.
reader := TonelReader new
readFrom: repoPath
filtering: [:packageName |
#('pharo' 'gemstone' 'squeak') noneSatisfy: [:pattern |
('*', pattern, '*') match: packageName ]
].
loader := TonelLoader on: reader.
loader beUnattended.
loader doNotCreateVersions.
loader mapTagsToSubapplications.
loader namingStrategy appSuffix: 'App'.
loader createsHookMethods: true.
loader doNotUseBaseEditions.
loader useApplicationPrerequisitesTable.
loader usePackageDependencyTable.
"Seaside Core"
loader prerequisitesStrategy
at: 'Seaside-Continuation' put: #('GreaseCoreApp' 'GreaseVASTCoreApp' );
at: 'Seaside-Core' put: #('GreaseCoreApp' 'GreaseVASTCoreApp' );
at: 'Seaside-Tests-JSON' put: #( 'SUnit' );
at: 'Seaside-Tests-FileSystem' put: #('SUnit').
loader packageDependencyStrategy
at: 'Seaside-Canvas' put: #('Seaside-Core');
at: 'Seaside-Session' put: #( 'Seaside-Core' 'Seaside-Canvas');
at: 'Seaside-Component' put: #( 'Seaside-Core' );
at: 'Seaside-InternetExplorer' put: #( 'Seaside-Core' );
at: 'Seaside-Widgets' put: #( 'Seaside-Component' 'Seaside-Canvas');
at: 'Seaside-RenderLoop' put: #( 'Seaside-Component' 'Seaside-Session');
at: 'Seaside-Tools-Core' put: #('Seaside-Core' 'Seaside-Component' 'Seaside-RenderLoop' 'Seaside-Session' );
at: 'Seaside-Flow' put: #( 'Seaside-Core' 'Seaside-Component' 'Seaside-RenderLoop' 'Seaside-Tools-Core');
at: 'Seaside-Environment' put: #('Seaside-Core' 'Seaside-Canvas' 'Seaside-Session' 'Seaside-Component' 'Seaside-RenderLoop' 'Seaside-Tools-Core' 'Seaside-Widgets').
loader loadApplicationsForPackagesNamed: #(
'Seaside-Continuation' 'Seaside-Core' 'Seaside-Canvas' 'Seaside-Session' 'Seaside-Component'
'Seaside-Widgets' 'Seaside-RenderLoop' 'Seaside-Tools-Core' 'Seaside-Flow' 'Seaside-Environment'
).
#( #SeasideVASTContinuationApp #SeasideVASTCoreApp #SeasideVASTWidgetsApp #SeasideVASTFlowApp #SeasideVASTEnvironmentApp )
do: [ :anAppName | EmImageBuilder loadApplications: (Array with: (Application shadowsFor: anAppName ) first ) ].
"Seaside Core Tests"
loader prerequisitesStrategy
at: 'Seaside-Tests-Core' put: #( 'GreaseTestsCoreApp' ).
loader packageDependencyStrategy
at: 'Seaside-Tests-Core' put: #( 'Seaside-Core' );
at: 'Seaside-Tests-Canvas' put: #( 'Seaside-Tests-Core' 'Seaside-Canvas' );
at: 'Seaside-Tests-Session' put: #( 'Seaside-Tests-Core' 'Seaside-Session' 'Seaside-Tests-Canvas' );
at: 'Seaside-Tests-Component' put: #( 'Seaside-Component' 'Seaside-Tests-Core' );
at: 'Seaside-Tests-RenderLoop' put: #( 'Seaside-RenderLoop' 'Seaside-Tests-Session' );
at: 'Seaside-Tests-Environment' put: #( 'Seaside-Environment' 'Seaside-Tests-Component' 'Seaside-Tests-RenderLoop' 'Seaside-Tests-Session' 'Seaside-Tests-Canvas' );
at: 'Seaside-Tests-Functional' put: #( 'Seaside-Environment' 'Seaside-Tests-Core' 'Seaside-Widgets' );
at: 'Seaside-Tests-Flow' put: #( 'Seaside-Flow' 'Seaside-Tests-Functional' );
at: 'Seaside-Tests-UTF8' put: #( 'Seaside-Tests-Core' );
at: 'Seaside-Tests-InternetExplorer' put: #( 'Seaside-InternetExplorer' 'Seaside-Tests-Core').
loader loadApplicationsForPackagesNamed: #(
'Seaside-Tests-Core' 'Seaside-Tests-Canvas' 'Seaside-Tests-Session' 'Seaside-Tests-Component' 'Seaside-Tests-RenderLoop'
'Seaside-Tests-Environment' 'Seaside-Tests-Functional' 'Seaside-Tests-Flow' 'Seaside-Tests-UTF8' 'Seaside-Tests-InternetExplorer'
).
#( #SeasideTestsVASTCoreApp #SeasideTestsVASTCanvasApp #SeasideTestsVASTFlowApp #SeasideTestsVASTUTF8App #SeasideTestsVASTFunctionalApp)
do: [ :anAppName | EmImageBuilder loadApplications: (Array with: (Application shadowsFor: anAppName ) first ) ].
"Seaside Adaptors"
#( #SeasideAdaptorsSstApp )
do: [ :anAppName | EmImageBuilder loadApplications: (Array with: (Application shadowsFor: anAppName ) first ) ].
"Seaside JSON"
loader packageDependencyStrategy
at: 'Seaside-JSON-Core' put: #('Seaside-Core' 'Seaside-Canvas' ).
loader loadApplicationsForPackagesNamed: #(
'Seaside-JSON-Core'
).
#( #SeasideVASTJSONCoreApp )
do: [ :anAppName | EmImageBuilder loadApplications: (Array with: (Application shadowsFor: anAppName ) first ) ].
"Seaside JSON Tests"
loader packageDependencyStrategy
at: 'Seaside-Tests-JSON' put: #( 'Seaside-JSON-Core' ).
loader loadApplicationsForPackagesNamed: #(
'Seaside-Tests-JSON'
).
#( #SeasideTestsVASTJSONApp )
do: [ :anAppName | EmImageBuilder loadApplications: (Array with: (Application shadowsFor: anAppName ) first ) ].
"Seaside Email"
loader packageDependencyStrategy
at: 'Seaside-Email' put: #('Seaside-Core' ).
loader loadApplicationsForPackagesNamed: #(
'Seaside-Email'
).
#( #SeasideVASTEmailApp )
do: [ :anAppName | EmImageBuilder loadApplications: (Array with: (Application shadowsFor: anAppName ) first ) ].
"Seaside Email Tests"
loader packageDependencyStrategy
at: 'Seaside-Tests-Email' put: #( 'Seaside-Email' 'Seaside-Tests-Core' ).
loader loadApplicationsForPackagesNamed: #(
'Seaside-Tests-Email'
).
#( #SeasideTestsVASTEmailApp )
do: [ :anAppName | EmImageBuilder loadApplications: (Array with: (Application shadowsFor: anAppName ) first ) ].
"Seaside Development"
loader packageDependencyStrategy
at: 'Seaside-Tools-Web' put: #( 'Seaside-Tools-Core' 'Seaside-RenderLoop' 'Seaside-Widgets' );
at: 'Seaside-Development' put: #( 'Seaside-Tools-Web' 'Seaside-Flow' ).
loader loadApplicationsForPackagesNamed: #(
'Seaside-Development' 'Seaside-Tools-Web'
).
#( #SeasideVASTDevelopmentApp #SeasideVASTToolsWebApp #SeasideVASTToolsServerMonitorApp )
do: [ :anAppName | EmImageBuilder loadApplications: (Array with: (Application shadowsFor: anAppName ) first ) ].
"Seaside Development Tests"
loader packageDependencyStrategy
at: 'Seaside-Tests-Tools-Web' put: #( 'Seaside-Tools-Web' 'Seaside-Tests-Environment' );
at: 'Seaside-Tests-Development' put: #( 'Seaside-Development' 'Seaside-Tests-Tools-Web' 'Seaside-Tests-Environment' ).
loader loadApplicationsForPackagesNamed: #(
'Seaside-Tests-Development' 'Seaside-Tests-Tools-Web'
).
#( #SeasideTestsVASTDevelopmentApp )
do: [ :anAppName | EmImageBuilder loadApplications: (Array with: (Application shadowsFor: anAppName ) first ) ].
"Seaside RSS"
loader packageDependencyStrategy
at: 'RSS-Core' put: #( 'Seaside-Core' 'Seaside-Canvas' ).
loader loadApplicationsForPackagesNamed: #(
'RSS-Core'
).
#( #RSSVASTCoreApp )
do: [ :anAppName | EmImageBuilder loadApplications: (Array with: (Application shadowsFor: anAppName ) first ) ].
"Seaside RSS Tests"
loader packageDependencyStrategy
at: 'RSS-Tests-Core' put: #( 'RSS-Core' 'Seaside-Tests-Core' ).
loader loadApplicationsForPackagesNamed: #(
'RSS-Tests-Core'
).
"Seaside RSS Examples"
loader packageDependencyStrategy
at: 'RSS-Examples' put: #( 'RSS-Core' 'Seaside-Tools-Core' ).
loader loadApplicationsForPackagesNamed: #(
'RSS-Examples'
).
"Seaside Javascript"
loader packageDependencyStrategy
at: 'Javascript-Core' put: #( 'Seaside-Core' 'Seaside-Canvas' ).
loader loadApplicationsForPackagesNamed: #(
'Javascript-Core'
).
#( #JavascriptVASTCoreApp )
do: [ :anAppName | EmImageBuilder loadApplications: (Array with: (Application shadowsFor: anAppName ) first ) ].
"Seaside Javascript Tests"
loader packageDependencyStrategy
at: 'Javascript-Tests-Core' put: #( 'Javascript-Core' 'Seaside-Tests-Core' ).
loader loadApplicationsForPackagesNamed: #(
'Javascript-Tests-Core'
).
#( #JavascriptTestsVASTCoreApp )
do: [ :anAppName | EmImageBuilder loadApplications: (Array with: (Application shadowsFor: anAppName ) first ) ].
"Seaside JQuery"
loader packageDependencyStrategy
at: 'JQuery-Core' put: #( 'Javascript-Core' );
at: 'JQuery-JSON' put: #( 'JQuery-Core' 'Seaside-JSON-Core' ).
loader loadApplicationsForPackagesNamed: #(
'JQuery-Core' 'JQuery-JSON'
).
#( #JQueryVASTCoreApp )
do: [ :anAppName | EmImageBuilder loadApplications: (Array with: (Application shadowsFor: anAppName ) first ) ].
"Seaside JQuery Tests"
loader packageDependencyStrategy
at: 'JQuery-Tests-Core' put: #( 'JQuery-Core' 'Javascript-Tests-Core' 'Seaside-Development' 'Seaside-Tests-Functional' );
at: 'JQuery-Tests-JSON' put: #( 'JQuery-JSON' 'JQuery-Tests-Core' ).
loader loadApplicationsForPackagesNamed: #(
'JQuery-Tests-Core' 'JQuery-Tests-JSON'
).
"Seaside JQueryUI"
loader packageDependencyStrategy
at: 'JQuery-UI' put: #( 'JQuery-Core' ).
loader loadApplicationsForPackagesNamed: #(
'JQuery-UI'
).
"Seaside JQueryUI Tests"
loader packageDependencyStrategy
at: 'JQuery-Tests-UI' put: #( 'JQuery-UI' 'JQuery-Tests-Core' ).
loader loadApplicationsForPackagesNamed: #(
'JQuery-Tests-UI'
).
"Seaside Scriptaculous"
loader packageDependencyStrategy
at: 'Prototype-Core' put: #( 'Javascript-Core' );
at: 'Scriptaculous-Core' put: #( 'Prototype-Core' );
at: 'Scriptaculous-Components' put: #( 'Scriptaculous-Core' 'Seaside-Flow' 'Seaside-Widgets' 'Seaside-Component' ).
loader loadApplicationsForPackagesNamed: #(
'Prototype-Core' 'Scriptaculous-Core' 'Scriptaculous-Components'
).
#( #ScriptaculousVASTComponentsApp #ScriptaculousVASTCoreApp )
do: [ :anAppName | EmImageBuilder loadApplications: (Array with: (Application shadowsFor: anAppName ) first ) ].
"Seaside Scriptaculous Tests"
loader packageDependencyStrategy
at: 'Prototype-Tests-Core' put: #( 'Prototype-Core' 'Javascript-Tests-Core' 'Seaside-Tests-Functional' 'Seaside-Tests-Canvas' );
at: 'Scriptaculous-Tests-Core' put: #( 'Scriptaculous-Core' 'Prototype-Tests-Core' );
at: 'Scriptaculous-Tests-Components' put: #( 'Scriptaculous-Components' 'Scriptaculous-Tests-Core' ).
loader loadApplicationsForPackagesNamed: #(
'Prototype-Tests-Core' 'Scriptaculous-Tests-Core' 'Scriptaculous-Tests-Components'
).
"Seaside REST"
loader packageDependencyStrategy
at: 'Seaside-REST-Core' put: #( 'Seaside-Core' ).
loader loadApplicationsForPackagesNamed: #(
'Seaside-REST-Core'
).
#( #SeasideVASTRESTCoreApp )
do: [ :anAppName | EmImageBuilder loadApplications: (Array with: (Application shadowsFor: anAppName ) first ) ].
"Seaside REST Tests"
loader packageDependencyStrategy
at: 'Seaside-Tests-REST-Core' put: #( 'Seaside-REST-Core' 'Seaside-Tests-Core' ).
loader loadApplicationsForPackagesNamed: #(
'Seaside-Tests-REST-Core'
).
#( #SeasideTestsVASTRESTCoreApp )
do: [ :anAppName | EmImageBuilder loadApplications: (Array with: (Application shadowsFor: anAppName ) first ) ].
"Seaside Examples"
loader packageDependencyStrategy
at: 'Seaside-Examples' put: #( 'Seaside-Component' 'Seaside-Canvas' 'Seaside-Tools-Core' 'Seaside-Development' 'JQuery-Core' );
at: 'Seaside-REST-Examples' put: #( 'Seaside-REST-Core' 'Seaside-Canvas' 'Seaside-RenderLoop' 'Seaside-Examples' ).
loader loadApplicationsForPackagesNamed: #(
'Seaside-Examples' 'Seaside-REST-Examples'
).
"Seaside Examples Tests"
loader packageDependencyStrategy
at: 'Seaside-Tests-Examples' put: #( 'Seaside-Examples' 'Seaside-Tests-Environment' ).
loader loadApplicationsForPackagesNamed: #(
'Seaside-Tests-Examples'
).
"Seaside REST Examples"
loader packageDependencyStrategy
at: 'Seaside-REST-Examples' put: #( 'Seaside-REST-Core' 'Seaside-Canvas' 'Seaside-RenderLoop' 'Seaside-Examples' ).
loader loadApplicationsForPackagesNamed: #(
'Seaside-REST-Examples'
).
"Seaside FileSystem"
loader packageDependencyStrategy
at: 'Seaside-FileSystem' put: #( 'Seaside-Core' ).
loader loadApplicationsForPackagesNamed: #(
'Seaside-FileSystem'
).
#( #SeasideVASTFileSystemApp )
do: [ :anAppName | EmImageBuilder loadApplications: (Array with: (Application shadowsFor: anAppName ) first ) ].
"Seaside FileSystem Tests"
loader packageDependencyStrategy
at: 'Seaside-Tests-FileSystem' put: #( 'Seaside-FileSystem' ).
loader loadApplicationsForPackagesNamed: #(
'Seaside-Tests-FileSystem'
).
#( #SeasideTestsVASTFileSystemApp )
do: [ :anAppName | EmImageBuilder loadApplications: (Array with: (Application shadowsFor: anAppName ) first ) ].Once we finished importing all of the code, the next step was to determine if dialect-specific changes were needed to work with the new versions of Seaside or Grease. If this is the case, we have to implement these changes and include them in our VAST-specific apps just like other Smalltalk dialects would.
In previous years, we've been collecting some general Seaside fixes that were integrated only in VAST, but we had not contributed them to the upstream repository.
Also, there were many parts of Seaside that were not as portable as they could be, and had caused issues during previous ports to VAST.
As a result, we now report all of these issues in Seaside's main repository, along with providing pull requests to address them. A special thanks goes out to the Seaside folks (especially to Johan Brichau), who accepted and merged all our changes. What a great community!
Porting the latest Seaside to VAST was a challenge, but it was much easier because of our Tonel Tools, which proved to be indispensable when porting a very large project like Seaside.
The excellent port results were well worth the effort, and they have paved the way for future and more frequent Seaside updates.
If you have questions when working on a port of your own, please do not hesitate to contact us.
