LTS Smooth Migration Subproject

A smooth upgrade path from TYPO3 CMS 4.5 LTS to TYPO3 CMS 6.2 LTS is one of the major goals during the development of the upcoming version. In this article we would like to line out what the plans are and how we need your help to make it happen.
Added by Ernesto Baschny about 6 years ago

Historic perspective: Deprecation Strategy

In the TYPO3 CMS world we were used to be able to upgrade from one version to the next with little to no effort. By upgrading we gain new functionality but usually the "old way" still works. This is called backwards compatibility, and TYPO3 CMS users are very spoiled to have had a high grade of backwards compatibility.

But with backwards compatibility comes the downside: The code base gets larger and larger, the "old code" needs to be further maintained (for security, tests and when considering new features or change of features / interfaces) and people (developers, integrators) get lazy in adapting to newer technologies that emerge with newer versions.

This is why several years ago (during 4.3 development back in 2009) we agreed to introduce a consistent deprecation policy: If some method / function in the Core is replaced by a new one and no longer in use (by the Core itself), it is marked as "deprecated" and can be removed two releases afterwards. This was also when the "deprecation log" was introduced.

At that time we already had lots of stuff marked as deprecated since "TYPO 3.5".

Up to version 4.5 LTS (2011) we haven't really practiced the "removal" part of the strategy consistently. In fact with 4.5 LTS we decided "for the last time" to keep all deprecated methods and start removing them in 4.6 (and later) - which is also why 4.6 is called "rebase".

Now we are in the situation that 3 years later (we had 4.6, 4.7, 6.0, 6.1) lots of methods are finally really gone. So for the first time, people upgrading from 4.5 to 6.2 will face some major breaking of their sites.

Future: Migration strategy

Considering the above situation, we are aware that this change might come to a surprise to some, and could cause some reputation loss on long time TYPO3 enthusiasts. Thus this sub-project was founded to focus on this area, considering that many users will upgrade their systems for the first time since 4.5 LTS.

We don't want to announce an unrealistic goal with our "Smooth Migration" project. So this task-force is about providing all the tools and knowledge you need to migrate your systems.

A successful upgrade consists of:

1. migration is possible in all areas
2. migration can be planned and is quotable
3. everything works out as planned

What we plan to do

Upgrade Report Extension

We will provide an "compatibility check" extension which you can install into your TYPO3 CMS 4.5 LTS.

This extensions will generate a report about the system, lining out the topics which need to be adapter for an upgrade. This report should be as detailed as possible, so it could work as an upgrade check-list for developers and integrators.

These reports can also describe the possible solution and needed changes. This is an ideal base for calculating an upgrade (for your site or for your customer).

We currently are in the phase of designing the interfaces for the "checks" to be performed and identifying the changes over the last 5 versions of TYPO3.

We will try to provide checks for all the core topics and expect agencies and extension developers to provide even more "checks" (by implementing the check-interface) to get a more comprehensive test-suite.

Steffen Ritter will kick start the extension and will need the help of the community for implementing more checks.

Rework Migration Wizards

The TYPO3 Install Tool provides "upgrade wizards" to assist the work of an upgrade.

The current user interface is sub-optimal.

The implementation is not very stable:

  • Long upgrade wizards could run out of your max_execution_time and abort on error. Half-run wizards may have undetermined states and may not be able to finish cleanly.
  • Furthermore there are no dependencies or execution ordering possible.
  • It might crash if there is an incompatible extension.

While stabilizing the install tool, the Upgrade Wizards will be reworked so that an upgrade can work out well even on big installations.

Steffen Ritter will be working on the Upgrade Wizard together with the "Install Tool" task force (Christian Kuhn and other active contributors).

Compatibility extension

There could be an extension that one can install on TYPO3 6.2 LTS which "mocks" the way older versions of TYPO3 work by providing a compatibility layer.

This extension should bring back deprecated methods. It comes with a down-side of a slower system, but could be a solution to keep some site running for three more years without going into too much trouble.

Oliver Hader is working on this extension and is glad to discuss the details with you.

Extension Developer Guidelines

The Smooth Migration Wiki will kick start the documentation for Extension-Developers providing best practices on how to support both Version 4.5 and 6.2 LTS or - if not possible - easily migrate their extensions.

The Installation and Upgrade Guide will be enhanced with a chapter on the "LTS to LTS" migration problems.

Ernesto Baschny will coordinate with the documentation team (Francois Suter) on this documentation topic.

Your help required

Many fancy extensions and functionality reside at the agencies or with freelancers in their work for their customers.

For identifying pitfalls we need as many as possible real-life test cases. For that we highly encourage volunteers to test out the new version: upgrade the system to the master branch (git) and tell us where the problems occurs and (if possible) provide the solution.

Get Agencies and users on board: Code Sprint

At the T3DD13 in Hamburg there will be a session targeted on upgrading, where we expect to meet lead-developers / deployment specialists of the agencies in the TYPO3 universe to discuss the problems they have and how we could tackle them as community.

Furthermore we are planning to organize a code sprint (more like a "meet up") in the time between Alpha3 and Beta1 (in July, 2013):

Agencies interested in upgrading their installations send one or two developers that know their problems and headaches. We meet up during the week (work-time) and try to collect, identify and solve these problems. Each agency would cover the costs for their developer themselves.

Sponsoring for location, food, drinks, amenities would be highly appreciated. Ingo Schmitt from Marketing Factory already offered to organize this event (thanks a lot for your help already!).

Summary

What we want to provide you with is a rock stable upgrade path, that will support you on upgrading any site. These can be automatic fixes to solve code base problems, or a report with "to-dos" or suggestions that need to be worked on manually. In the end you will be able to upgrade smoothly to 6.2 LTS.

If you want to help, get involved, have questions or wishes, don't hesitate to contact Steffen Ritter or get more information through the project page


Comments

Added by Federico Bernardin about 6 years ago

Great, this extension seems very cool!!!

Added by Bernd Wilke about 6 years ago

Thanks in advance for all that work.
a great help will be the "Upgrade Report Extension".
for a lot of installations it would be helpful if all old function calls will be logged. especially all that which are removed in 6.2 and not deprecated in 4.5.

Added by Ernesto Baschny about 6 years ago

Bernd, good idea, but difficult to achieve technically, as an extension would have to "hook" into all potential classes / methods which were later dropped to be able to "log" them. Hooking into "all" function calls in all classes might be even possible (http://stackoverflow.com/questions/6369570/hooking-into-function-calls-in-php) but I think it gets darn slow. If you have an idea around it, please share.

Otherwise the "static" checks done in extensions and PHP code itself will be able to catch most issues.

Added by Mathias Schreiber about 6 years ago

Any info yet where the Meet-Up will be?
IIRC there are no tickets anymore for T3DD13.

Furthermore:
Backwards compatibility does not only stick to the codebase.
There are a LOT of things inside the TYPO3 Core which make current core team members (or active contributers or whatever the correct term is nowadays) heads spin.

Something one does not know about does not mean it is not needed.
Refactoring things is a good thing and I fancy that.
Removing stuff that is actually used because of whatever reason is bad.

I will try to elaborate on that during the meet up (I fear it's pretty late by then though).

Added by Siva Prasad about 6 years ago

Can any one please tell how this is going to work in typo3 website which is using DAM . Do we have any migration script for DAM to fal . Again do we need to change the extension which are using DAM atm ??

Added by Steffen Ritter about 6 years ago

@Mathias: I think most of the people by now your opinion on that. Even if it's worth to get to know completeley different points of your or got ass-kicked by reality from the outside time to time the planned session about upgrading is not the right place to elaborate on your concerns regarding strategy and development.The session is NOT to discuss why things have been done or will be done - It's a plain we had this in 4.5 and have that in 6.2 - what problems do you see with your installations out in the wild which we did not think about yet and how WE (in terms of participants) can work on that.

@Sivaprassad: I tend to answer both of your questions with YES.

Added by Julian Hofmann about 6 years ago

I agree with Bernd. It would be nice to have removed function/classes logged anywhere. The current deprecationlog only covers such methods which were already deprecated at time of releasing 4.5. Methods which were deprecated (and removed) later are currently not logged in 4.5.
@Ernesto: Maybe, it would be possible to have a special "4.5-LTS-upgrading version". I could imagine a version, where all methods are marked as deprecated, which have been deprecated in the later versions until 6.2. So, the normal deprecationlog could log these methods...

Perhaps, it would be a good way to upgrade step by step, instead of doing one big step, too. Is there one version between 4.5 and 6.2 which could be used as intermediate step? One Core version which could be used to analyze the neccessary changes for further upgrade?

Added by Marcel Burkhalter about 6 years ago

One major issue IMHO is that after an 4.5 -> 6.2 upgrade you will have a system which has lots of duplicate files (/uploads/ directory - on file per usage in a content element) but all new files are powered by FAL. This leaves the content editors with a system where they can centrally overwrite some files (thanks to FAL) but not the migrated files. Hence it would be really nice if my feature request regarding file consolidation could be implemented into the core or maybe even better into an extension which can be removed after the upgrade: http://forge.typo3.org/issues/43233

Added by Ernesto Baschny about 6 years ago

@Julian,

@Ernesto: Maybe, it would be possible to have a special "4.5-LTS-upgrading version". I could > imagine a version, where all methods are marked as deprecated, which have been deprecated in > the later versions until 6.2. So, the normal deprecationlog could log these methods...

That would indeed be a good idea. Maybe having a switch to turn this kind of "enhanced deprecation logging". I will discuss that with the team in the next opportunity.