Bug #51069

Updating extension changes load order

Added by Francois Suter about 6 years ago. Updated almost 2 years ago.

Status:
Closed
Priority:
Should have
Assignee:
Category:
Extension Manager
Target version:
Start date:
2013-08-14
Due date:
% Done:

100%

TYPO3 Version:
6.2
PHP Version:
5.3
Tags:
Complexity:
no-brainer
Is Regression:
No
Sprint Focus:
Stabilization Sprint

Description

With the 6+ EM when an extension is updated, it is placed at the end of the "extListArray". This is wrong; the extension should stay at its place in the list. Changing the extension's place in the list also changes its load order, which may result in problems in case of dependencies between extensions.


Related issues

Related to TYPO3 Core - Bug #57825: Configuration option for defining loading order of extensions Closed 2014-04-11
Related to TYPO3 Core - Bug #67130: Invalid loading order of packages with suggestions Closed 2015-05-25

Associated revisions

Revision 5d8de35b (diff)
Added by Markus Klein over 4 years ago

[BUGFIX] Adhere "suggestions" when resolving ext loading order

The issue #58701 introduced the functionality to fetch suggestions defined
in composer.json or ext_emconf.php into the Package class.
A copy and paste error causes the loading of the suggestions to
actually fail, so those never reach the DependencyResolver.

This patch fixes the tiny mistake and thus ensures that suggestions
really influence the loading order of extensions.

Resolves: #51069
Releases: master, 6.2
Change-Id: Ib12e7f91f49c0b2bda56fdb4394a604e322b741c
Reviewed-on: http://review.typo3.org/39692
Reviewed-by: Helmut Hummel <>
Tested-by: Helmut Hummel <>
Reviewed-by: Wouter Wolters <>
Reviewed-by: Xavier Perseguers <>
Tested-by: Xavier Perseguers <>

Revision d1481310 (diff)
Added by Markus Klein over 4 years ago

[BUGFIX] Adhere "suggestions" when resolving ext loading order

The issue #58701 introduced the functionality to fetch suggestions defined
in composer.json or ext_emconf.php into the Package class.
A copy and paste error causes the loading of the suggestions to
actually fail, so those never reach the DependencyResolver.

This patch fixes the tiny mistake and thus ensures that suggestions
really influence the loading order of extensions.

Resolves: #51069
Releases: master, 6.2
Change-Id: Ib12e7f91f49c0b2bda56fdb4394a604e322b741c
Reviewed-on: http://review.typo3.org/39711
Reviewed-by: Xavier Perseguers <>
Tested-by: Xavier Perseguers <>

History

#1 Updated by Francois Suter about 6 years ago

The new installation process uninstalls an extension before installing it again during the update process. This is even documented in the phpDoc of the following method (from class \TYPO3\CMS\Extensionmanager\Service\ExtensionManagementService):

    /**
     * Uninstall extensions that will be updated
     * This is not strictly necessary but cleaner all in all
     *
     * @param array<Tx_Extensionmanager_Domain_Model_Extension> $updateQueue
     * @return array
     */
    protected function uninstallDependenciesToBeUpdated(array $updateQueue) {
        $resolvedDependencies = array();
        foreach ($updateQueue as $extensionToUpdate) {
            $this->installUtility->uninstall($extensionToUpdate->getExtensionKey());
            $resolvedDependencies['updated'][$extensionToUpdate->getExtensionKey()] = $extensionToUpdate;
        }
        return $resolvedDependencies;
    }

So if it's not strictly necessary, could we avoid doing it?

#2 Updated by Philipp Gampe about 6 years ago

  • Status changed from New to Needs Feedback

IMHO the order should be based on dependecies. Then we would have no problems with this, right?

#3 Updated by Francois Suter about 6 years ago

It could, yes, while making sure to avoid endless loops.

What this means - though - is that every time you install an extension, you need to recalculate all the dependencies. It's an overhead, but then again this does not happen so often. It would also be the right place to respect the priority property ("top" and "bottom"), which is not taken into account anymore (as I reported in a separate issue).

#4 Updated by Alexander Opitz about 6 years ago

  • Status changed from Needs Feedback to New

#5 Updated by Markus Klein almost 6 years ago

  • Is Regression set to No

I guess this one is not a topic anymore since 6.0 is in security fixes only mode now.

#6 Updated by Francois Suter almost 6 years ago

  • Target version changed from next-patchlevel to 6.2.0
  • TYPO3 Version changed from 6.0 to 6.2

Fair enough. So I raised the target version to 6.2, since the problem still exists.

#7 Updated by Christian Kuhn over 5 years ago

For 6.2, the extListArray is gone and substituted with PackageStates.php. A new algorithm takes care of load orders depending on the extension dependencies. For example, if your extension "bar" depends on other extension "foo", then it is taken care, that "foo"'s ext_localconf and ext_tables are loaded before "bar". Status of this is afaik "should work" in 6.2. Please feedback if there are still issues in this area that we may have missed (Markus Klein may be a good partner here since he wrote the sorting algorithm).

#8 Updated by Francois Suter over 5 years ago

I'm sorry but priority is not the same as dependencies.

My use case is the following: I generally store all my customisations in a specific extension. I need to know that this extension is loaded last, hence the use of priority "bottom". Of course, I can set up the list of all modified extensions in the dependencies inside my extension, but it's more constraining than having just the priority.

I guess I can live with that, but we should then officially deprecate the "priority" property.

And BTW that does not change the question I posted in the list about the performance gain we could have by caching ext_emconf data.

#9 Updated by Susanne Moog about 5 years ago

  • Status changed from New to Needs Feedback

Hi,

@Francois: yes we should deprecate the priority property - how do we do that?

Cheers,

Susi

#10 Updated by Francois Suter about 5 years ago

Hey Susanne,

Good to see you active again!

I don't think there's any way to deprecate extension settings in the Core itself, since it is really information for extension developers. It could be an addition to the TER, some kind of report informing about wrong and deprecated settings.

So it would say it is mostly a documentation issue.

#11 Updated by Christian Ludwig about 5 years ago

Even better would be "loadBefore" and "loadAfter" parameters because there are extensions that do not depend on others but need to be loaded before or after an other extension (if it is installed). So at the moment - and without "priority" - it is not guaranteed that loading order is fail safe.

#12 Updated by Franz Koch almost 5 years ago

yes please bring back a way to specify a load order independent of any requirements. In my case I have extensions that are used in two different contexts (websites) while on one website they must be loaded after a certain backend module was loaded and in the other context this management module doesn't (must not) exist - so I can't use dependencies but still need to ensure a certain load order. I tried to use the "priority" option, but this is not working anymore. So please add a more flexible way to ensure load orders other than hard dependencies.

#13 Updated by Alexander Opitz over 4 years ago

  • Status changed from Needs Feedback to New
  • Target version changed from 6.2.0 to 7.3 (Packages)

#14 Updated by DANIEL Rémy over 4 years ago

Hi

Another usecase: I have an extension which prepend a hash to every resource url, in order to invalidate easily the client cache.
The rewriting is done in the hook "TSFE->contentPostProc-output", and must be done after merging/compression of another extension which use the same hook array (scriptmerger).

#15 Updated by Stanislas Rolland over 4 years ago

Another use case: Extension MyExt provides a configuration file for realurl. The configuration file contains references to configuration variables that are set by MyExt. Therefore, realurl must be loaded AFTER MyExt is loaded. But the loading order of extensions keeps changing every time an extension is installed/updated.

#16 Updated by Markus Klein over 4 years ago

  • Status changed from New to Accepted
  • Sprint Focus set to Remote Sprint

#17 Updated by Nicole Cordes over 4 years ago

  • Status changed from Accepted to Needs Feedback
  • Assignee set to Nicole Cordes

The only way to influence the order of the extensions is to integrate a helper extension which defines the proper order as own dependencies. Unfortunately there isn't any other way to influence the sorting.

#18 Updated by Markus Klein over 4 years ago

The actual flaw is that the various APIs do not provide a "priority system".

Even if we add a loadBefore/After setting to emconf/composer.json, I see the next reporter coming around the corner telling us that he now has the issue that Ext A says "load me after Ext B, because I add myself to API X" and Ext B says "load me after Ext A, since I use API Y".

The problem is that with "global" loadBefore you may cause other (unsolvable) issues. The only correct way to solve these kinds of issues, is to have a priority per configuration/hook.

Instead of attaching configuration with

$GLOBALS...['somehooking'][] = me::class;

it has to be
$GLOBALS...['somehooking'][] = [
   'handler' => me::class,
   'priority' => 50,
   'maybeSomeOtherOptions => 'snafu'
];

#19 Updated by Markus Klein over 4 years ago

Never the less we have loads of "legacy code" also in the Core, where APIs to provide those priorities. Examples: Extbase SignalDispatcher, almost all hooks, ...

It seems unrealistic to be able to change all those APIs in a short time, so we could at least ease the pain by adding the global loadBefore/*After mechanism for extensions.

#20 Updated by Markus Klein over 4 years ago

  • Assignee deleted (Nicole Cordes)
  • Target version deleted (7.3 (Packages))
  • Sprint Focus deleted (Remote Sprint)

Could it be that this is actually already solved with #57825?

#21 Updated by Markus Klein over 4 years ago

  • Status changed from Needs Feedback to Accepted
  • Assignee set to Markus Klein
  • Target version set to 6.2.13
  • Complexity set to no-brainer
  • Sprint Focus set to Stabilization Sprint

I found out now, why the "suggest" feature didn't work. A mistake in a preparation patch happened: https://review.typo3.org/#/c/30015/3/typo3/sysext/core/Classes/Package/Package.php

I'll fix the bug with this ticket.

#22 Updated by Gerrit Code Review over 4 years ago

  • Status changed from Accepted to Under Review

Patch set 1 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/39692

#23 Updated by Gerrit Code Review over 4 years ago

Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/39692

#24 Updated by Gerrit Code Review over 4 years ago

Patch set 3 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/39692

#25 Updated by Gerrit Code Review over 4 years ago

Patch set 1 for branch TYPO3_6-2 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/39711

#26 Updated by Markus Klein over 4 years ago

  • Status changed from Under Review to Resolved
  • % Done changed from 0 to 100

#27 Updated by Riccardo De Contardi almost 2 years ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF