Bug #54491

Epic #47018: Implement Composer support and clean package manager API

PackageManager doesn't solve/recognize real composer packages

Added by Stefano Kowalke over 8 years ago. Updated almost 6 years ago.

Status:
Rejected
Priority:
Should have
Assignee:
-
Category:
composer
Target version:
-
Start date:
2013-12-18
Due date:
% Done:

0%

Estimated time:
TYPO3 Version:
6.2
PHP Version:
5.4
Tags:
Complexity:
Is Regression:
No
Sprint Focus:

Description

I add this snippet to the PackageStates.php:

'doctrine_dbal' =>
    array (
      'manifestPath' => '',
      'composerName' => 'typo3/doctrine_dbal',
      'state' => 'inactive',
      'packagePath' => 'typo3conf/ext/doctrine_dbal/',
      'classesPath' => 'Classes/',
    ),

and this is how my composer.json looks like:

{
    "name": "typo3/doctrine_dbal",
    "type": "typo3-cms-framework",
    "description": "TYPO3 Core",
    "homepage": "http://typo3.org",
    "license": ["GPL-2.0+"],
    "version": "6.2.0",
    "require": {
        "typo3/cms/core": "*",
        "doctrine/dbal": "2.4.*" 
    },
    "replace": {
        "dbal": "*" 
    }
}

"doctrine/dbal": "2.4.*" is a composer dependency which will downloaded into the vendor folder of the extension.

I have a fresh installation with an empty database. When I start TYPO3 I will get the following error:

Fatal error: Uncaught exception 'UnexpectedValueException' with message 'The package "doctrine_dbal" depends on "doctrine/dbal" which is not present in the system.' in /Volumes/HDD/Users/sok/Development/TYPO3/CMS/core/master/typo3/sysext/core/Classes/Package/PackageManager.php on line 608

( ! ) UnexpectedValueException: The package "doctrine_dbal" depends on "doctrine/dbal" which is not present in the system. in /Volumes/HDD/Users/sok/Development/TYPO3/CMS/core/master/typo3/sysext/core/Classes/Package/PackageManager.php on line 608
Call Stack
#    Time    Memory    Function    Location
1    0.0076    248008    {main}( )    ../Install.php:0
2    66.8783    2631112    TYPO3\CMS\Install\Controller\StepController->execute( )    ../Install.php:134
3    67.0292    3212456    TYPO3\CMS\Install\Controller\StepController->executeSpecificStep( )    ../StepController.php:69
4    67.0460    3237232    TYPO3\CMS\Install\Controller\Action\Step\DatabaseConnect->execute( )    ../StepController.php:90
5    72.9332    3301616    TYPO3\CMS\Install\Controller\Action\Step\DatabaseConnect->executeLoadDoctrineExtension( )    ../DatabaseConnect.php:61
6    76.1516    3323008    TYPO3\CMS\Core\Utility\ExtensionManagementUtility::loadExtension( ??? )    ../DatabaseConnect.php:618
7    76.1517    3323096    TYPO3\CMS\Core\Package\PackageManager->activatePackage( ??? )    ../ExtensionManagementUtility.php:1805
8    76.1518    3323160    TYPO3\Flow\Package\PackageManager->activatePackage( ??? )    ../PackageManager.php:530
9    76.1520    3328552    TYPO3\CMS\Core\Package\FailsafePackageManager->sortAndSavePackageStates( )    ../PackageManager.php:542
10    76.1520    3328600    TYPO3\Flow\Package\PackageManager->sortAndSavePackageStates( )    ../FailsafePackageManager.php:92
11    76.1520    3328808    TYPO3\CMS\Core\Package\PackageManager->sortAvailablePackagesByDependencies( )    ../PackageManager.php:914
12    76.1520    3329176    TYPO3\CMS\Core\Package\PackageManager->getAvailablePackageLoadingOrder( )    ../PackageManager.php:748
13    76.1520    3329408    TYPO3\CMS\Core\Package\PackageManager->buildDependencyGraph( )    ../PackageManager.php:691
14    76.2027    3514248    TYPO3\CMS\Core\Package\PackageManager->buildDependencyGraphForPackages( ??? )    ../PackageManager.php:659


Related issues

Related to TYPO3 Core - Bug #57604: PackageManager is unable to resolve composer PSR-4 namespacesClosed2014-04-03

Actions
Related to TYPO3 Core - Bug #78054: PackageManager doesn't solve/recognize real composer packagesClosed2016-09-23

Actions
#1

Updated by Markus Klein over 8 years ago

Why are you modifying PackageStates.php manually??

#2

Updated by Stefano Kowalke over 8 years ago

Ok, then tell me how to manage them otherwise.

#3

Updated by Markus Klein over 8 years ago

  • Status changed from New to Needs Feedback
  • Target version deleted (6.2.0)
#4

Updated by Thomas Maroschik over 8 years ago

Discussed that in person some minutes ago. Will post a guideline for composer dependencies once the workflow becomes clear. For now it means composer still unsupported.

#5

Updated by Daniel Corn over 8 years ago

I am very interested in this topic. Is 6.2 supposed to allow the installation of external/packagist composer packages?
I am the creator of cundd_composer. The extensions aim is to provide the possibility to use non-TYPO3 composer packages.
Unfortunately the current TYPO3 6.2 version not only seems to not fully support composer - it even makes installations using cundd_composer crash. My extension relies on the composer.json in the extensions root, which is now read by the package manager. If it detects an external dependency an exception is thrown. In 6.1 and below you could simply navigate to the composer backend module and install the dependencies - now there is no way to install the extension which relies on the external composer package.

#6

Updated by Stefano Kowalke over 8 years ago

Hey Daniel,

I can provide some informations which I found out and made my extension with external dependencies working.

TYPO3 doesn't support Composer at the moment - that means external dependencies won't be resolved. To make a composer project with ext deps working you need to create a new folder called Packages at the same level like typo3conf and fileadmin and copy the vendor folder into this folder. Rename "vendor" to "Libraries", delete the PackageStates.php and restart the installation.

.
 Packages
└── Libraries
    ├── autoload.php
    ├── composer
    └── doctrine
├── fileadmin
├── index.php -> typo3_src/index.php
├── php.php
├── typo3 -> typo3_src/typo3
├── typo3_src -> /Volumes/HDD/Users/sok/Development/TYPO3/CMS/core/master
├── typo3conf
├── typo3temp
└── uploads
#7

Updated by Daniel Corn over 8 years ago

Hy Stefano!

Thank you for that information!
This could help, but I'm not sure if I really want to try to write into the root :-S
I think I will change my extension to look for cundd_composer.json instead for composer.json files inside the extensions.

I am disappointed how the next TYPO3 version will handle this. I don't really understand why the composer support is implemented incompletely. Isn't the great thing about composer it's ability to install all kinds of libraries from different sources and soften the borders between PHP projects?

#8

Updated by Ernesto Baschny over 8 years ago

  • Parent task set to #47018
#9

Updated by Daniel Corn over 8 years ago

Hy

If someone is interested: I updated my composer extension to work with 6.2 (currently only on Github, not in TER).
As a notice I wrote the following:

Upgrade to TYPO3 6.2

TYPO3 6.2 will parse the extensions composer.json files and retrieve different information from it. TYPO3's new Package Manager also allows the definition of dependencies. Unfortunately these dependencies are limited to real TYPO3 extension. If you want to install an extension, which requires a non-TYPO3 package in composer.json, the Package Manager tries to resolve this dependency in vain and the installation fails.

To work around this issue cundd_composer will look for the file cundd_composer.json instead of composer.json. For legacy reasons cundd_composer will still use composer.json TYPO3 versions below 6.2.

CunddComposer on Github

#10

Updated by Jacob Rasmussen about 8 years ago

While you might get some mileage by using the Packages/Libraries structure, it is currently not possible to include PSR-4 namespaced packages - so I've created a bug report for this #57604

#11

Updated by Alexander Opitz over 7 years ago

Is there something left over?

#12

Updated by Mathias Schreiber over 7 years ago

  • Status changed from Needs Feedback to Closed

Taken into account in #63909.
The issues are not linked on purpose, so the refactoring ticket does not get bloated with relations.

The new EM will import only composer packages :)

#13

Updated by Patrik Karisch over 7 years ago

Will there be a fix for 6.2? This is an annoying bug, which prevents us from installing extensions with composer.json files, which also requires non Typo3-package.

For example:

One extension has this composer.json file:

{
    "name": "acme/foobar",
    "type": "typo3-cms-extension",
    "require": {
        "doctrine/cache": "~1.4",

        "composer/installers": "~1.0" 
    }
}

The composer/installers enables me to require the extension in my project roots composer.json and it gets installed into typo3conf/ext/foobar correctly. But I can't enable it, because the EM is complaining about doctrine/cache. But the extension needs this require, so the project roots composer.json gets not clutterd with. So this would be enough in the project root:

{
    "name": "acme/project",
    "require": {
        "acme/foobar": "~1.0" 
    }
}

This is a serious blocker to require extensions in the project via composer.

#14

Updated by Tilo Baller over 7 years ago

Fully agree with Patrik.

Why are the TYPO3 6.2 Release Notes claiming something like "Introducing the concept of Package Management brings interoperability with the whole world of Composer based PHP libraries." and "It is now possible to natively load Composer based libraries." if is just not true and hardly restricted?

#15

Updated by Daniel Corn over 7 years ago

Unfortunately Tilo is right. In my opinion TYPO3 has no composer support. Or even worse: it would be better to have no composer integration than the current one.
Does anybody know how things are with version 7?

Anyway I love many parts of TYPO3!

#16

Updated by Patrik Karisch over 7 years ago

If typo3 doesn't support composer it would be no problem, because we use it out of scope of the EM and only activate the composer installed extensions in the EM. The problem is how the EM wanna handle the composer.json of the extensions as addition to the depends contraint in the ext_emconf.php. There it got stuck if your extension relies on non typo3 stuff... :(

#17

Updated by Gerrit Code Review about 7 years ago

  • Status changed from Closed to Under Review

Patch set 1 for branch TYPO3_6-2 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/39055

#18

Updated by Christian Opitz about 7 years ago

Sorry, to reopen this but I needed at least a patch for this in order to get it into my TYPO3 installation. If you need this too, you can add the patch via your composer.json:

{
    "name": "vendor/project",
    "description" : "My TYPO3 project",
    "config": {
        "vendor-dir": "Packages/Libraries" 
    },
    "require": {
        "typo3/cms": "6.2.12",
        "vendor/typo3-cms-patches": "*" 
    },
    "repositories": [
        { "type": "composer", "url": "http://composer.typo3.org/" }
        {
            "type": "package",
            "package": {
                "name": "vendor/typo3-cms-patches",
                "version": "0.0.1",
                "type": "patches",
                "require": {
                    "netresearch/composer-patches-plugin": "~1.0" 
                },
                "extra": {
                    "patches": {
                        "typo3/cms": {
                            ">=6.2.0 <6.2.15": [
                                {
                                    "title": "[BUGFIX] Ignore dependencies on non typo3-cms-extension",
                                    "url": "https://git.typo3.org/Packages/TYPO3.CMS.git/patch/9fe856ac96e6a53fef8277f36a4a80bace6f0ae9",
                                    "sha1": "b56a1c47a67d1596c0bd8270e61c44f8911af425" 
                                }
                            ],
                            ">=6.2.15 <7.0": [
                                {
                                    "title": "[BUGFIX] Ignore dependencies on non typo3-cms-extension",
                                    "url": "https://git.typo3.org/Packages/TYPO3.CMS.git/patch/bf4d8291f4069aec4b90c9cc74fcc6aeaa5aa451",
                                    "sha1": "d90a39aaedce093d2252d56c2236fc6487e15e4d" 
                                }
                            ]
                        }
                    }
                }
            }
        }
    ]
}

If you want, you can close the issue again.

#19

Updated by Helmut Hummel almost 7 years ago

  • Category changed from Extension Manager to 1594
#20

Updated by Gerrit Code Review almost 7 years ago

Patch set 2 for branch TYPO3_6-2 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/39055

#21

Updated by Gerrit Code Review almost 7 years ago

Patch set 3 for branch TYPO3_6-2 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/39055

#22

Updated by Gerrit Code Review almost 7 years ago

Patch set 4 for branch TYPO3_6-2 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/39055

#23

Updated by Gerrit Code Review almost 7 years ago

Patch set 6 for branch TYPO3_6-2 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/39055

#24

Updated by Helmut Hummel over 6 years ago

  • Category changed from 1594 to composer

This is solved in master by completely ignoring composer packages and only looking at extensions

#25

Updated by Patrik Karisch over 6 years ago

Helmut Hummel wrote:

This is solved in master by completely ignoring composer packages and only looking at extensions

And what about 6.2?

#26

Updated by Helmut Hummel over 6 years ago

  • Status changed from Under Review to Rejected

this is solved for master, but will not be fixed in 6.2.

It works nicely, when using Packages/Libraries as vendor dir and when activating the found composer packages manually in PackageStates.php file.

This can even be automated using typo3_console :


    "scripts": {
        "post-update-cmd": [
            "Helhum\\Typo3Console\\Composer\\InstallerScripts::postUpdateAndInstall",
            "[ -f typo3conf/PackageStates.php ] || ./typo3cms install:generatepackagestates" 
        ],
        "post-install-cmd": [
            "Helhum\\Typo3Console\\Composer\\InstallerScripts::postUpdateAndInstall",
            "[ -f typo3conf/PackageStates.php ] || ./typo3cms install:generatepackagestates" 
        ]
    },

prerequisite is a active-packages list in root composer.json:


    "extra": {
        "active-packages": [
            "belog",
            "beuser",
            "context_help",
            "css_styled_content",
            "extra_page_cm_options",
            "felogin",
            "filelist",
            "func",
            "func_wizards",
            "impexp",
            "info",
            "info_pagetsconfig",
            "lowlevel",
            "opendocs",
            "perm",
            "reports",
            "rsaauth",
            "rtehtmlarea",
            "scheduler",
            "setup",
            "tstemplate",
            "viewpage",
            "wizard_crpages",

            "realurl",
            "symfony/expression-language",
            "site_package" 
        ],

    }
#27

Updated by Helmut Hummel over 6 years ago

Patrik Karisch wrote:

And what about 6.2?

As pointed out in the previous comment:

Although it has limitations, it works nicely in 6.2 as well. I use it in several projects.

#28

Updated by Patrik Karisch over 6 years ago

Helmut Hummel wrote:

Patrik Karisch wrote:

And what about 6.2?

As pointed out in the previous comment:

Although it has limitations, it works nicely in 6.2 as well. I use it in several projects.

So you don't care bugfixing your LTS versions? Workarounds are not bugfixes.

#29

Updated by Markus Klein over 6 years ago

Patrik, to my knowledge composer installations were never officially supported by 6.2.

#30

Updated by Patrik Karisch over 6 years ago

Markus Klein wrote:

Patrik, to my knowledge composer installations were never officially supported by 6.2.

Officially, not officially, doesn't matter honestly. Its included in 6.2, so it should be supported and fixed. Especially if a contributor has taken time for it and already made a fix. Why you would reject it?

#31

Updated by Christian Opitz over 6 years ago

Patrik, I'm as well not very happy with the patch being abandoned but it is however still available for application - just include it as shown in my previous comment.

#32

Updated by Ingo Fabbri almost 6 years ago

Christian Opitz wrote:

Sorry, to reopen this but I needed at least a patch for this in order to get it into my TYPO3 installation. If you need this too, you can add the patch via your composer.json:

[...]

If you want, you can close the issue again.

Can you confirm, this is working with the latest TYPO3 6.2.27 too?
I can't get it to work. The patch applies fine, but then the extensions manager goes crazy:

The package "crawler" depends on "typo3/cms" which is not present in the system.

So the patch makes it even worse. Without the patch, the EM just complains about my external libs (like monolog, guzzle, ...).

#33

Updated by Helmut Hummel almost 6 years ago

Patrik Karisch wrote:

Especially if a contributor has taken time for it and already made a fix. Why you would reject it?

Just for the records: It is not, that we didn't want this to be fixed in 6.2, it was just that we did not want to risk breaking existing installations.
So we agreed to keep it like that for 6.2.

Also available in: Atom PDF