Project

General

Profile

Actions

Feature #86803

closed

Possibility to import other yaml files into a yaml site configuration

Added by Jan Kornblum about 6 years ago. Updated almost 5 years ago.

Status:
Closed
Priority:
Must have
Assignee:
-
Category:
Site Handling, Site Sets & Routing
Start date:
2018-10-30
Due date:
% Done:

100%

Estimated time:
PHP Version:
7.2
Tags:
url,yaml,routing,configuration
Complexity:
Sprint Focus:

Description

Having multi domain instances with same plugins (e.g. news, events etc.) located inside each instance it is currently neccessary to write down the same site configuration (here especially: route enhancers) for each domain. It would be really a nice feature to add the possibility to include / import other yaml (partial) configurations into a yaml site configuration file. Like ckeditor rte configuration does by...

imports:
    - { resource: "EXT:rte_ckeditor/Configuration/RTE/Processing.yaml" }
    - { resource: "EXT:rte_ckeditor/Configuration/RTE/Editor/Base.yaml" }
    - { resource: "EXT:rte_ckeditor/Configuration/RTE/Editor/Plugins.yaml" }

...the same might be possible for site configurations:

imports:
    - { resource: "typo3conf/sites/global/route-enhancers-news.yaml" }
    - { resource: "typo3conf/sites/gloabl/route-enhancers-events.yaml" }

And the referenced files might contain some partial configuration like:

routeEnhancers:
  NewsPlugin:
    type: Extbase
    limitToPages: [182,190,211,309] <- here all news detailpids inside a multidomain environment could be set
    extension: Extension
    plugin: News
    routes:
      - { routePath: '/{news_title}', _controller: 'News::show', _arguments: {'news_title': 'news'} }
    defaultController: 'News::show'
    aspects:
      news_title:
        type: PersistedAliasMapper
        tableName: 'tx_ext_domain_model_news'
        routeFieldName: 'title'

Related issues 3 (0 open3 closed)

Related to TYPO3 Core - Bug #89116: Selective writing of site config broken for nested structuresClosed2019-09-09

Actions
Related to TYPO3 Core - Feature #88742: Allow relative paths in Yaml file importsClosedBenni Mack2019-07-13

Actions
Related to TYPO3 Core - Feature #95307: Allow glob imports in site configuration yaml filesClosed2021-09-21

Actions
Actions #1

Updated by Jan Kornblum about 6 years ago

  • Category set to Site Handling, Site Sets & Routing
Actions #2

Updated by Wouter Wolters about 6 years ago

  • Target version changed from Candidate for patchlevel to Candidate for Major Version
Actions #3

Updated by Felix Nagel almost 6 years ago

According to the blog extension (created by the TYPO3 GmbH), this should already work. See here: https://bitbucket.typo3.com/projects/EXT/repos/blog/browse/Configuration/Routes/Default.yaml

I'm able to confirm this is working for TYPO3 9.5.3.

Actions #4

Updated by Ricky Mathew almost 6 years ago

I just managed to do it by tweaking the load() function of YAML loader class(TYPO3\CMS\Core\Configuration\Loader\YamlFileLoader) in a way it don't affect the configuration writing or reading of yaml files other than site configuration yaml file(config.yaml).see the changes:

   public function load(string $fileName, int $flags = self::PROCESS_PLACEHOLDERS | self::PROCESS_IMPORTS): array
   {
         $content = $this->getFileContents($fileName);
         $content = Yaml::parse($content);
         if (!is_array($content)) {
             throw new \RuntimeException('YAML file "' . $fileName . '" could not be parsed into valid syntax, probably empty?', 1497332874);
         }
+        else{
+           //checking whether it is a routing configuration file of a specific domain.
+           $siteConfiguration = isset($content['rootPageId'])?1:0;
+        }
         if (($flags & self::PROCESS_IMPORTS) === self::PROCESS_IMPORTS) {
-            $content = $this->processImports($content);
+            //patch to avoid writing import file configuration into config.yaml of site configuration
+            if(TYPO3_MODE == 'BE'){
+                //Ensures it's not a routing configuration file.
+                if($siteConfiguration == 0){
+                  $content = $this->processImports($content);
+                }
+            }
+            else{
+               $content = $this->processImports($content);
+            }
         }
         if (($flags & self::PROCESS_PLACEHOLDERS) === self::PROCESS_PLACEHOLDERS) {
              // Check for "%" placeholders
            $content = $this->processPlaceholders($content, $content);
        }

        return $content;
    }

Before this patch i was able to import files into config.yaml as follows:

imports:
  -
    resource: fileadmin/pluginconfig/news.yaml

But the problem is , on saving the configuration via 'sites' module in backend for a specific domain, the configurations in news.yaml file got directly written into config.yaml instead of preserving 'import' directinve in config.yaml since reading and writing of yaml files are dealt with the same load() functionality in YamlFileLoader.

After applying this patch i achieved what i intended.ie preserving 'import' directive in the config.yaml itself(by checking whether the context of loading yaml is front end or backend) even after saving the configuration via sites module.

If this finds helpful and sensible,can anyone push this to typo3 gerrit review?

Actions #5

Updated by Ricky Mathew almost 6 years ago

  • Priority changed from Should have to Must have
  • Target version changed from Candidate for Major Version to Candidate for patchlevel
Actions #6

Updated by Ricky Mathew almost 6 years ago

  • Tags set to url,yaml,routing,configuration
Actions #7

Updated by hhjsfe no-lastname-given almost 6 years ago

I think the Problem is not the YamlFileLoader but the SiteConfiguration. I find for me a temporary solution. I'm override the SiteConfiguration with XCLASS

localconf.php

$GLOBALS['TYPO3_CONF_VARS']['SYS']['Objects'][\TYPO3\CMS\Core\Configuration\SiteConfiguration::class] = array(
    'className' => \Vendor\Ext\Configuration\SiteConfiguration::class
);

Extend SiteConfiguration

<?php
declare(strict_types = 1);

namespace Vendor\Ext\Configuration;

use Symfony\Component\Finder\Finder;
use TYPO3\CMS\Core\Configuration\Loader\YamlFileLoader;
use TYPO3\CMS\Core\Utility\GeneralUtility;

class SiteConfiguration extends \TYPO3\CMS\Core\Configuration\SiteConfiguration
{

    /**
     * Read the site configuration from config files.
     *
     * @return array
     * @throws \TYPO3\CMS\Core\Cache\Exception\NoSuchCacheException
     */
    protected function getAllSiteConfigurationFromFiles(): array
    {
        // Check if the data is already cached
        if (TYPO3_MODE != 'BE' && $siteConfiguration = $this->getCache()->get($this->cacheIdentifier)) {
            // Due to the nature of PhpFrontend, the `<?php` and `#` wraps have to be removed
            $siteConfiguration = preg_replace('/^<\?php\s*|\s*#$/', '', $siteConfiguration);
            $siteConfiguration = json_decode($siteConfiguration, true);
        }
        // Nothing in the cache (or no site found)
        if (empty($siteConfiguration)) {
            $finder = new Finder();
            try {
                $finder->files()->depth(0)->name($this->configFileName)->in($this->configPath . '/*');
            } catch (\InvalidArgumentException $e) {
                // Directory $this->configPath does not exist yet
                $finder = [];
            }
            /** @var YamlFileLoader $loader */
            $loader = GeneralUtility::makeInstance(YamlFileLoader::class);
            $siteConfiguration = [];
            $siteConfigurationCache = [];
            foreach ($finder as $fileInfo) {
                $configuration = $loader->load(GeneralUtility::fixWindowsFilePath((string)$fileInfo), 0);
                $configurationCache = $loader->load(GeneralUtility::fixWindowsFilePath((string)$fileInfo));
                $identifier = basename($fileInfo->getPath());

                $siteConfiguration[$identifier] = $this->handleDeprecated($configuration);
                $siteConfigurationCache[$identifier] = $this->handleDeprecated($configurationCache);;
            }
            $this->getCache()->set($this->cacheIdentifier, json_encode($siteConfigurationCache));
        }
        return $siteConfiguration ?? [];
    }

    /**
     * @param $configuration array
     * @return array
     */
    private function handleDeprecated($configuration)
    {
        if (isset($configuration['site'])) {
            trigger_error(
                'Site configuration with key \'site\' has been deprecated, remove indentation level and site key.',
                E_USER_DEPRECATED
            );
            $configuration = $configuration['site'];

        }
        return $configuration;
    }

}

I render the Yaml File twice. One time for the config.yaml without processing the imports. And the second time for the Cache file with processing the imports.

Actions #8

Updated by Michael Stopp over 5 years ago

I would go as far as calling the current implementation for saving a SiteConfiguration buggy, because it basically deactivates the imports feature of config.yaml. This can't possibly be the intended behaviour...

Actions #9

Updated by Gerrit Code Review about 5 years ago

  • Status changed from New to Under Review

Patch set 4 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/61573

Actions #10

Updated by Gerrit Code Review about 5 years ago

Patch set 5 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/61573

Actions #11

Updated by Gerrit Code Review about 5 years ago

Patch set 6 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/61573

Actions #12

Updated by Gerrit Code Review about 5 years ago

Patch set 7 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/61573

Actions #13

Updated by Gerrit Code Review about 5 years ago

Patch set 8 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/61573

Actions #14

Updated by Gerrit Code Review about 5 years ago

Patch set 9 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/61573

Actions #15

Updated by Gerrit Code Review about 5 years ago

Patch set 1 for branch 9.5 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/61603

Actions #16

Updated by Susanne Moog about 5 years ago

  • Status changed from Under Review to Resolved
  • % Done changed from 0 to 100
Actions #17

Updated by Gerrit Code Review about 5 years ago

  • Status changed from Resolved to Under Review

Patch set 2 for branch 9.5 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/61603

Actions #18

Updated by Andreas Kienast about 5 years ago

  • Related to Bug #89116: Selective writing of site config broken for nested structures added
Actions #19

Updated by Gerrit Code Review about 5 years ago

Patch set 3 for branch 9.5 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/61603

Actions #20

Updated by Susanne Moog about 5 years ago

  • Status changed from Under Review to Resolved
Actions #21

Updated by Benni Mack almost 5 years ago

  • Status changed from Resolved to Closed
Actions #22

Updated by Christian Kuhn almost 2 years ago

  • Related to Feature #88742: Allow relative paths in Yaml file imports added
Actions #23

Updated by Christian Kuhn almost 2 years ago

  • Related to Feature #95307: Allow glob imports in site configuration yaml files added
Actions

Also available in: Atom PDF