Bug #75399

Extbase: Class/Table Mapping doesn't work sometimes

Added by Christian Huppert about 3 years ago. Updated 11 months ago.

Status:
New
Priority:
Must have
Assignee:
-
Category:
Extbase
Target version:
-
Start date:
2016-04-04
Due date:
% Done:

0%

TYPO3 Version:
8
PHP Version:
7.0
Tags:
Complexity:
Is Regression:
No
Sprint Focus:

Description

TYPO3: 6.2.19
PHP-Version: 5.4.x
SQL-Version: 5.5.x
Powermail-Version: 2.17.1 (last version working with 5.4.x)

Hi,

sometimes it happens on our Production instance that a page where a Powermail form is located is not visible as the "Oops an error occured" page is shown up.

If you investigate the corresponding issue on https://forge.typo3.org/issues/70024 you will find out that the cause for this is that Extbase did not correctly map the tables
like setup in config.tx_extbase of the powermail extension.

If one deletes all caches then the mapping is correct and the page displays properly again until after some time (hours or days or weeks) in future the error reappears again.

Also it seems that the error only appears in Production mode. I never had it on my dev environment.

Please do not think that it is a powermail issue. It is an issue of the Extbase class/table mapping procedure.

Maybe this could be also of interest to you:

Stefan Gruber says in the forge ticket above the following:

I could reproduce the issue if this fails in DataMapFactory.php*

94: $frameworkConfiguration = $this->configurationManager->getConfiguration(\TYPO3\CMS\Extbase\Configuration\ConfigurationManagerInterface::CONFIGURATION_TYPE_FRAMEWORK);

Definitely not a powermail issue!

* He means: ./typo3/sysext/extbase/Classes/Persistence/Mapper/DataMapFactory.php

Could you please take some time to investigate this issue?

This would be great :)
Thank you.

Cheers
Christian

DataMapFactory.php View (20.9 KB) Sander Leeuwesteijn, 2018-05-03 13:22


Related issues

Related to TYPO3 Core - Bug #78091: cf_extbase_datamapfactory_datamap Enties are generated wrong after expiring New 2016-09-28
Related to TYPO3 Core - Bug #83232: Table 'tx_extbase_domain_model_filereference' doesn't exist in \/var\/www\/live.ejw-manager.de\/typo3_src-8.7.8\/typo3\/sysext\/extbase\/Classes\/Persistence\/Generic\/Storage\/Typo3DbBackend.php:384 Closed 2017-12-05
Duplicated by TYPO3 Core - Bug #77321: Similar Behaviour of mapping error as described in Bug #66952 Closed 2016-08-01
Duplicated by TYPO3 Core - Bug #78292: TYPO3 6.2.27 creates wrong cache files Rejected 2016-10-14

History

#1 Updated by Christian Huppert about 3 years ago

So today this behaviour arose again. The last time it arose was 1st April 2016.

#2 Updated by Christian Huppert about 3 years ago

I found out that the problem lies in the fact that under not yet found conditions the expire-date in the table "cf_extbase_datamapfactory_datamap" is not updated and if the expiration date is reached the config.tx_extbase for the corresponding identifier is not delivered and hence the extension cannot work and hence the "Oops an error occured" message shows up.

#3 Updated by Muriel le Pair over 2 years ago

TYPO3 6.2.27
PHP 5.4.45

Same issue here.

I have updated a multidomain configuration to TYPO3 6.2.27.
Sometimes the cache files are created incorrect, resulting in a nasty error (the odd thing is that some sites will work and others will produce the following error (all sites use the extension):

Oops, an error occurred!
Table '[DB].tx_sfaccordion_domain_model_ttcontent' doesn't exist
refering to: https://wiki.typo3.org/Exception/CMS/1247602160

The mapping seems to be correct:

#sf: added
    persistence {
        classes {
            SF\SfAccordion\Domain\Model\TtContent {
                subclasses {
                    itemrows = SF\SfAccordion\Domain\Model\Plugins\ItemRows
                }
            }
            SF\SfAccordion\Domain\Model\Plugins\ItemRows {
                mapping {
                    #ctype
                    recordType = sfaccordion_pi1 
                    tableName = tt_content
                    columns {
                        tx_sfaccordion_content.mapOnProperty = rows
                    }
                }
            }
        }
    }

When I go to 'Install' > click on 'Remove all cache', the site works again. But then all of a sudden the error reappears.
So it looks like sometimes the cache is created wrong.

#4 Updated by Muriel le Pair over 2 years ago

I have transfered the site to a server with PHP 5.6.16, the same error occurs. So the bug is not related to the PHP 5.4.
The error seems to occur randomly on a daily base, but mostly in the morning.

#5 Updated by Muriel le Pair over 2 years ago

I can confirm that this bug is releted to https://forge.typo3.org/issues/78091.

The first time the content is created in cf_extbase_datamapfactory_datamap it works fine, the content size is 57,7 KiB.
After about 24 hours the cache expires and the content is recreated, it now has a size of 735 B and non of the sites work anymore until the cache is deleted from the install tool.
Doing so will empty the table, and when the cache is created for the first time all works fine.

#6 Updated by Muriel le Pair over 2 years ago

I found the cause and a temporary solution.

I have a multiple site setup:
- site 1
- site 2
- site 3
- site 4

Site 1 uses extensions A,C, site 2 uses extensions A,B,C,D, site 3 uses extension A,E,F,G etc etc.
When the cache expires and the first site is called before the others, this will trigger the error.

The sollution is to add all extbase extensions to the first site. Not nessasery to do with all the others.

#7 Updated by Artur Cichosz almost 2 years ago

We experienced the same problem described in #6 in a multisite project.

In our case we used a basic EXT:news on site A, but on site B we used an extension to the news called eg. EXT:some_news_extended, which extended some base news subclasses.
So the extbase property map for eg. the model GeorgRinger\News\Domain\Model\News would be different for both sites because we extended a subclass GeorgRinger\News\Domain\Model\NewsDefault.
Now if we hit site A first, its mapping will be cached under the identifier GeorgRinger\News\Domain\Model\News.
Site B looks for the model GeorgRinger\News\Domain\Model\News as well and it gets the cached version from site A which references the wrong subclass (base GeorgRinger\News\Domain\Model\NewsDefault) which in turn lacks all the new properties from our extended subclass.

Two sollutions came in my mind after I discovered what's going on:

1) Let the PropertyMapFactory create distinct cache identifiers according to the active site tree we are on in frontend. But what about the backend? Will it cause follow up issues there?

2) Disable property map cache by $GLOBALS['TYPO3_CONF_VARS']['SYS']['caching']['cacheConfigurations']['extbase_datamapfactory_datamap']['backend'] = 'TYPO3\CMS\Core\Cache\Backend\NullBackend'. This will slow down the entire system of course, but it is a quick workaround to make the sites work properly.

I will go now for solution 2) mybe somebody can give some hints how to achieve 1)

#8 Updated by Susanne Moog over 1 year ago

  • Category set to Extbase

#9 Updated by Daniel Haupt over 1 year ago

We are experiencing the same issue with Typo3 8 LTS and PHP 7.2.

#10 Updated by Alexander Opitz over 1 year ago

  • Related to Bug #83232: Table 'tx_extbase_domain_model_filereference' doesn't exist in \/var\/www\/live.ejw-manager.de\/typo3_src-8.7.8\/typo3\/sysext\/extbase\/Classes\/Persistence\/Generic\/Storage\/Typo3DbBackend.php:384 added

#11 Updated by Jan Radecker about 1 year ago

  • TYPO3 Version changed from 6.2 to 8
  • PHP Version changed from 5.4 to 7.0

We have the same problem with TYPO3 8.7 and Php 7.0.
Sometimes it loses the mappig configuration for the extended fe_user table.

#12 Updated by Sander Leeuwesteijn about 1 year ago

We are experiencing the same issue on 6.2.31! (PHP 5.6.35)

We also have an extended fe_user model (with mapping in TS) and sometimes when the cache entry expires for this model in cf_extbase_datamapfactory_datamap the entry BLOB is generated much smaller. And the object only contains the uid. (stuff like username etc is missing)

This causes the site to malfunction and obviously stuff like logging in with an fe_user does not work.

We are still looking for the root cause. (We were using igbinary for serialization and are moving back to regular php serialization, but not sure if this is the root cause)

#13 Updated by Sander Leeuwesteijn about 1 year ago

It was not caused by igbinary.

Further debugging shows that when this happens:

sysext/extbase/Classes/Persistence/Generic/Mapper/DataMapFactory.php method buildDataMapInternal around line 94:

$frameworkConfiguration = $this->configurationManager->getConfiguration(\TYPO3\CMS\Extbase\Configuration\ConfigurationManagerInterface::CONFIGURATION_TYPE_FRAMEWORK);

returns only a small, almost empty array:

Array
(
    [persistence] => Array
        (
            [storagePid] => 0
        )

    [controllerConfiguration] => Array
        (
        )

)

which obviously is wrong, it should contain the mapping information of many extensions.

Trying to find why this happens..

#14 Updated by Sander Leeuwesteijn about 1 year ago

I don't have a real way to reproduce this and run it through the debugger (would probably find it very quickly then)

But i do have a way of running uncached on production which causes the error alot, and i added some logging to the DataMapFactory, getting closer.

Just found that this can only happen if typoscript isnt available, and when it happens, the value of $GLOBALS['TSFE']->tmpl->setup is only:

Array
(
    [styles.] => Array
        (
            [insertContent] => CONTENT
            [insertContent.] => Array
                (
                    [table] => tt_content
                    [select.] => Array
                        (
                            [orderBy] => sorting
                            [where] => colPos=0
                            [languageField] => sys_language_uid
                        )

                )

        )

    [config.] => Array
        (
            [extTarget] => _top
            [uniqueLinkVars] => 1
        )

)

This should be fetched in: sysext/extbase/Classes/Configuration/AbstractConfigurationManager.php line 191:

$setup = $this->getTypoScriptSetup();

Trying to find why it's not available now..

#15 Updated by Sander Leeuwesteijn about 1 year ago

I have found the source of these problems in this particular website. It has many custom made extensions. It occured in 2 cases:

1) In one extension the DataMapper was manually called, providing the class name with a leading \ in front of it. Which is not how it appears in the TSFE array of typoscript, the key name there is without leading \.

2) When an extension manually creates a new TSFE object, make sure it is a complete one. There was an extension here which created a new TypoScriptFrontendController and manually called some methods like:

$GLOBALS['TSFE']->initFEuser();
$GLOBALS['TSFE']->determineId();
$GLOBALS['TSFE']->initTemplate();

How ever it did not call $GLOBALS['TSFE']->getConfigArray(); so there was no TS. When you have extensions which manually build a new TSFE environment, (for example solr does this), make sure all the calls are done and the TSFE is a complete one before you call any Extbase methods like findByUid.

It does remain strange why on some occasions it does work fine and on others it does not. But i have installed some logging in sysext/extbase/Classes/Persistence/Generic/Mapper/DataMapFactory.php and the problem does not seem to occur anymore. For now ;)

I have attached the modified core file with the logging, should anyone be interested. See the // LOGGING START comments.

#16 Updated by Sander Leeuwesteijn about 1 year ago

Found another cause, watch out for core hooks which are too early in the rendering process, mainly before $TSFE->getConfigArray(); in index_ts.php.

If you use hooks before that and use Extbase style queries in them, TS is not loaded yet and the Extbase mapper will fail.

#17 Updated by Sander Leeuwesteijn about 1 year ago

Last update, the best place to find how this is happening is adding logging in sysext/extbase/Classes/Configuration/AbstractConfigurationManager.php, modify method getExtbaseConfiguration() to be something like: (and add a log method ofc)

/**
     * Returns the TypoScript configuration found in config.tx_extbase
     *
     * @return array
     */
    protected function getExtbaseConfiguration() {
        $setup = $this->getTypoScriptSetup();
        $extbaseConfiguration = array();
        if (isset($setup['config.']['tx_extbase.'])) {
            $extbaseConfiguration = $this->typoScriptService->convertTypoScriptArrayToPlainArray($setup['config.']['tx_extbase.']);
        } else {
            $this->log('config tx_extbase. was empty!!'.PHP_EOL.print_r(debug_backtrace(2), true));
        }
        return $extbaseConfiguration;
    }

public function log($msg) {
        @file_put_contents(PATH_site . 'typo3log/errors/'.date('Ym').'_ConfigManager.log', '['.date('d-m-y H:i:s').'] - '.$GLOBALS['UNIQUE_REQUEST_ID'].' - '.$msg.PHP_EOL, FILE_APPEND);
    }

With the backtrace generated there you should be able to pinpoint where it's coming from.

To generate more hits on your site to reproduce it, disable the cache for the mapper first:

$GLOBALS['TYPO3_CONF_VARS']['SYS']['caching']['cacheConfigurations']['extbase_datamapfactory_datamap']['backend'] = 'TYPO3\\CMS\\Core\\Cache\\Backend\\NullBackend';

#18 Updated by Steffen Wargalla 11 months ago

Thank you so Sander for this excellent bug tracking! This helps a lot. Much appreciated!
I was able to reproduce this issue and confirm that the real cause of this is the empty Typoscript template.

Also available in: Atom PDF