Project

General

Profile

Actions

Bug #75399

closed

Extbase: Class/Table Mapping doesn't work sometimes

Added by Christian Huppert about 8 years ago. Updated almost 2 years ago.

Status:
Closed
Priority:
Must have
Assignee:
Category:
Extbase
Target version:
-
Start date:
2016-04-04
Due date:
% Done:

0%

Estimated time:
TYPO3 Version:
9
PHP Version:
7.3
Tags:
Complexity:
Is Regression:
No
Sprint Focus:

Description

TYPO3: 6.2.19
PHP-Version: 5.4.x
SQL-Version: 5.5.x
Powermail-Version: 2.17.1 (last version working with 5.4.x)

Hi,

sometimes it happens on our Production instance that a page where a Powermail form is located is not visible as the "Oops an error occured" page is shown up.

If you investigate the corresponding issue on https://forge.typo3.org/issues/70024 you will find out that the cause for this is that Extbase did not correctly map the tables
like setup in config.tx_extbase of the powermail extension.

If one deletes all caches then the mapping is correct and the page displays properly again until after some time (hours or days or weeks) in future the error reappears again.

Also it seems that the error only appears in Production mode. I never had it on my dev environment.

Please do not think that it is a powermail issue. It is an issue of the Extbase class/table mapping procedure.

Maybe this could be also of interest to you:

Stefan Gruber says in the forge ticket above the following:

I could reproduce the issue if this fails in DataMapFactory.php*

94: $frameworkConfiguration = $this->configurationManager->getConfiguration(\TYPO3\CMS\Extbase\Configuration\ConfigurationManagerInterface::CONFIGURATION_TYPE_FRAMEWORK);

Definitely not a powermail issue!

* He means: ./typo3/sysext/extbase/Classes/Persistence/Mapper/DataMapFactory.php

Could you please take some time to investigate this issue?

This would be great :)
Thank you.

Cheers
Christian


Files


Related issues 4 (0 open4 closed)

Related to TYPO3 Core - Bug #78091: cf_extbase_datamapfactory_datamap Enties are generated wrong after expiringClosed2016-09-28

Actions
Related to TYPO3 Core - Bug #83232: Table 'tx_extbase_domain_model_filereference' doesn't exist in \/var\/www\/live.ejw-manager.de\/typo3_src-8.7.8\/typo3\/sysext\/extbase\/Classes\/Persistence\/Generic\/Storage\/Typo3DbBackend.php:384Closed2017-12-05

Actions
Has duplicate TYPO3 Core - Bug #77321: Similar Behaviour of mapping error as described in Bug #66952 Closed2016-08-01

Actions
Has duplicate TYPO3 Core - Bug #78292: TYPO3 6.2.27 creates wrong cache filesRejected2016-10-14

Actions
Actions #1

Updated by Christian Huppert about 8 years ago

So today this behaviour arose again. The last time it arose was 1st April 2016.

Actions #2

Updated by Christian Huppert about 8 years ago

I found out that the problem lies in the fact that under not yet found conditions the expire-date in the table "cf_extbase_datamapfactory_datamap" is not updated and if the expiration date is reached the config.tx_extbase for the corresponding identifier is not delivered and hence the extension cannot work and hence the "Oops an error occured" message shows up.

Actions #3

Updated by Muriel le Pair over 7 years ago

TYPO3 6.2.27
PHP 5.4.45

Same issue here.

I have updated a multidomain configuration to TYPO3 6.2.27.
Sometimes the cache files are created incorrect, resulting in a nasty error (the odd thing is that some sites will work and others will produce the following error (all sites use the extension):

Oops, an error occurred!
Table '[DB].tx_sfaccordion_domain_model_ttcontent' doesn't exist
refering to: https://wiki.typo3.org/Exception/CMS/1247602160

The mapping seems to be correct:

#sf: added
    persistence {
        classes {
            SF\SfAccordion\Domain\Model\TtContent {
                subclasses {
                    itemrows = SF\SfAccordion\Domain\Model\Plugins\ItemRows
                }
            }
            SF\SfAccordion\Domain\Model\Plugins\ItemRows {
                mapping {
                    #ctype
                    recordType = sfaccordion_pi1 
                    tableName = tt_content
                    columns {
                        tx_sfaccordion_content.mapOnProperty = rows
                    }
                }
            }
        }
    }

When I go to 'Install' > click on 'Remove all cache', the site works again. But then all of a sudden the error reappears.
So it looks like sometimes the cache is created wrong.

Actions #4

Updated by Muriel le Pair over 7 years ago

I have transfered the site to a server with PHP 5.6.16, the same error occurs. So the bug is not related to the PHP 5.4.
The error seems to occur randomly on a daily base, but mostly in the morning.

Actions #5

Updated by Muriel le Pair over 7 years ago

I can confirm that this bug is releted to https://forge.typo3.org/issues/78091.

The first time the content is created in cf_extbase_datamapfactory_datamap it works fine, the content size is 57,7 KiB.
After about 24 hours the cache expires and the content is recreated, it now has a size of 735 B and non of the sites work anymore until the cache is deleted from the install tool.
Doing so will empty the table, and when the cache is created for the first time all works fine.

Actions #6

Updated by Muriel le Pair over 7 years ago

I found the cause and a temporary solution.

I have a multiple site setup:
- site 1
- site 2
- site 3
- site 4

Site 1 uses extensions A,C, site 2 uses extensions A,B,C,D, site 3 uses extension A,E,F,G etc etc.
When the cache expires and the first site is called before the others, this will trigger the error.

The sollution is to add all extbase extensions to the first site. Not nessasery to do with all the others.

Actions #7

Updated by Artur Cichosz over 6 years ago

We experienced the same problem described in #6 in a multisite project.

In our case we used a basic EXT:news on site A, but on site B we used an extension to the news called eg. EXT:some_news_extended, which extended some base news subclasses.
So the extbase property map for eg. the model GeorgRinger\News\Domain\Model\News would be different for both sites because we extended a subclass GeorgRinger\News\Domain\Model\NewsDefault.
Now if we hit site A first, its mapping will be cached under the identifier GeorgRinger\News\Domain\Model\News.
Site B looks for the model GeorgRinger\News\Domain\Model\News as well and it gets the cached version from site A which references the wrong subclass (base GeorgRinger\News\Domain\Model\NewsDefault) which in turn lacks all the new properties from our extended subclass.

Two sollutions came in my mind after I discovered what's going on:

1) Let the PropertyMapFactory create distinct cache identifiers according to the active site tree we are on in frontend. But what about the backend? Will it cause follow up issues there?

2) Disable property map cache by $GLOBALS['TYPO3_CONF_VARS']['SYS']['caching']['cacheConfigurations']['extbase_datamapfactory_datamap']['backend'] = 'TYPO3\CMS\Core\Cache\Backend\NullBackend'. This will slow down the entire system of course, but it is a quick workaround to make the sites work properly.

I will go now for solution 2) mybe somebody can give some hints how to achieve 1)

Actions #8

Updated by Susanne Moog about 6 years ago

  • Category set to Extbase
Actions #9

Updated by Daniel Haupt about 6 years ago

We are experiencing the same issue with Typo3 8 LTS and PHP 7.2.

Actions #10

Updated by Alexander Opitz about 6 years ago

  • Related to Bug #83232: Table 'tx_extbase_domain_model_filereference' doesn't exist in \/var\/www\/live.ejw-manager.de\/typo3_src-8.7.8\/typo3\/sysext\/extbase\/Classes\/Persistence\/Generic\/Storage\/Typo3DbBackend.php:384 added
Actions #11

Updated by Jan Radecker almost 6 years ago

  • TYPO3 Version changed from 6.2 to 8
  • PHP Version changed from 5.4 to 7.0

We have the same problem with TYPO3 8.7 and Php 7.0.
Sometimes it loses the mappig configuration for the extended fe_user table.

Actions #12

Updated by Sander Leeuwesteijn almost 6 years ago

We are experiencing the same issue on 6.2.31! (PHP 5.6.35)

We also have an extended fe_user model (with mapping in TS) and sometimes when the cache entry expires for this model in cf_extbase_datamapfactory_datamap the entry BLOB is generated much smaller. And the object only contains the uid. (stuff like username etc is missing)

This causes the site to malfunction and obviously stuff like logging in with an fe_user does not work.

We are still looking for the root cause. (We were using igbinary for serialization and are moving back to regular php serialization, but not sure if this is the root cause)

Actions #13

Updated by Sander Leeuwesteijn almost 6 years ago

It was not caused by igbinary.

Further debugging shows that when this happens:

sysext/extbase/Classes/Persistence/Generic/Mapper/DataMapFactory.php method buildDataMapInternal around line 94:

$frameworkConfiguration = $this->configurationManager->getConfiguration(\TYPO3\CMS\Extbase\Configuration\ConfigurationManagerInterface::CONFIGURATION_TYPE_FRAMEWORK);

returns only a small, almost empty array:

Array
(
    [persistence] => Array
        (
            [storagePid] => 0
        )

    [controllerConfiguration] => Array
        (
        )

)

which obviously is wrong, it should contain the mapping information of many extensions.

Trying to find why this happens..

Actions #14

Updated by Sander Leeuwesteijn almost 6 years ago

I don't have a real way to reproduce this and run it through the debugger (would probably find it very quickly then)

But i do have a way of running uncached on production which causes the error alot, and i added some logging to the DataMapFactory, getting closer.

Just found that this can only happen if typoscript isnt available, and when it happens, the value of $GLOBALS['TSFE']->tmpl->setup is only:

Array
(
    [styles.] => Array
        (
            [insertContent] => CONTENT
            [insertContent.] => Array
                (
                    [table] => tt_content
                    [select.] => Array
                        (
                            [orderBy] => sorting
                            [where] => colPos=0
                            [languageField] => sys_language_uid
                        )

                )

        )

    [config.] => Array
        (
            [extTarget] => _top
            [uniqueLinkVars] => 1
        )

)

This should be fetched in: sysext/extbase/Classes/Configuration/AbstractConfigurationManager.php line 191:

$setup = $this->getTypoScriptSetup();

Trying to find why it's not available now..

Actions #15

Updated by Sander Leeuwesteijn almost 6 years ago

I have found the source of these problems in this particular website. It has many custom made extensions. It occured in 2 cases:

1) In one extension the DataMapper was manually called, providing the class name with a leading \ in front of it. Which is not how it appears in the TSFE array of typoscript, the key name there is without leading \.

2) When an extension manually creates a new TSFE object, make sure it is a complete one. There was an extension here which created a new TypoScriptFrontendController and manually called some methods like:

$GLOBALS['TSFE']->initFEuser();
$GLOBALS['TSFE']->determineId();
$GLOBALS['TSFE']->initTemplate();

How ever it did not call $GLOBALS['TSFE']->getConfigArray(); so there was no TS. When you have extensions which manually build a new TSFE environment, (for example solr does this), make sure all the calls are done and the TSFE is a complete one before you call any Extbase methods like findByUid.

It does remain strange why on some occasions it does work fine and on others it does not. But i have installed some logging in sysext/extbase/Classes/Persistence/Generic/Mapper/DataMapFactory.php and the problem does not seem to occur anymore. For now ;)

I have attached the modified core file with the logging, should anyone be interested. See the // LOGGING START comments.

Actions #16

Updated by Sander Leeuwesteijn almost 6 years ago

Found another cause, watch out for core hooks which are too early in the rendering process, mainly before $TSFE->getConfigArray(); in index_ts.php.

If you use hooks before that and use Extbase style queries in them, TS is not loaded yet and the Extbase mapper will fail.

Actions #17

Updated by Sander Leeuwesteijn almost 6 years ago

Last update, the best place to find how this is happening is adding logging in sysext/extbase/Classes/Configuration/AbstractConfigurationManager.php, modify method getExtbaseConfiguration() to be something like: (and add a log method ofc)

/**
     * Returns the TypoScript configuration found in config.tx_extbase
     *
     * @return array
     */
    protected function getExtbaseConfiguration() {
        $setup = $this->getTypoScriptSetup();
        $extbaseConfiguration = array();
        if (isset($setup['config.']['tx_extbase.'])) {
            $extbaseConfiguration = $this->typoScriptService->convertTypoScriptArrayToPlainArray($setup['config.']['tx_extbase.']);
        } else {
            $this->log('config tx_extbase. was empty!!'.PHP_EOL.print_r(debug_backtrace(2), true));
        }
        return $extbaseConfiguration;
    }

public function log($msg) {
        @file_put_contents(PATH_site . 'typo3log/errors/'.date('Ym').'_ConfigManager.log', '['.date('d-m-y H:i:s').'] - '.$GLOBALS['UNIQUE_REQUEST_ID'].' - '.$msg.PHP_EOL, FILE_APPEND);
    }

With the backtrace generated there you should be able to pinpoint where it's coming from.

To generate more hits on your site to reproduce it, disable the cache for the mapper first:

$GLOBALS['TYPO3_CONF_VARS']['SYS']['caching']['cacheConfigurations']['extbase_datamapfactory_datamap']['backend'] = 'TYPO3\\CMS\\Core\\Cache\\Backend\\NullBackend';

Actions #18

Updated by Steffen Wargalla over 5 years ago

Thank you so Sander for this excellent bug tracking! This helps a lot. Much appreciated!
I was able to reproduce this issue and confirm that the real cause of this is the empty Typoscript template.

Actions #19

Updated by Frank Krüger over 4 years ago

  • TYPO3 Version changed from 8 to 9
  • PHP Version changed from 7.0 to 7.2

We have the same issues with TYPO3 8 and 9 LTS, especially when using mapped tables via TS (persistence.classes.<model>.mapping.tableName).

Actions #20

Updated by Jan Kornblum over 4 years ago

Same issues here with mapped tables (fe_users in this case). We try to restore domain objects after serialization by implementing \Serializable interface to the outer class. On unserialization, the objects are fetched back from repository, which works fine for regular (not mapped) objects. For mapped objects instead, the typoscript mapping isn't respected.

class Test implements \Serializable {

    public function unserialize($serialized) {
        $unserialized = unserialize($serialized);
        $objectManager = GeneralUtility::makeInstance(ObjectManager::class);
        // Works fine...
        $this->project = $objectManager->get(ProjectRepository::class)->findByUid($unserialized['project']);
        // Fails, tries to ask db.tx_myext_domain_model_employee instead of db.fe_users
        $this->employee = $objectManager->get(EmployeeRepository::class)->findByUid($unserialized['employee']);;
    }

}
Actions #21

Updated by Matthias Toscanelli over 4 years ago

I have the same problem with one of my sites. I have some calls made via EID.
The script the "DataMapper" to map the objects.
If a call to the script is made before the mapping cache is generated, then it corrupts the cache.
The reason is that the TypoScript of the framework is not loaded and therefore it cannot generate the mapping.

The issue is in DataMapFactory in method buildDataMapInternal (line 114).
We could perhaps detect that the framework is not loaded before calling a mapping and throw an exception so developers will be forced to make sur that the framework is loaded before calling a script that require data mapping.

To reproduce the problem:
  • This will work only with entity that use a specific mapping (not the standard tx_myextension_domain_model_myentity), because by default DataMapFactory will use this name.
  • Create a category (you need at least an entity)
  • Create a script that load this category from database
    $objectManager = GeneralUtility::makeInstance(ObjectManager::class);
    $objectManager->get(CategoryRepository::class)->findAll();
    
  • Configure "EID" to load this script or any kind of alternative that don't load TSFE configuration
  • Clear all the caches
  • Call the script
  • Call a page that require also the "Category" entity, and you will get "Table 'tx_extbase_domain_model_category' doesn't exist

Matthias

Actions #22

Updated by Alexander Vogt over 4 years ago

I have a similar problem with a middleware that uses extbase models. My middlware sometimes throws exceptions because of missing extbase mapping configurations. The error occurs when a cache entry in the cache "cf_extbase_datamapfactory_datamap" is deleted by the redis garbage collector (or something else). In order to fix the error, all caches have to be cleared.

Steps to reproduce
  • Create a middleware with configuration "after: typo3/cms-frontend/page-argument-validator" to ensure tsfe loading
  • Add a extbase query that relies on a mapping typoscript configuration to the middleware. Example config: config.tx_extbase.persistence.classes.Demo\Domain\Model\MyModel.mapping.tableName = demotablename
  • Call the middleware in the frontend: Everything works.
  • Clear the mapping cache (cf_extbase_datamapfactory_datamap) with the identifier "Demo\Domain\Model\MyModel" manually (to simulate the redis garbage collector)
  • Call the middleware in the frontend: In this case the TypoScript setup ($GLOBALS['TSFE']>tmpl>setup) is empty and the missing datamap entry will be recreated without the typoscript mapping (DataMapFactory->buildDataMapInternal) and this leads to a extbase mapping error. To fix the error, all caches have to be cleared.
Actions #23

Updated by Paul Beck about 4 years ago

I suspect this behavior will also be caused by alot of custom Routing-Aspects written for extensions Slugs. If match id2alias or alias2id and use Extbase repositories for those functions, it may also blows up your Datamap as described middleware problem from Alexander Vogt.

Actions #24

Updated by Alexander Grein about 4 years ago

I had realized the same problem after updating a page from 8.7 (using realurl) to 9.5 (using typo3 route enhancer).

After this update the speaking link generation of ext:calendarize fails after a while, because of a missing table.
The cause is, that I used an extended event model with a table mapping defined in ext_typoscript_setup.txt.

The correct mapping gets lost after a while (maybe if the datamapfactory cache expires?) and will be generated wrong again.

As far as I understand the code, if the DatamapFactory::buildDataMap didn't find a (valid) datamap, it calls the DatamapFactory::buildDataMapInternal method.

Because of the already by Sander Leeuwesteijn descripted problem (see https://forge.typo3.org/issues/75399#note-13), since the following call inside DatamapFactory::buildDataMapInternal

$frameworkConfiguration = $this->configurationManager->getConfiguration(\TYPO3\CMS\Extbase\Configuration\ConfigurationManagerInterface::CONFIGURATION_TYPE_FRAMEWORK);

only returns a small, almost empty array, the tableName gets not overriden by the mapping definition and subsequently all slug generations fails with an exception.

Meanwhile I thought I found a workaround, but it's not working :-(

My idea was to add the missing template service by changing the TYPO3\CMS\Extbase\Configuration\FrontendConfigurationManager::getTypoScriptSetup() method to the following:

    /**
     * Returns TypoScript Setup array from current Environment.
     *
     * @return array the raw TypoScript setup
     */
    public function getTypoScriptSetup()
    {
        if (!$GLOBALS['TSFE']->tmpl instanceof TemplateService) {
            // template service is not yet initialized -> do it
            $GLOBALS['TSFE']->tmpl = $this->objectManager->get(TemplateService::class);
            $GLOBALS['TSFE']->tmpl->generateConfig();
        }

        return $GLOBALS['TSFE']->tmpl->setup;
    }

Unfortunately this would fix this problem, but produce others in the end.

After digging deeper into this, I think the main reason for all of this problems is, that the TYPO3\CMS\Extbase\Middleware\TypoScriptFrontendInitialization::process(), which should initialize the typoscript frontend object, did not get any params over GPs if its called by the slug resolving routine:

$GLOBALS['TSFE'] = GeneralUtility::makeInstance(
            TypoScriptFrontendController::class,
            null,
            GeneralUtility::_GP('id'),
            GeneralUtility::_GP('type'),
            null,
            GeneralUtility::_GP('cHash'),
            null,
            GeneralUtility::_GP('MP')
        );

Because of this missing information or the fact that the TYPO3\CMS\Extbase\Middleware\PrepareTypoScriptFrontendRendering::process() will never be reached in this case, the typoscript frontend controller is not fully initialized.
But it has to be, otherwise the later call
$frameworkConfiguration = $this->configurationManager->getConfiguration(\TYPO3\CMS\Extbase\Configuration\ConfigurationManagerInterface::CONFIGURATION_TYPE_FRAMEWORK);

will not work as aspected.

This all is a chicken or the egg problem.

Actions #25

Updated by Alexander Schnitzler about 4 years ago

  • Assignee set to Alexander Schnitzler
Actions #26

Updated by Ralf Hettinger about 4 years ago

Further debugging leads to the following scenario for me with an explanation why it actually might fail and a workaround:

1 Instantiation of TSFE

TSFE instantiated $cached (with $no_cache = false):

$tsFe = GeneralUtility::makeInstance(
  TypoScriptFrontendController::class, 
  null,
  0,
  0,
  $no_cache = false
);

2 Cache hit

Cache hit ($tsFe->config is set)

$tsFe->getFromCache();

Afterwards, the config array is built:

$tsFe->getConfigArray();

Since there has been a cache hit before, $tsFe->config is already set by $tsFe->getFromCache(). Therefore $tsFe->tmpl->start() is not called by $tsFe->getConfigArray() and thus $GLOBALS['TSFE']->tmpl->setup of TemplateService is not set (even though it could have been set by the cached data).
Imo, the initial thought was: if TSFE is coming from Cache (cache hit), then there is no need for TypoScript to be set within the TemplateService.

3 Extbase

Extbase currently needs the TypoScript via TemplateService; a call against Extbase's getTypoScriptSetup may return an empty TS template:

    /**
     * Returns TypoScript Setup array from current Environment.
     *
     * @return array the raw TypoScript setup
     */
    public function getTypoScriptSetup()
    {
        return $GLOBALS['TSFE']->tmpl->setup;
    }

4 Workaround

This workaround works for me:

    /**
     * Returns TypoScript Setup array from current Environment.
     *
     * @return array the raw TypoScript setup
     */
    public function getTypoScriptSetup()
    {
        $tsFe = &$GLOBALS['TSFE'];
        // TypoScript template was not marked as loaded
        if (!$tsFe->tmpl->loaded) {
            // try to load the TypoScript template
            $tsFe->tmpl->start($tsFe->rootLine);
        }
        return $tsFe->tmpl->setup;
    }

Actions #27

Updated by Johannes Seipelt almost 4 years ago

I have the same issue as described (properties of subclasses emtpy ~1 hour after cache cleared) in two projects I upgraded from 8 LTS to 9 LTS. Current workaround for 9:

https://github.com/n3amil/extbase/commit/c1085101b0d4c80a079e413df82428c69c6b17ec.patch

The bug also occurred in 8, workaround is the same.

For 8:

Index: typo3/sysext/extbase/Classes/Persistence/Generic/Mapper/DataMapFactory.php
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- typo3/sysext/extbase/Classes/Persistence/Generic/Mapper/DataMapFactory.php    (date 1513872003482)
+++ typo3/sysext/extbase/Classes/Persistence/Generic/Mapper/DataMapFactory.php    (date 1513871962000)
@@ -240,6 +240,7 @@
      */
     protected function getColumnsDefinition($tableName)
     {
+        \TYPO3\CMS\Core\Utility\ExtensionManagementUtility::loadExtTables();
         return is_array($GLOBALS['TCA'][$tableName]['columns']) ? $GLOBALS['TCA'][$tableName]['columns'] : [];
     }

Actions #28

Updated by Markus Klein almost 4 years ago

  • Assignee changed from Alexander Schnitzler to Markus Klein

@Ralf I think your workaround is a feasible solution. Would you be so kind as to push a patch to Gerrit?

Actions #29

Updated by Markus Klein almost 4 years ago

  • Status changed from New to Accepted
Actions #30

Updated by Markus Mächler almost 4 years ago

We are encountering the same issue when trying to use a Repository in a Middleware with TYPO3 9. Whenever the cache is empty and the middleware is called before any regular page hit, the cache entry is wrong. Afterwards regular pages do not work either. If however a regular page is called first, the cache is correct and also our middleware works.

The fix from @Ralf did not work for us. We worked around this issue by not using the Extbase ORM in our middleware but fetching the records using the QueryBuilder.

Actions #31

Updated by Markus Klein almost 4 years ago

@Markus: Can you precise which cache is exactly causing the issue?

Whenever the cache is empty and the middleware is called before any regular page hit, the cache entry is wrong.

You mean that "there is no cache entry" not "cache entry is wrong", right? (caches are empty you say as first part of the sentence)

Updated by Markus Mächler almost 4 years ago

@Markus No, to me it really looks like the cache entry for TYPO3\CMS\Extbase\Domain\Model\FileReference in cf_extbase_datamapfactory_datamap is wrong.

If the cache is empty and I call the middleware first, I get the cache entry in cf_extbase_datamapfactory_datamap-content.wrong.bin. Afterwards regular pages (probably only when using an Extbase file reference) do not work.

If however I call a regular page first, I get the cache entry in cf_extbase_datamapfactory_datamap-content.correct.bin.
Afterwards all the pages and also the middleware do work.

Actions #33

Updated by Markus Klein almost 4 years ago

Thanks for clarification. Okay so we simply have missing data (as denoted above already by others) when constructing the datamap, which is cached then.

Actions #34

Updated by Wojciech Proma almost 4 years ago

Hi All!

So I've just encountered the same issue working with Middleware on Typo3 9.5.19. The problem occurs not always and mostly after clearing the cache in Typo3 backend.
When I want to directly use Extbase ORM to get one object via findByUid() I obtain an exception

Table 'db.tx_extbase_domain_model_filereference' doesn't exist

My code on that place is:

/** @var SoldatRepository $repo */
$repo = GeneralUtility::makeInstance(ObjectManager::class)->get(SoldatRepository::class);
/** @var Soldat $soldier */
$soldier = $repo->findByUid($this->soldierUid);

On the line with findByUid() I become an exception with stack trace:

#1472074485 TYPO3\CMS\Extbase\Persistence\Generic\Storage\Exception\SqlErrorException Table 'db.tx_extbase_domain_model_filereference' doesn't exist in <strong>/var/www/html/public/typo3/sysext/extbase/Classes/Persistence/Generic/Storage/Typo3DbBackend.php</strong> line 375

I will try workarounds proposed earlier in this ticket. I would not like to migrate the logic to use the QueryBuilder as I use here as well update() and add...() for subobjects.
Are there any plans to fix it in the next Typo3 Release?

UPDATE
After small research with colleagues in the company, I wrote a small and very very very ugly workaround for this problem. Now the repository is called correctly because TSFE is initiated. The function is called always at the beginning of the Middlware->process():

    /**
     * Creates an instance of TSFE and sets it as a global variable
     *
     * @return void
     * @throws ServiceUnavailableException
     */
    private function initTsfe()
    {
        // very ugly workaround to handle with no TSFE problem in middleware
        $GLOBALS['TSFE'] = GeneralUtility::makeInstance(
            TypoScriptFrontendController::class,
            $GLOBALS['TYPO3_CONF_VARS'],
            GeneralUtility::_GP('id'),
            GeneralUtility::_GP('type'),
            GeneralUtility::_GP('no_cache'),
            GeneralUtility::_GP('cHash'),
            GeneralUtility::_GP('jumpurl'),
            GeneralUtility::_GP('MP'),
            GeneralUtility::_GP('RDCT')
        );
        $GLOBALS['TSFE']->initFEuser();
        $GLOBALS['TSFE']->initUserGroups();
        $GLOBALS['TSFE']->determineId();
        $GLOBALS['TSFE']->initTemplate();
        $GLOBALS['TSFE']->getConfigArray();
    }

Actions #35

Updated by Nigg no-lastname-given over 3 years ago

We also have the same Problem like described in https://forge.typo3.org/issues/75399#note-7
We are using TYPO3 9.5.20 and PHP 7.2

Is there a workaround that doesn't slow down the full system?
Maybe a way to disable extbase_datamapfactory_datamap only for tx_news? (cf_extbase_datamapfactory_datamap)

Actions #36

Updated by Stefan Froemken over 3 years ago

Can we close this issue?

Because in most cases an eID or Middleware containing Extbase operations was called before TSFE.
Or TSFE was build on your own way but missing 1 or 2 further methods to call.
Or you want subclass A for Page 1, but the sane subclass A in extended version for Site 2. Please build your own subclass for Site 2.

I know this issue and it costs me hours to reproduce, but it was always my fault. Don‘t do any Extbase stuff before TSFE.

IMO it‘s not a TYPO3 Issue.

Stefan

Actions #37

Updated by Daniel Corn over 3 years ago

@stefan

I don't agree here. We experienced the problem although our middleware is registered after typo3/cms-frontend/tsfe but before typo3/cms-frontend/prepare-tsfe-rendering

Actions #38

Updated by Jan Kornblum over 3 years ago

I agree the issue should be kept open. In my case domain objects had to be restored after unserialization (see https://forge.typo3.org/issues/75399#note-20).

Actions #39

Updated by Bastian Stargazer over 3 years ago

Daniel Corn wrote:

I don't agree here. We experienced the problem although our middleware is registered after typo3/cms-frontend/tsfe but before typo3/cms-frontend/prepare-tsfe-rendering

Exactly the same here! My middleware sits in between of typo3/cms-frontend/tsfe and typo3/cms-frontend/prepare-tsfe-rendering, and the extension fails because I need some TypoScript configuration but $GLOBALS['TSFE']->tmpl is NULL in v10.4.

Workaround: Move middleware after typo3/cms-frontend/prepare-tsfe-rendering.

Actions #40

Updated by Jacco van der Post over 3 years ago

  • PHP Version changed from 7.2 to 7.3

I have this problem too on extended fe_users. Mapping does irregularly not get read.

Hook loaded in ext_localconf.php to redirect to a personal page after login:

$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['felogin']['login_confirmed']['penauth'] = 'Pen\Penauth\Hooks\FeLogin->redirectToMyPePage';

In the FeLogin hook class:

    
$user = $this->getUserRepository()->findByUid($feUserUid);

Error :

Table 'penned.tx_penauth_domain_model_user' doesn't exist

It does occur every couple times when logging in at the frontend, clearing the cache resolves this problem temporary. I fixed it by using QueryBuilder instead.

Actions #41

Updated by Rostyslav Matviyiv about 3 years ago

The same issue occurs here on our side on TYPO3 8.7.32:

We have some hook ( ['tslib/class.tslib_fe.php']['contentPostProc-output'] ) where we've just added one small Extbase query to retrieve record which contains reference on country model of static_info_tables extension. Country model of static_info_tables has mapping in " typo3conf/ext/static_info_tables/ext_typoscript_setup.txt " to use special table - static_countries .

Now, when we deployed it on the DEV environment, we have noticed that after some time this error occurs:

Table 'typo3.tx_staticinfotables_domain_model_country' doesn't exist

https://jmp.sh/eMSfQGc

If we flush the cache completely the frontend is getting fixed and working as expected.. for some time..

Why it happens?

This was happening when entry of SJBR/StaticInfoTables/Domain/Model/Country class in extbase_datamapfactory_datamap caches was getting expired (entry in cf_extbase_datamapfactory_datamap table in case of TYPO3\CMS\Core\Cache\Backend\ Typo3DatabaseBackend caching framework backend)

Then TYPO3 tried to regenerate it in the extbase/Classes/Persistence/Mapper/DataMapFactory.php->buildDataMapInternal() however now Typoscript was not yet generated and $this->configurationManager->getConfiguration(\TYPO3\CMS\Extbase\Configuration\ConfigurationManagerInterface::CONFIGURATION_TYPE_FRAMEWORK) just didn't return Extbase mappings that are set in typo3conf/ext/static_info_tables/ext_typoscript_setup.txt and thus TYPO3 used standard scheme for mapping the class and tried to use not existing table - tx_staticinfotables_domain_model_country

But if you clean the whole cache (by GUI or CLI) and reload the frontend then at this moment when it comes to the SJBR/StaticInfoTables/Domain/Model/Country class mappings generation TSFE now contains whole typoscript already prepared and Extbase is able to build proper mappings.

The place where code is going differently in those both cases is frontend/Classes/Controller/ TypoScriptFrontendController.php->getConfigArray() , there is the following code line:

if (empty($this->config) || is_array($this->config['INTincScript']) || $this->forceTemplateParsing) {

https://jmp.sh/1PRks2E

And in case when only mapping cache entry is expired the code inside of that condition will not be running (at least before the method in DataMapFactory.php->buildDataMapInternal() is running).
However in case when you flush all the caches and then reload the frontend you will see that code is going inside of that condition and then Typoscript is ready before DataMapFactory.php->buildDataMapInternal() is running.

BTW - we are using ['tslib/class.tslib_fe.php']['contentPostProc-output'] hook where Country model from static_info_tables is requested by extbase query, but it is the hook that is triggered when the page is already rendered, so we always should have already typoscript ready.. but only in case when mappings are expired in extbase_datamapfactory_datamap and TYPO3 tries to regenerate it only then Typoscript is not ready and you getting this issue..

Workaround / Solution

To avoid such issue we've decided to disable auto-expiration of extbase_datamapfactory_datamap caches by setting ( defaultLifetime: 0 ) which will never expire the cache entries in cf_extbase_datamapfactory_datamap so mapping caches will be reloaded only when full cache will be flushed (but GUI on backend or by cli command) but in this case it is working properly (see explanation above)

Actions #42

Updated by Christian Hinreiner almost 3 years ago

Issue also here with TYPO3 7.6.51 and the realurl backend modules.
At least is realurl not broken in the frontend as it doesn't make use of extbase components there.

Actions #43

Updated by Nico de Haen about 2 years ago

Not sure if this error is related to this issue but it looks similar. Happens with TYPO3 10.4.25 randomly and only in production mode:

Core: Exception handler (WEB): Uncaught TYPO3 Exception: #1580056272: The type of property SJBR\StaticInfoTables\Domain\Model\Country::addressFormat could not be identified, as property addressFormat is unknown to the TYPO3\CMS\Extbase\Reflection\ClassSchema instance of class SJBR\StaticInfoTables\Domain\Model\Country. Please make sure said property exists and that you cleared all caches to trigger a new build of said TYPO3\CMS\Extbase\Reflection\ClassSchema instance. | TYPO3\CMS\Extbase\Persistence\Generic\Mapper\Exception\NonExistentPropertyException thrown in file /~/typo3_src-10.4.25/typo3/sysext/extbase/Classes/Persistence/Generic/Mapper/DataMapper.php in line 258.

sometimes the missing property is SJBR\StaticInfoTables\Domain\Model\Language::getIsoCodeA2()

Static Info Table version is 6.9.5.

Needless to say that SJBR\StaticInfoTables\Domain\Model\Country has a property addressFormat etc.

After clearing all caches it works again.

The recommended workaround to set extbase_datamapfactory_datamap caches defaultLifetime to 0 is not applicable since that cache is not configurable anymore in version 10.4.x

Actions #44

Updated by Markus Klein about 2 years ago

@Nico Nope, that's a bug in static info tables and is fixed with version 6.9.6.

Actions #45

Updated by Riccardo De Contardi almost 2 years ago

Is there still work to be done or can this issue be closed?

Actions #46

Updated by Markus Klein almost 2 years ago

  • Status changed from Accepted to Closed

I didn't experience any issues in v10 anymore. I'm closing this now.

Actions

Also available in: Atom PDF