Bug #21421

slow t3lib_TSparser::parseSub

Added by Bernhard Kraft over 9 years ago. Updated over 8 years ago.

Status:
Closed
Priority:
Must have
Category:
-
Target version:
-
Start date:
2009-11-01
Due date:
% Done:

0%

TYPO3 Version:
4.3
PHP Version:
5.2
Tags:
Complexity:
Is Regression:
Sprint Focus:

Description

There was a report about slow parsing of TS Templates in the dev list containing a description of the problem:
http://lists.typo3.org/pipermail/typo3-dev/2009-October/037545.html

This issue got raised by commit with rev. 5395 which was conducted by me. It should solve bug #17627 "External TypoScripts files will always be cached".

Cause of problem:
When the "procesIncludes" method is called now, it adds an array entry to the $this->rowSum array, for each included file. For each included file an array containing name of the file and last modification time (mtime) is appended to rowSum.

But "procesIncludes" only gets called from "generatedConfig" when finally the TypoScript config gets parsed, and not by the "runThroughTemplates" method, which simply builds up the "rowSum" array to check if a parsed TypoScript can get retrieved from cache.

Solution:
Add a call to "processIncludes" at the end of method "runThroughTemplates". This wont have a big performance impact, as nothing gets parsed. The TypoScript code just gets searched for the "<INCLUDE_TYPOSCRIPT ...>" tag using a regular expression. Also all included files will get loaded and searched for the include-tag and all mtimes will get evaluated. I tried this on my local machine and achieved parse times as usual.

See attached patch against 4.3.0beta2 and TRUNK.

(issue imported from #M12428)

patch_slowTSparser_4.3.0beta2.diff View (489 Bytes) Administrator Admin, 2009-11-01 22:22

patch_slowTSparser_TRUNK.diff View (380 Bytes) Administrator Admin, 2009-11-01 22:23

tstemplate_charset.diff View (3.01 KB) Administrator Admin, 2009-11-18 21:31


Related issues

Related to TYPO3 Core - Bug #17627: External TypoScripts files will always be cached Closed 2007-09-24
Related to TYPO3 Core - Feature #21525: No typoscript template found - Addon Closed 2009-11-10
Related to TYPO3 Core - Bug #21569: Wrong character encoding in cache tables breaks frontend rendering Closed 2009-11-16

History

#1 Updated by Bartosz Aninowski over 9 years ago

works great
thank you

#2 Updated by Bernhard Kraft over 9 years ago

resolved in svn rev. 6365

#3 Updated by Steffen Kamper over 9 years ago

there are problems reported in dev list, therfore i reopened the entry

#4 Updated by Stefan Geith over 9 years ago

Problem: I get 'No TypoScript template found!' in FE with this patch.

This error occurs with all my WebSites (all with 4.3 latest trunk) on my dev-server (Debian lenny, Php5.3).

I always use a plugin, that hold my complete TypoScript and all Templates for a website. So I do it always the same:
Plugin's TS has always something like
lib.myWebsite.defaultPage = PAGE
lib.myWebsite.defaultPage {
typeNum = 0
...
And the only thing in the Root-Template that I insert, is:
page < lib.myWebsite.defaultPage

How to reproduces (on my Testsites):
1. Clear all cache
2. View in FE works: renedered o.k.
3. Reload in FE: allways error 'No TypoScript template found!'

#5 Updated by Stefan Geith over 9 years ago

Btw.:
Removing the call to $this->procesIncludes(); in line 513 fixes the problem for me.

#6 Updated by Stefan Geith over 9 years ago

Additional Note:
Setting
$TYPO3_CONF_VARS['SYS']['useCachingFramework'] = '1';
fixes this issue.

Nevertheless, it should also work with CachingFramework disabled.

#7 Updated by Bernhard Kraft over 9 years ago

You should check what character encoding the included files have, and if there are any umlauts in them.

If there are some, you could try to convert them to utf-8 to see if this resolves your problem.

Of course something has to be done about including iso-8859-1 files when the system is set to UTF-8. I'll try to find a solutin tomorrow.

#8 Updated by Stefan Geith over 9 years ago

OK, I finally found this:

all my (selfmade) plugins do contain a TS-Setup file
ext_typoscript_setup.txt
for the realy basic configurations.
(I know, static-templates are the preferred ones, but I love to have the needed_ basic settings in ext_typoscript_setup.txt ...)

One of these plugins realy had an Umlaut-character and was not saved as utf8.
Changing this file to utf8 resolves the problem.

So I move further discussion to Thread http://bugs.typo3.org/view.php?id=12549 and this one can be closed again.

#9 Updated by Bernhard Kraft over 9 years ago

Hello again,

I just uploaded a file "tstemplate_charset.diff". Try to apply this to the latest version (RC1). In your case it should show an error message (yellow box) telling you that there was an error caching the templates, and you should check the charset of included TypoScript files:

------------
Error while caching TypoScript template!
Check charset of included TypoScript files!
------------

What you can do then is to change the file to utf-8, or set an attribute charset="" in your <INCLUDE_TYPOSCRIPT ...> tag like:

<INCLUDE_TYPOSCRIPT:source="FILE:EXT:tvt_nightshots/typoscript/setup.txt" charset="iso-8859-1">

#10 Updated by Stefan Geith over 9 years ago

Right - I get the mentioned error-message.
But it would be nice to see the path/name of the TS-File (at least the first TS-file that has wrong charset) ...

#11 Updated by Alexander Stehlik over 9 years ago

Thank you for the patch. I tried it, and I have 2 remarks:

After applying the patch I get a fatal error in the backend:
"Fatal error: Call to a member function conv() on a non-object in t3lib/class.t3lib_tsparser.php on line 542"

A simple check if $GLOBALS['TSFE']->csConvObj is null before calling the conf() method solves this problem.

Another problem are the ext_typoscript_setup.txt files. I think for a normal user it can be very confusing when he installs an extension an then gets the charset error message without any more hints.

#12 Updated by Oliver Hader over 9 years ago

Find a patch for this behaviour in issue #21569 (0012613_v2.patch).
In the long term view it would be good to have automatic conversion of external files if they don't match the used character encoding.

Also available in: Atom PDF