TYPO3 Core

Hi Dmitry, nice one! :)

• "AUTOLOAD_REGISTRY_FILENAME_BE" could be "FILENAME_BE" since it's already in t3lib_autoload
• line 235: "(?:class|interface))[ \t]*" should be "(?:class|interface))[ \t]+" since at least one space is required
• line 235: class/interface names have to start with an character out of a-z

I'm going to test this the next few days. Do you think we could already integrate this into TYPO3 4.3-alpha1 if it seems to be almost stable?

olly

Hi Dmitry,

after apllying both patches i get error in BE when calling a page with classic page-module:
Fatal error: Class 't3lib_transl8tools' not found in C:\xampp\htdocs\t3\home\t3lib\class.t3lib_div.php on line 4592

vg Steffen

Steffen, this is because autoloading is not enabled in BE yet. I removed all require_once in cms, probably page module got affected accidentally.

You can add the following code in typo3/init.php:

// *********************
// Autoloader
// *********************
require_once(PATH_t3lib . 'class.t3lib_autoload.php');
t3lib_autoload::getInstance()->installAutoloader();

Olly, the patches do not enable autoloading in BE yet. But it is easy to do. I think I will post the third patch here soon. Next we can put it all into action :)

first thanks for working on this
second why didn't you inform Masi and me about it as we signed up for the task initially: #19192 (I thought we talked about communication during T3TD)

now some comments:

• the autoloader (AL) should use the new caching framework so that there's no need for the *.ser files
• use a cache with file backend to store everything in typo3temp/
• the AL should not need to be triggered manually, instead I'd say it should build the registry on its first run
• a registry the way you did it is not necessary for most classes and takes up a lot of memory (at least a lot more then needed) during execution
• the way you scan for classes might be optimized and made more reliable when using PHP's tokenizer instead of the regexes (http://us2.php.net/manual/en/intro.tokenizer.php)
• t3lib/sttdb could happen to not be writable
• if a class implements t3lib_Singleton, there shouldn't be a getInstance method, do it either way

thanks anyways, but there's still room for improvement and simplyfication

Part 3 of the patch enables autoloading for Backend. I removed require_once for classes, which are not always used. But I left require_once for classes that are always in use. This way Backend will be faster.

Steffen, this patch will solve the problem with page module that you have :)

Just one more note: I run SVN version of TYPO3 with this patch for several days now. I do not see any problem so far.

• memory_limit is set to 128M.
• 26 extensions installed (most of them have lots of classes – MVC stuff)
• There 671 classes in the registry for my installation (BE mode, which has more classes than FE). Without typo3conf/ext it is 495 classes.
• 671 classes in registry take 56K (56949 bytes) in serialized format. This should be approximately the same in PHP memory.
• The speed of loading serialized format is extremely fast. Other formats will be slower (tested with plain PHP file and database). I made tests on my local PC, which does not have any load. Storing registry in other places will have bad impact on performance on a real server. I think only memcached can beat it but data still have to be stored serialized there and memcached can erase data leaving us without any autoload data (crash!!!)

Hi Dmitry,

after applying 3rd patch all is working fine ;-)

One question:
You store the classes in t3lib/stddb/autoload_xx.ser
This might conflict with servers using a central installation and symlinks.

Imho it would be better to use typo3conf as it is different for each installation.

Indeed the autoloading is very fast and works proper, good work! This is only a subjective impression as i didn't reviewed code yet, but very promising! Thanks for your work.

It will not conflict :) There are two registries: one is system, which is in the t3lib/stddb/, another is site-wide, it is in the typo3conf/.

System registry is used only if site-wide is not found. It contains classes from t3lib/ and typo3/ and generally used at the first log in to the system.

User will be prompted to create site registry when he logs in. Also site registry will be created automatically each time when an extension is (un)installed. It takes approximately 5 seconds on 2MHz CPU to rebuild registry.

So it is covered :)

Following the discussion about singletons, I removed t3lib_Singleton from the t3lib_autoload (see part 1 version 2).

This is my final work on this patch. The solution is provided, I am switching to other tasks now.

Hi Dmitry,

thx for the explanation. The problem on my site was that the files wasn't created after login, no prompt. After i changed status of an extension in EM the files were created, so i will look why they wasn't created for the first time.

Everything works fine without any error.

Works for me too. I haven't tested performance, though.

Hi! Any news on this great feature?
What is currently missing to put it into the next alpha of TYPO3 4.3?

I sent a status request to Masi last week but didn't get a reply yet either...

"To be or not to be", that is a question...

• integrating the autoloader and file parser ("essentials" patch)
  + changed to fetch file contents with php_strip_whitespace() so save some memory
  + "copied" lang.php to class.language.php
  + fixed bug in builder class - only "ux_" classes were taken into account
• defining typo3/sysext/lang/lang.php as deprecated ("lang" patch)
• cleanup by removing require(_once) statements from core ("cleanup" patch)

Notice: The cleanup patch has some strange behaviour on repatching... it produces some hunk failures, however it's about whitespaces or SVN controlls like "Id: $Id$"...

• the autoloading follow now a naming scheme
  1) t3lib_TCEmain -> /t3lib/class.t3lib_tcamain.php
  2) tx_ttnews_controller_frontController -> .../tt_news/controller/class.tx_ttnews_controller_frontcontroller.php (could also be in the sub-dir classes, as often used)
  3) t3lib_browseLinksHook -> /t3lib/interfaces/interface.t3lib_browselinkshook.php (since no regular class was found in previous check)
• The core come with a prebuild legacy registry file
• Extensions have the possibility to create their own registry with extdeveval but should follow the naming scheme (thus, they won't need the registry file at all)

Problem:
Currently, lots of files get included at each request, without being needed at all (Example: Caching framework). To get rid of this problem, we're introducing the new autoloader here.

Solution:
The autoloader has to deal with quite a lot of different situations:
- core files (tslib, t3lib, but also core files not having these naming conventions)
- classical extensions (tx_myextension_piX, ...)
- new extensions with FLOW3 naming conventions, f.e. all Extbase based extensions
The FLOW3 naming convention is the easiest to implement, because it has very detailed rules where a file implementing a certain class must reside in the filesystem. Unfortunately, no such strict rules exist inside TYPO3 core and classical extensions (example: t3lib_Singleton resides in t3lib/interfaces/class.t3lib_singleton.php).

Thus, the autoloader is divided into two parts:
1) If the class which should be loaded starts with Tx_ (the backported naming convention from FLOW3), we can use a very simple autoloader because of the rigid conventions (implemented in t3lib_autoloader::getClassPathByDirectoryStructure).

2) For classical extensions and the TYPO3 core, we use the "registry" concept which is basically an associative array mapping from class names to fully qualified file names.
Every "classical" extension wanting to use the autoloader needs to ship a file called ext_autoload.php in its root directory (can be autogenerated with extdeveval). The core ships such a file as well, located in t3lib/core_autoload.php.
When the autoloader is started, the core files are automatically added to the registry by loading the t3lib/core_autoload.php file. ext_autoload.php files are loaded on demand when needed.

Resolving a class:
If the autoloader should load a class, it checks if the class starts with Tx_. If yes, it'll use the resolver for the new naming conventions. If no, it looks into the registry and checks if the class name is already in there (This happens in two cases: 1. it is implemented in a core file, 2. it is in an extension which has already been loaded).
If the class is not yet in the registry, then the class name is splitted into its parts and the extension key is resolved. Now, the entries of the ext_autoload.php file of the extension are added to the registry. We maintain a separate cache to speed up this process ($extensionHasAutoloadConfiguration).

Building the ext_autoload.php / core_autoload.php:
We implemented a new extdeveval module to do this. It makes sense to pregenerate it, in order to speed up the autoloader and to make the autoloader as a whole work a lot more predictable. Basically, ext_autoload.php contains metainformation about an extension which will only ever change if the extension is restructured.
The reason why we (currently) use multiple files for the autoloader registry is that typically, we only need very few extensions to render a page. Additional caching can be implemented lateron if we see a need for it.

The autoloader is static to ensure maximum speed.

Notes:
This patch is part of a patch series to implement the autoloader. The patches have been worked out together with Olly, based on work by Dmitry, Ingo, Ingmar, Masi and Jochen.

• integrate autoloading functionality
• enable basic autoloading registry

The extdeveval patch belongs to a different RFC for the accordant extension.

• functionality (rev. 5448)
• enabling the autoloader (rev. 5449)