TYPO3 Core

The TS parser is essentially three parts: tokenizer, include / condition tree builder, ast builder.

While it seems we got tokenizer and ast builder pretty right in v12, the include tree is a bit clumsy.

One detail is that the object construct of @import / include_typoscript is a bit unfortunate:

When a snippet contains @import, that snipped is marked isSplit=true. That's good, since "split=true" says that the entire snipped is represented by it's child nodes. Having two lines, one a simple assignment, another one the @import, will then create two child nodes: one for the assignment "segment", one for the import. However, the import is one object. The content of the imported file is set as lineStream, and the @import code line itself is set as "originalLine". This leads to the situation that no child node is generated when an @import does not resolve to a file, which then gives headaches with the syntax scanner looking for invalid imports, as well as the source renderer in the backend modules.

It would be better to split this: An @import / include_typoscript should always create a child node that represents the @import code line. If it does not resolve to something, it should create a "invalidImport" object or similar. The content of the included file(s) should then be child nodes of that node.

Doing this will simplify the overall code, will improve the BE module renderings, and we can get rid of the "original" properties in the include classes, will simplify the code that looks for 'empty' includes, and will streamline the source rendering in the backend modules.