TYPO3 Core

Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/51552

Patch set 3 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/51552

Patch set 4 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/51552

Patch set 5 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/51552

Patch set 6 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/51552

Patch set 7 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/51552

Patch set 8 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/51552

Patch set 9 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/51552

Patch set 10 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/51552

Patch set 11 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/51552

See: Detailed analysis of FormHandler, including the effect of this issue on the computational complexity of page-tree copying.

@Kasper Skårhøj: Interesting slides which show that around 40% of processing time is wasted in EXT:solr or XCLASSes...

@Oliver: The take away from the slide-show should be that all the problems found have been solved and merged. The exceptions is this issue and the three other TYPO3 DataHandler issue presented on slide 11. (The slide-show was presented as part of the TYPO3 8 LTS Code Sprint we hosted at our Aarhus offices two weeks ago; Helmut Hummel and Nicole Cordes saw the presentation).

The Solr issue has been fixed in Solr. The xclass is also Solr, since Solr har a prolem; we are in close contact with Timo Hund and dkd and contribute patches to solve the Solr problems (See our 100+ accepted pull requests here, https://github.com/TYPO3-Solr/ext-solr/pulls?utf8=%E2%9C%93&q=is%3Apr%20author%3Athomashohn%20 - it cover the last few months only!).

So you can disregard both the Solr and XClassing problems are solved now.

PS: Also note that the slide-show does not delve into the host of other DataHandler and RefIndexer issues we have solved and contributed over the last 3-4 months; you can browse our recent contributions here: https://review.typo3.org/#/q/owner:thomas%2540hohn.dk+branch:master+OR+owner:%22Christer+%253Cchrisv10%2540gmail.com%253E%22+branch:master+OR+owner:%22Claus+Due+%253Cclaus%2540phpmind.net%253E%22+branch:master - we would greatly appreciate if you also find time to review some of the still-not-merged issues in this list :-)

@Kasper Skårhøj, thanks for the information and I appreciate your dedication and contribution for TYPO3. Especially when DataHandler as low-level boss-class is involved we need to be very careful since only tiny adjustments could lead to inconsistent data and relations. I'm aware of the fact, that the DataHandler is not optimal in terms of software architecture, that's why I was working on event sourcing in the 2nd and 3rd quarter of 2016 (https://www.slideshare.net/ohader/typo3-event-sourcing, e.g. slide 22 shows nested calls for copying a record). But introducing that is nothing that could be done on the fly and not for CMS 8 anymore.

For the current issue, we need to clear what currently happens, what should happen and what the patch currently does.

The problems you ran into are caused by the remapStack in DataHandler that triggers the hook (if registered) for each relation field that contains at least one new reference (a record that has been copied during the process). I consider this as a bug since the calling the hook should happen for a complete record that has been modified once and not for each field.

Combining the hook calls for all records of the same table would be a new feature and lead to a new hook - however, the current implementation of the patch behaves differently depending on being called inside or outside of the remapStack method.

From my point of view, this issue should concentrate on combining the calls to the hook inside remapStack without introducing a new hook or any new behavior. I can support with writing functional tests to assert the the concerning hook of this issue is called correctly with the expected data.

Patch set 12 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/51552

Patch set 13 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/51552

@Oliver: I believe Thomas Hohn with patch set 13 accomodates the concerns you raised, i.e. only fixing the performance bug without introducing new or changed behaviour. To me the code looks correct, but I think that only you and Thomas has the required deep knowledge of DataHandler to be able to truly verify.

Also, we (and TYPO3) would greatly appreciate if you find the time to do some functional tests for this part of TYPO3; we would be prepared to compensate you for your time, if done before TYPO3 8 LTS.

Patch set 14 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/51552

Patch set 15 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/51552

Patch set 16 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/51552

Patch set 1 for branch TYPO3_7-6 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/52257