Bug #26993
closedPagetree: Copying large branches duplicates branch
100%
Description
Copying large branches (500+ pages, depending on the server) has a big problem: The AJAX request is answered too late (after Ext.Direct timeout), so Ext.Direct will resend the request. This will copy the branch once again. So the copied branch appears twice after copying.
The whole process fails with a connection timeout error message and the user has to reload the page tree.
Updated by Alban Cousinie almost 13 years ago
I confirm the issue.
This is a real problem, as :
- it doubles the count of records created in the database (imagine when you duplicate a big website with thousand objects).
- also the copy /paste performance is currently very bad, so doubling the database copy operations is likely slowing down even more the operation
- there is no indication of the operation's progress status as stated in Bug #25056 (http://forge.typo3.org/issues/25056) so you don't really know when it's finished
- the page tree doesn't refresh during the operation, and you don't know if you can refresh it manually without compromising the operation (I assume as it is an AJAX based operation, you can refresh the operation, but I don't know it for sure).
- when duplicating large trees, often the operation expires because it takes so long to copy (I really mean several minutes) and PHP likely ends killing the script if the max_execution_time has not been extended to a huge value like 1200 secondes.
Overall, copying large page trees with Typo3 has become a tedious and buggy operation.
Updated by Steffen Gebert almost 13 years ago
It didn't get slower with the new Page tree - it only became more unpredictable, what's going on. After 30sec (Ext.Direct timeout) the second request is fired. I already fixed that for the Extension Manager, will try to disable this also for the page tree.
Updated by Alban Cousinie almost 13 years ago
Great news, thanks Steffen.
After having tested, I confirm the page tree view can be reloaded without affecting the copy operation.
I have been working since 6 hours on duplicating a separate working copy of that big website I was talking above and I am not close from being done with it. The performance is really poor on that dedicated server I'm running on, which is a 2.5Ghz intel quad core running only this website. Typo3 is currently duplicating folders of tt_address records at a rate of 272 records per minute only. This is extremely low given the horsepower of the server. (I can see however only one core is being used by the copy operation since scripts cannot be threaded inside web pages).
Also I have noticed when duplicating another website, when there are plugins in pages, instead of being copied, they are re-created as references to the source page tree instead of copies. The problem it raises is when you delete the source tree, you get all the plugins in pages erased from the target tree. I don't know if it is a core or a templavoila related issue.
Regards,
Alban
Updated by Alban Cousinie almost 13 years ago
Here is the result of another test : copying a single folder full of 1230 tt_adress records is done in 4 seconds. So it copies 307 records / sec, while in recursive mode the rate of copy was 4.5 records / sec
Something is really bizarre in the recursive copy algorithm.
While looking at the tt_address records duplicated using the single folder copy, and those duplicated with the recursive copy I noticed in the first case the IDs of the records contained in the folder are unchanged, so the folder is copied, but the records are the same. In the second case (recursive copy) the records ID have are different, which suggests they are truly duplicated. This should explain the performance difference.
Updated by Gerrit Code Review almost 13 years ago
- Status changed from New to Under Review
Patch set 1 for branch master has been pushed to the review server.
It is available at http://review.typo3.org/8981
Updated by Steffen Gebert almost 13 years ago
- Category set to Backend User Interface
- Assignee set to Steffen Gebert
Updated by Thorsten Kahler almost 13 years ago
- Subject changed from Pagegree: Copying large branches duplicates branch to Pagetree: Copying large branches duplicates branch
Updated by Gerrit Code Review over 12 years ago
Patch set 2 for branch master has been pushed to the review server.
It is available at http://review.typo3.org/8981
Updated by Gerrit Code Review over 12 years ago
Patch set 3 for branch master has been pushed to the review server.
It is available at http://review.typo3.org/8981
Updated by Gerrit Code Review over 12 years ago
Patch set 1 for branch TYPO3_4-7 has been pushed to the review server.
It is available at http://review.typo3.org/11334
Updated by Gerrit Code Review over 12 years ago
Patch set 1 for branch TYPO3_4-6 has been pushed to the review server.
It is available at http://review.typo3.org/11335
Updated by Gerrit Code Review over 12 years ago
Patch set 1 for branch TYPO3_4-5 has been pushed to the review server.
It is available at http://review.typo3.org/11336
Updated by Steffen Gebert over 12 years ago
- Status changed from Under Review to Resolved
- % Done changed from 0 to 100
Applied in changeset c35860c62319be89d695c34abf8786b55aa51fbe.
Updated by Xavier Perseguers over 12 years ago
- Status changed from Resolved to Under Review
- % Done changed from 100 to 90
Updated by Steffen Gebert over 12 years ago
- Status changed from Under Review to Resolved
- % Done changed from 90 to 100
Applied in changeset d4da7991eeca06a1028ab701f3decce12f7ad61e.
Updated by Gerrit Code Review over 12 years ago
- Status changed from Resolved to Under Review
Patch set 1 for branch TYPO3_4-7 has been pushed to the review server.
It is available at http://review.typo3.org/13146
Updated by Gerrit Code Review over 12 years ago
Patch set 1 for branch TYPO3_4-6 has been pushed to the review server.
It is available at http://review.typo3.org/13262
Updated by Stefan Galinski about 12 years ago
- Status changed from Under Review to Resolved
Updated by Christian Holzmann about 9 years ago
Hello!
This ist still not solved in TYPO3 6.2.15!
Until a timeout occours, the page will be duplicated over and over...
If if extend max_execution_time to 1200 i get 3...4.. duplicated...
each of them is not completed...
Regards,
Christian Holzmann
Updated by Jan Greth over 8 years ago
Christian Holzmann wrote:
Hello!
This ist still not solved in TYPO3 6.2.15!
Until a timeout occours, the page will be duplicated over and over...
If if extend max_execution_time to 1200 i get 3...4.. duplicated...
each of them is not completed...Regards,
Christian Holzmann
Same here with 6.2.19!
Is there any workaround or manual fix to solve this?
Regards,
Jan Greth
Updated by Ulrik H. Kold over 8 years ago
This is also an issue with v7.6.4 which I'm using here now.