Bug #26993

Pagetree: Copying large branches duplicates branch

Added by Steffen Gebert about 10 years ago. Updated over 2 years ago.

Status:
Closed
Priority:
Should have
Category:
Backend User Interface
Target version:
-
Start date:
2011-05-23
Due date:
% Done:

100%

Estimated time:
TYPO3 Version:
4.5
PHP Version:
Tags:
Complexity:
Is Regression:
Sprint Focus:

Description

Copying large branches (500+ pages, depending on the server) has a big problem: The AJAX request is answered too late (after Ext.Direct timeout), so Ext.Direct will resend the request. This will copy the branch once again. So the copied branch appears twice after copying.

The whole process fails with a connection timeout error message and the user has to reload the page tree.


Related issues

Related to TYPO3 Core - Bug #44055: ExtJS connection timeout when mass publishing workspace changesClosed2012-12-17

Actions
Related to TYPO3 Core - Bug #49813: Copying large branches creates multiple copiesNew2013-07-09

Actions
Has duplicate TYPO3 Core - Bug #29123: Duplicate copy created when copying larger page treesClosed2011-08-19

Actions
Has duplicate TYPO3 Core - Bug #32632: Doppelter Seitenbaum nach KopieRejected2011-12-16

Actions
#1

Updated by Alban Cousinie over 9 years ago

I confirm the issue.

This is a real problem, as :
- it doubles the count of records created in the database (imagine when you duplicate a big website with thousand objects).
- also the copy /paste performance is currently very bad, so doubling the database copy operations is likely slowing down even more the operation
- there is no indication of the operation's progress status as stated in Bug #25056 (http://forge.typo3.org/issues/25056) so you don't really know when it's finished
- the page tree doesn't refresh during the operation, and you don't know if you can refresh it manually without compromising the operation (I assume as it is an AJAX based operation, you can refresh the operation, but I don't know it for sure).
- when duplicating large trees, often the operation expires because it takes so long to copy (I really mean several minutes) and PHP likely ends killing the script if the max_execution_time has not been extended to a huge value like 1200 secondes.

Overall, copying large page trees with Typo3 has become a tedious and buggy operation.

#2

Updated by Steffen Gebert over 9 years ago

It didn't get slower with the new Page tree - it only became more unpredictable, what's going on. After 30sec (Ext.Direct timeout) the second request is fired. I already fixed that for the Extension Manager, will try to disable this also for the page tree.

#3

Updated by Alban Cousinie over 9 years ago

Great news, thanks Steffen.

After having tested, I confirm the page tree view can be reloaded without affecting the copy operation.

I have been working since 6 hours on duplicating a separate working copy of that big website I was talking above and I am not close from being done with it. The performance is really poor on that dedicated server I'm running on, which is a 2.5Ghz intel quad core running only this website. Typo3 is currently duplicating folders of tt_address records at a rate of 272 records per minute only. This is extremely low given the horsepower of the server. (I can see however only one core is being used by the copy operation since scripts cannot be threaded inside web pages).

Also I have noticed when duplicating another website, when there are plugins in pages, instead of being copied, they are re-created as references to the source page tree instead of copies. The problem it raises is when you delete the source tree, you get all the plugins in pages erased from the target tree. I don't know if it is a core or a templavoila related issue.

Regards,

Alban

#4

Updated by Alban Cousinie over 9 years ago

Here is the result of another test : copying a single folder full of 1230 tt_adress records is done in 4 seconds. So it copies 307 records / sec, while in recursive mode the rate of copy was 4.5 records / sec

Something is really bizarre in the recursive copy algorithm.

While looking at the tt_address records duplicated using the single folder copy, and those duplicated with the recursive copy I noticed in the first case the IDs of the records contained in the folder are unchanged, so the folder is copied, but the records are the same. In the second case (recursive copy) the records ID have are different, which suggests they are truly duplicated. This should explain the performance difference.

#5

Updated by Gerrit Code Review over 9 years ago

  • Status changed from New to Under Review

Patch set 1 for branch master has been pushed to the review server.
It is available at http://review.typo3.org/8981

#6

Updated by Steffen Gebert over 9 years ago

  • Category set to Backend User Interface
  • Assignee set to Steffen Gebert
#7

Updated by Thorsten Kahler over 9 years ago

  • Subject changed from Pagegree: Copying large branches duplicates branch to Pagetree: Copying large branches duplicates branch
#8

Updated by Gerrit Code Review over 9 years ago

Patch set 2 for branch master has been pushed to the review server.
It is available at http://review.typo3.org/8981

#9

Updated by Gerrit Code Review about 9 years ago

Patch set 3 for branch master has been pushed to the review server.
It is available at http://review.typo3.org/8981

#10

Updated by Gerrit Code Review about 9 years ago

Patch set 1 for branch TYPO3_4-7 has been pushed to the review server.
It is available at http://review.typo3.org/11334

#11

Updated by Gerrit Code Review about 9 years ago

Patch set 1 for branch TYPO3_4-6 has been pushed to the review server.
It is available at http://review.typo3.org/11335

#12

Updated by Gerrit Code Review about 9 years ago

Patch set 1 for branch TYPO3_4-5 has been pushed to the review server.
It is available at http://review.typo3.org/11336

#13

Updated by Steffen Gebert about 9 years ago

  • Status changed from Under Review to Resolved
  • % Done changed from 0 to 100
#14

Updated by Xavier Perseguers about 9 years ago

  • Status changed from Resolved to Under Review
  • % Done changed from 100 to 90
#15

Updated by Steffen Gebert about 9 years ago

  • Status changed from Under Review to Resolved
  • % Done changed from 90 to 100
#16

Updated by Gerrit Code Review almost 9 years ago

  • Status changed from Resolved to Under Review

Patch set 1 for branch TYPO3_4-7 has been pushed to the review server.
It is available at http://review.typo3.org/13146

#17

Updated by Gerrit Code Review almost 9 years ago

Patch set 1 for branch TYPO3_4-6 has been pushed to the review server.
It is available at http://review.typo3.org/13262

#18

Updated by Stefan Galinski over 8 years ago

  • Status changed from Under Review to Resolved
#19

Updated by Christian Holzmann over 5 years ago

Hello!

This ist still not solved in TYPO3 6.2.15!

Until a timeout occours, the page will be duplicated over and over...
If if extend max_execution_time to 1200 i get 3...4.. duplicated...
each of them is not completed...

Regards,
Christian Holzmann

#20

Updated by Jan Greth about 5 years ago

Christian Holzmann wrote:

Hello!

This ist still not solved in TYPO3 6.2.15!

Until a timeout occours, the page will be duplicated over and over...
If if extend max_execution_time to 1200 i get 3...4.. duplicated...
each of them is not completed...

Regards,
Christian Holzmann

Same here with 6.2.19!

Is there any workaround or manual fix to solve this?

Regards,
Jan Greth

#21

Updated by Ulrik H. Kold about 5 years ago

This is also an issue with v7.6.4 which I'm using here now.

#22

Updated by Michal Cygankiewicz almost 5 years ago

Same with 6.2.25!

#23

Updated by Benni Mack over 2 years ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF