Project

General

Profile

Actions

Bug #26993

closed

Pagetree: Copying large branches duplicates branch

Added by Steffen Gebert almost 13 years ago. Updated over 5 years ago.

Status:
Closed
Priority:
Should have
Category:
Backend User Interface
Target version:
-
Start date:
2011-05-23
Due date:
% Done:

100%

Estimated time:
TYPO3 Version:
4.5
PHP Version:
Tags:
Complexity:
Is Regression:
Sprint Focus:

Description

Copying large branches (500+ pages, depending on the server) has a big problem: The AJAX request is answered too late (after Ext.Direct timeout), so Ext.Direct will resend the request. This will copy the branch once again. So the copied branch appears twice after copying.

The whole process fails with a connection timeout error message and the user has to reload the page tree.


Related issues 4 (0 open4 closed)

Related to TYPO3 Core - Bug #44055: ExtJS connection timeout when mass publishing workspace changesClosed2012-12-17

Actions
Related to TYPO3 Core - Bug #49813: Copying large branches creates multiple copiesClosed2013-07-09

Actions
Has duplicate TYPO3 Core - Bug #29123: Duplicate copy created when copying larger page treesClosed2011-08-19

Actions
Has duplicate TYPO3 Core - Bug #32632: Doppelter Seitenbaum nach KopieRejected2011-12-16

Actions
Actions #1

Updated by Alban Cousinie about 12 years ago

I confirm the issue.

This is a real problem, as :
- it doubles the count of records created in the database (imagine when you duplicate a big website with thousand objects).
- also the copy /paste performance is currently very bad, so doubling the database copy operations is likely slowing down even more the operation
- there is no indication of the operation's progress status as stated in Bug #25056 (http://forge.typo3.org/issues/25056) so you don't really know when it's finished
- the page tree doesn't refresh during the operation, and you don't know if you can refresh it manually without compromising the operation (I assume as it is an AJAX based operation, you can refresh the operation, but I don't know it for sure).
- when duplicating large trees, often the operation expires because it takes so long to copy (I really mean several minutes) and PHP likely ends killing the script if the max_execution_time has not been extended to a huge value like 1200 secondes.

Overall, copying large page trees with Typo3 has become a tedious and buggy operation.

Actions #2

Updated by Steffen Gebert about 12 years ago

It didn't get slower with the new Page tree - it only became more unpredictable, what's going on. After 30sec (Ext.Direct timeout) the second request is fired. I already fixed that for the Extension Manager, will try to disable this also for the page tree.

Actions #3

Updated by Alban Cousinie about 12 years ago

Great news, thanks Steffen.

After having tested, I confirm the page tree view can be reloaded without affecting the copy operation.

I have been working since 6 hours on duplicating a separate working copy of that big website I was talking above and I am not close from being done with it. The performance is really poor on that dedicated server I'm running on, which is a 2.5Ghz intel quad core running only this website. Typo3 is currently duplicating folders of tt_address records at a rate of 272 records per minute only. This is extremely low given the horsepower of the server. (I can see however only one core is being used by the copy operation since scripts cannot be threaded inside web pages).

Also I have noticed when duplicating another website, when there are plugins in pages, instead of being copied, they are re-created as references to the source page tree instead of copies. The problem it raises is when you delete the source tree, you get all the plugins in pages erased from the target tree. I don't know if it is a core or a templavoila related issue.

Regards,

Alban

Actions #4

Updated by Alban Cousinie about 12 years ago

Here is the result of another test : copying a single folder full of 1230 tt_adress records is done in 4 seconds. So it copies 307 records / sec, while in recursive mode the rate of copy was 4.5 records / sec

Something is really bizarre in the recursive copy algorithm.

While looking at the tt_address records duplicated using the single folder copy, and those duplicated with the recursive copy I noticed in the first case the IDs of the records contained in the folder are unchanged, so the folder is copied, but the records are the same. In the second case (recursive copy) the records ID have are different, which suggests they are truly duplicated. This should explain the performance difference.

Actions #5

Updated by Gerrit Code Review about 12 years ago

  • Status changed from New to Under Review

Patch set 1 for branch master has been pushed to the review server.
It is available at http://review.typo3.org/8981

Actions #6

Updated by Steffen Gebert about 12 years ago

  • Category set to Backend User Interface
  • Assignee set to Steffen Gebert
Actions #7

Updated by Thorsten Kahler about 12 years ago

  • Subject changed from Pagegree: Copying large branches duplicates branch to Pagetree: Copying large branches duplicates branch
Actions #8

Updated by Gerrit Code Review about 12 years ago

Patch set 2 for branch master has been pushed to the review server.
It is available at http://review.typo3.org/8981

Actions #9

Updated by Gerrit Code Review almost 12 years ago

Patch set 3 for branch master has been pushed to the review server.
It is available at http://review.typo3.org/8981

Actions #10

Updated by Gerrit Code Review almost 12 years ago

Patch set 1 for branch TYPO3_4-7 has been pushed to the review server.
It is available at http://review.typo3.org/11334

Actions #11

Updated by Gerrit Code Review almost 12 years ago

Patch set 1 for branch TYPO3_4-6 has been pushed to the review server.
It is available at http://review.typo3.org/11335

Actions #12

Updated by Gerrit Code Review almost 12 years ago

Patch set 1 for branch TYPO3_4-5 has been pushed to the review server.
It is available at http://review.typo3.org/11336

Actions #13

Updated by Steffen Gebert almost 12 years ago

  • Status changed from Under Review to Resolved
  • % Done changed from 0 to 100
Actions #14

Updated by Xavier Perseguers almost 12 years ago

  • Status changed from Resolved to Under Review
  • % Done changed from 100 to 90
Actions #15

Updated by Steffen Gebert almost 12 years ago

  • Status changed from Under Review to Resolved
  • % Done changed from 90 to 100
Actions #16

Updated by Gerrit Code Review over 11 years ago

  • Status changed from Resolved to Under Review

Patch set 1 for branch TYPO3_4-7 has been pushed to the review server.
It is available at http://review.typo3.org/13146

Actions #17

Updated by Gerrit Code Review over 11 years ago

Patch set 1 for branch TYPO3_4-6 has been pushed to the review server.
It is available at http://review.typo3.org/13262

Actions #18

Updated by Stefan Galinski over 11 years ago

  • Status changed from Under Review to Resolved
Actions #19

Updated by Christian Holzmann over 8 years ago

Hello!

This ist still not solved in TYPO3 6.2.15!

Until a timeout occours, the page will be duplicated over and over...
If if extend max_execution_time to 1200 i get 3...4.. duplicated...
each of them is not completed...

Regards,
Christian Holzmann

Actions #20

Updated by Jan Greth almost 8 years ago

Christian Holzmann wrote:

Hello!

This ist still not solved in TYPO3 6.2.15!

Until a timeout occours, the page will be duplicated over and over...
If if extend max_execution_time to 1200 i get 3...4.. duplicated...
each of them is not completed...

Regards,
Christian Holzmann

Same here with 6.2.19!

Is there any workaround or manual fix to solve this?

Regards,
Jan Greth

Actions #21

Updated by Ulrik H. Kold almost 8 years ago

This is also an issue with v7.6.4 which I'm using here now.

Actions #22

Updated by Michal Cygankiewicz over 7 years ago

Same with 6.2.25!

Actions #23

Updated by Benni Mack over 5 years ago

  • Status changed from Resolved to Closed
Actions

Also available in: Atom PDF