Bug #35172

Connection timeout when parsing large number of changed records

Added by Michal Krypczyk over 6 years ago. Updated 2 months ago.

Status:
Closed
Priority:
Must have
Assignee:
-
Category:
Workspaces
Target version:
-
Start date:
2012-03-23
Due date:
% Done:

100%

TYPO3 Version:
4.5
PHP Version:
5.3
Tags:
Complexity:
Is Regression:
Sprint Focus:

Description

Connection timeout occurs when large number of modified records is being loaded in Workspaces module.

AJAX timeout (after 30 secods) seems to be the cause.

Some more details:
- workspace (live/draft) doesn't matter
- multiple domains in TYPO3 installation

Alosi the Depth selection is reset to Infinite when another page in pagetree is selected.


Related issues

Related to TYPO3 Core - Bug #30405: Switching Workspaces results in Connection Timeout Closed 2011-09-28
Related to TYPO3 Core - Bug #39731: Calculate percentage of differences if column is shown Closed 2012-08-12
Related to TYPO3 Core - Bug #39727: Calculating percentage of differences is slow Rejected 2012-08-12
Related to TYPO3 Core - Task #44189: Workspaces: Preview all for editors Closed 2012-12-21

Associated revisions

Revision 896668d5 (diff)
Added by Ingo Pfennigstorf over 6 years ago

[BUGFIX] Prevent time-outs when calculating change percentage

Fixes the complex and unnecessary calculation of the change percentage
of a record. This may have led to various timeouts when calculating this
for long strings.

Change-Id: I726236a37b0cf71772befeeabb3f0f18fe86a037
Fixes: #35172
Releases: 6.0
Reviewed-on: http://review.typo3.org/10949
Reviewed-by: Christian Kuhn
Tested-by: Christian Kuhn

Revision 9ead6b3c (diff)
Added by Ingo Pfennigstorf almost 6 years ago

[BUGFIX] Prevent time-outs when calculating change percentage

Fixes the complex and unnecessary calculation of the change percentage
of a record. This may have led to various timeouts when calculating this
for long strings.

Since this is still a nasty issue in older branches too, the
patch that was already released to 6.0 is now backported
to 4.7, 4.6, 4.5

Change-Id: I726236a37b0cf71772befeeabb3f0f18fe86a037
Fixes: #35172
Releases: 4.7, 4.6, 4.5
Reviewed-on: https://review.typo3.org/17932
Reviewed-by: Christian Kuhn
Tested-by: Christian Kuhn

Revision 278a4771 (diff)
Added by Ingo Pfennigstorf almost 6 years ago

[BUGFIX] Prevent time-outs when calculating change percentage

Fixes the complex and unnecessary calculation of the change percentage
of a record. This may have led to various timeouts when calculating this
for long strings.

Since this is still a nasty issue in older branches too, the
patch that was already released to 6.0 is now backported
to 4.7, 4.6, 4.5

Change-Id: I726236a37b0cf71772befeeabb3f0f18fe86a037
Fixes: #35172
Releases: 4.7, 4.6, 4.5
Reviewed-on: https://review.typo3.org/17931
Reviewed-by: Christian Kuhn
Tested-by: Christian Kuhn

Revision 99287a01 (diff)
Added by Ingo Pfennigstorf almost 6 years ago

[BUGFIX] Prevent time-outs when calculating change percentage

Fixes the complex and unnecessary calculation of the change percentage
of a record. This may have led to various timeouts when calculating this
for long strings.

Since this is still a nasty issue in older branches too, the
patch that was already released to 6.0 is now backported
to 4.7, 4.6, 4.5

Change-Id: I726236a37b0cf71772befeeabb3f0f18fe86a037
Fixes: #35172
Releases: 4.7, 4.6, 4.5
Reviewed-on: https://review.typo3.org/17616
Reviewed-by: Christian Kuhn
Tested-by: Christian Kuhn

History

#1 Updated by Jigal van Hemert over 6 years ago

Found the cause: Tx_Workspaces_Service_GridData::calculateChangePercentage() uses similar_text() to calculate the degree of difference between the content. This function is very slow for longer strings (complexity O(N**3); N is length of longest string). For some records the time needed to calculate the change percentage took several seconds. With a few hundred records in a workspace list it would easily cause a time-out.
Limiting the strings to maximum 512 chars solves the time-outs, at least until a faster function is found.

#2 Updated by Ingo Pfennigstorf over 6 years ago

@Jigal That should not be hard to implement. Is there any use case where displaying the percentage really matters that much, that the string length cannot be cropped?

Does it work for you with a limitation to 512 chars?

#3 Updated by Ingo Pfennigstorf over 6 years ago

So, why is this calculation still in workspaces?
With #11585 the display of the percentage was dropped (also see http://lists.typo3.org/pipermail/typo3-team-workspace/2011-January/000184.html
So I think that this calculation is obsolete and I'm going to write a patch for this, or did I miss something?

#4 Updated by Gerrit Code Review over 6 years ago

  • Status changed from New to Under Review

Patch set 1 for branch master has been pushed to the review server.
It is available at http://review.typo3.org/10949

#5 Updated by Gerrit Code Review over 6 years ago

Patch set 2 for branch master has been pushed to the review server.
It is available at http://review.typo3.org/10949

#6 Updated by Joe Schäfer over 6 years ago

Gerrit Code Review wrote:

Patch set 2 for branch master has been pushed to the review server.
It is available at http://review.typo3.org/10949

Looks good to and works fine for me.
Fixed my issues with "connection problems".

I will report if errors/bugs occur.

#7 Updated by Gerrit Code Review over 6 years ago

Patch set 3 for branch master has been pushed to the review server.
It is available at http://review.typo3.org/10949

#8 Updated by Ingo Pfennigstorf over 6 years ago

  • Status changed from Under Review to Resolved
  • % Done changed from 0 to 100

Applied in changeset commit:4141276eb63ced02f048149c7560cf883aae89be.

#9 Updated by Jean-Sébastien Gervais almost 6 years ago

Sorry to revive a closed bug, but can the patch be backported to TYPO3 4.5 LTS as well?

#10 Updated by Gerrit Code Review almost 6 years ago

  • Status changed from Resolved to Under Review

Patch set 1 for branch 4.5 has been pushed to the review server.
It is available at https://review.typo3.org/17616

#11 Updated by Dan Untenzu almost 6 years ago

Hey, since I need this in a 4.5 version as well, I tried to make a backport of Ingos patch and pushed it to Gerrit.

Disclaimer: It's my first backport in a submodule, I hope I've satisfied all requirements.

#12 Updated by Jean-Sébastien Gervais almost 6 years ago

Thank you, tested patch on a 4.5.20 instance

#13 Updated by Gerrit Code Review almost 6 years ago

Patch set 2 for branch 4.5 has been pushed to the review server.
It is available at https://review.typo3.org/17616

#14 Updated by Gerrit Code Review almost 6 years ago

Patch set 1 for branch 4.6 has been pushed to the review server.
It is available at https://review.typo3.org/17931

#15 Updated by Gerrit Code Review almost 6 years ago

Patch set 1 for branch workspaces_4-7 has been pushed to the review server.
It is available at https://review.typo3.org/17932

#16 Updated by Gerrit Code Review almost 6 years ago

Patch set 2 for branch 4.6 has been pushed to the review server.
It is available at https://review.typo3.org/17931

#17 Updated by Ingo Pfennigstorf almost 6 years ago

  • Status changed from Under Review to Resolved

Applied in changeset commit:06034536a5f3f50c30a91a6dc02e13083876809f.

#18 Updated by Michael Stucki about 5 years ago

  • Category changed from Bugs to Workspaces

#19 Updated by Michael Stucki about 5 years ago

  • Project changed from Workspaces & Versioning to TYPO3 Core
  • Category changed from Workspaces to Workspaces

#20 Updated by Benni Mack 2 months ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF