Project

General

Profile

Actions

Bug #21269

closed

md5_file() to check if a file has been changed is very expensive [performance]

Added by Martin Holtz over 14 years ago. Updated about 13 years ago.

Status:
Closed
Priority:
Should have
Category:
-
Target version:
-
Start date:
2009-10-14
Due date:
% Done:

0%

Estimated time:
TYPO3 Version:
PHP Version:
4.3
Tags:
Complexity:
Is Regression:
Sprint Focus:

Description

i am generating thumbnails from very large images. I thought, after rendering once, it should not be an performance issue anymore.

But with every new page creation, there is a md5_file() check, if the file has been changed. That is a nice feature - which takes 300-500ms for a 40MB file - with 20 thumbnails the page generation takes about 10 seconds.

There was a discussion about that on the performance newsgroup with the conclusion, that a flag to use only mtime would be a great enhancement.

Joey Hasenau and Martin Kutschker wrote that pseudocode, which should be ok, to handle that issue:

t3lib/class.t3lib_stdgraphic.php -> getCachedImageDimensions($imageFile)

if (size changed) { // any size change, means content change
make new image
} else {
if (mtime changed) { // mtime change, could mean content change
// make md5 test only, if mtime is not enough for you
if (noContentCheckFlag set || content changed)
make new image
}
}
}

it would introduce a new flag to disable the md5_file() check - only mtime would has been used than.

Additional informations:

http://lists.typo3.org/pipermail/typo3-performance/2009-September/000264.html
http://lists.typo3.org/pipermail/typo3-performance/2009-September/000277.html
(issue imported from #M12232)


Files

bug_12232.patch (6.2 KB) bug_12232.patch Administrator Admin, 2010-09-30 17:12
class.tx_example_converttask.php (1.24 KB) class.tx_example_converttask.php Administrator Admin, 2011-01-17 22:21
12232_06.diff (2.11 KB) 12232_06.diff Administrator Admin, 2011-01-18 22:37
Actions #1

Updated by Sigfried Arnold over 13 years ago

That issue is even worse (slower by around factor 5) on NFS enviroments, md5_file() takes ages and blocks the whole process.

Actions #2

Updated by Christian Kuhn over 13 years ago

trunk rev. 10126

Actions

Also available in: Atom PDF