Feature #27160

Do not delete uniqueArray in TYPO3 4.6.*

Added by Stefan Froemken almost 10 years ago. Updated over 8 years ago.

Status:
Closed
Priority:
Should have
Assignee:
-
Category:
-
Target version:
-
Start date:
2011-05-31
Due date:
% Done:

0%

Estimated time:
PHP Version:
Tags:
Complexity:
Sprint Focus:

Description

Hello Core-Team,

I just see that you want to delete t3lib_div::uniqueArray in TYPO3 4.6.*. Please don't do that, be faster than PHP: unique_array():

// the following is much faster than array_unique()
$tempArray = array();
foreach($myArrayWithDoubledValues as $key => $val) {
$tempArray[$val] = true;
}
$uniqueArray = array_keys($tempArray);

I have tested it...this is really fast and I think this should be stayed in TYPO3-API.

Stefan


Files

array_unique.php (816 Bytes) array_unique.php Stefan Froemken, 2011-06-01 10:11
#1

Updated by Georg Ringer almost 10 years ago

please provide testcases + numbers

#2

Updated by Ernesto Baschny almost 10 years ago

  • Status changed from New to Needs Feedback

Your version is also not compatible with array_unique() because the PHP function preserves the existing keys, while your throws them away.

So if you want that to stay, you have to provide:

a) an alternative that provides the same API as before (in all use-cases)
b) is provenly faster (provide some benchmarks, test-scripts etc...)

#3

Updated by Stefan Froemken almost 10 years ago

Here is a benchmark. I hope it helps:

$milliSec = t3lib_div::milliseconds();
$count = count($tagTempArray);
$tagArray = array();
foreach($tagTempArray as $key => $val) {
$tagArray[$val] = true;
}
$this->tagsInSearchResult = array_keys($tagArray);
t3lib_div::devLog('test', 'test', -1, array(
t3lib_div::milliseconds() - $milliSec,
'Count before unique: ' . $count,
'Count after unique: ' . count($tagArray)
));

Output of devlog:
93
Count before unique: 337048
Count after unique: 785

--------------------------------------------------------
$milliSec = t3lib_div::milliseconds();
$count = count($tagTempArray);
$tagArray = array();
$tagArray = array_unique($tagTempArray);
$this->tagsInSearchResult = $tagArray;
t3lib_div::devLog('test', 'test', -1, array(
t3lib_div::milliseconds() - $milliSec,
'Count before unique: ' . $count,
'Count after unique: ' . count($tagArray)
));

Output of devlog:
1105
Count before unique: 337048
Count after unique: 785

Stefan

#4

Updated by Stefan Froemken almost 10 years ago

I have changed the code to work like array_unique:

$milliSec = t3lib_div::milliseconds();
$count = count($tagTempArray);
$tagArray = array();
foreach($tagTempArray as $key => $val) {
$tagArray[$val] = $key;
}
$this->tagsInSearchResult = array_combine(array_values($tagArray), array_keys($tagArray));
t3lib_div::devLog('test', 'test', -1, array(
t3lib_div::milliseconds() - $milliSec,
'Count before unique: ' . $count,
'Count after unique: ' . count($tagArray)
));

Output of devlog:
112
Count before unique: 337048
Count after unique: 785

You see: It's only 15 milliseconds slower than my previous idea.

I have attached a file where you can see, that my idea works like array_unique.

Stefan

#5

Updated by Xavier Perseguers almost 10 years ago

  • Target version changed from 4.6.0 to 4.6.0-beta1
#6

Updated by Xavier Perseguers almost 10 years ago

  • Target version deleted (4.6.0-beta1)
#7

Updated by Stefan Froemken almost 10 years ago

Is there someone how can tell me what I have done wrong? I have a solution, I have delivered a test/benchmark, it is faster and now it was deleted for 4.6.0?!?! I don't want to start a neverending discussion...but please give me a try to understand it.

Stefan

#8

Updated by Stefan Galinski about 9 years ago

IMHO trying to rebuilding php functionality just because it's faster than the native version isn't a good idea. You always can get the problem that the non-native functionality may be buggy or simply doesn't works like the php version and shouldn't be proposed like a drop-in replacement. Also you don't profit from changes in the further PHP developement and I'am pretty sure that it must have reasons that the native version is slower. PHP 5.4 has maybe alredy fixed the performance issue!? If not the reasons should be discussed in their bugtracker... Just my point of view. :-)

Here is also a nice result found by Google regarding the array_unique issue.

http://www.puremango.co.uk/2010/06/fast-php-array_unique-for-removing-duplicates/

#9

Updated by Stefan Froemken over 8 years ago

I think you can close this report now. It's over half a year old.

For all the others: Here is my latest code to keep the keys:

$tempArray = array();
foreach($arr as $key => $val) {
    if(isset($tempArray[$val])) continue;
    $tempArray[$val] = $key;
}
$arrUnique = array_flip($tempArray);

#10

Updated by Stefan Galinski over 8 years ago

  • Status changed from Needs Feedback to Closed

Also available in: Atom PDF