Bug #88524

rmdir not respecting POSIX standard - missing files

Added by Pascal Christen 2 months ago. Updated 15 days ago.

Status:
Resolved
Priority:
Should have
Assignee:
-
Category:
-
Target version:
-
Start date:
2019-06-07
Due date:
% Done:

100%

TYPO3 Version:
10
PHP Version:
Tags:
CodingNight
Complexity:
Is Regression:
Sprint Focus:

Description

Hi

Having a problem with typo3 and it's own written rmdir function: https://github.com/TYPO3/TYPO3.CMS/blob/master/typo3/sysext/core/Classes/Utility/GeneralUtility.php#L2105

POSIX standard says the following:

If a file is removed from or added to the directory after 
the most recent call to opendir() or rewinddir(), whether 
a subsequent call to readdir() returns an entry for that file is unspecified

So the problem is, that it doesn't return all files while deleting it - for more details read the issue about the same topic here: https://github.com/emscripten-core/emscripten/issues/2528

As a consequence, e.g. deleting extensions does not work because it did not delete all files.

Thanks, greetings Pascal

Associated revisions

Revision 28d1f6af (diff)
Added by Georg Gro├čberger 15 days ago

[BUGFIX] Ensure all files in a directory are deleted

The function readdir may skip files inside a directory when
its content is changed, like a file being deleted. This can
cause not all files inside being deleted

To avoid this, we collect all entries inside a directory and
close the read descriptor before deleting all entries inside.

Resolves: #88524
Releases: master, 9.5
Change-Id: I06168e77dccc41ae60b49abd7644ee3748e85588
Reviewed-on: https://review.typo3.org/c/Packages/TYPO3.CMS/+/61408
Tested-by: TYPO3com <>
Tested-by: Marco Bresch <>
Tested-by: Susanne Moog <>
Reviewed-by: Matthias Vogel <>
Reviewed-by: Marc von Schalscha <>
Reviewed-by: Susanne Moog <>

Revision b847b3a3 (diff)
Added by Georg Gro├čberger 13 days ago

[BUGFIX] Ensure all files in a directory are deleted

The function readdir may skip files inside a directory when
its content is changed, like a file being deleted. This can
cause not all files inside being deleted

To avoid this, we collect all entries inside a directory and
close the read descriptor before deleting all entries inside.

Resolves: #88524
Releases: master, 9.5
Change-Id: I06168e77dccc41ae60b49abd7644ee3748e85588
Reviewed-on: https://review.typo3.org/c/Packages/TYPO3.CMS/+/61364
Reviewed-by: Frank Naegler <>
Reviewed-by: Andreas Fernandez <>
Tested-by: TYPO3com <>
Tested-by: Frank Naegler <>
Tested-by: Andreas Fernandez <>

History

#1 Updated by Pascal Christen 2 months ago

So I just "rebuilt" the function:

showfiles.php
<?php
function removeFiles($path)
{
    $handle = opendir($path);
        while (false !== ($file = readdir($handle))) {
            if ($file === '.' || $file === '..') {
                    continue;
            }
        echo "PATH $path/$file\n";
        #unlink($path . '/' . $file);
        }
        closedir($handle);
}
$path = realpath('/home/christes/test/');
removeFiles($path);

So, I'm touching 200 new files in the directory /home/christes/test/
[christes@xxx:~/test] $ touch test{001..200}.test

Now if I run the PHP-script without the unlink (commented out) - it's all fine:
[christes@xxx:~] $ php listdir.php | wc -l
     200

Now, if I remove the # in front of the ulink this happens:
[christes@xxx:~] $ php listdir.php | wc -l
     126
[christes@xxx:~] $ php listdir.php | wc -l
      74

On the first round, it only removes the first 126 files, if I rerun it again it deletes the remaining files.

And it deletes less files if the filename is longer:
[christes@xxx:~/test] $ touch testtesttesttesttesttesttesttesttesttesttest{001..200}.test
[christes@xxx:~] $ php listdir.php | wc -l
     111

#2 Updated by Pascal Christen about 2 months ago

Anyone?

#3 Updated by Susanne Moog 16 days ago

  • Tags set to CodingNight

#4 Updated by Gerrit Code Review 16 days ago

  • Status changed from New to Under Review

Patch set 1 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/61408

#5 Updated by Gerrit Code Review 15 days ago

Patch set 1 for branch 9.5 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/61364

#6 Updated by Anonymous 15 days ago

  • Status changed from Under Review to Resolved
  • % Done changed from 0 to 100

Also available in: Atom PDF