Bug #83580

GeneralUtility::xml2array() can't parse bigger files (> 10MB)

Added by Frank Naegler over 4 years ago. Updated about 1 month ago.

Status:
Under Review
Priority:
Should have
Assignee:
-
Category:
System/Bootstrap/Configuration
Target version:
-
Start date:
2018-01-16
Due date:
% Done:

0%

Estimated time:
TYPO3 Version:
9
PHP Version:
Tags:
Complexity:
Is Regression:
Sprint Focus:

Description

Because of an internal hard limit, GeneralUtility::xml2array() can't parse XML files bigger 10MB.
With #83578 a test is added to document this issue. The added test can be removed (or adjusted) after the bug is fixed.


Related issues

Related to TYPO3 Core - Task #83578: Add more UnitTests for GeneralUtility::xml2array()ClosedFrank Naegler2018-01-16

Actions
Related to TYPO3 Core - Epic #72584: sysext:impexpClosed2008-07-25

Actions
Related to TYPO3 Core - Story #92526: Concept for improved EXT:impexpAccepted2020-10-09

Actions
Related to TYPO3 Core - Task #95398: Replace XML handling by GeneralUtility with XmlEncoderUnder Review2021-09-29

Actions
Related to TYPO3 Core - Task #96635: Improve XML handling of XmlEncoder and XmlDecoderNew2022-01-24

Actions
Has duplicate TYPO3 Core - Feature #95021: Handle "XML could not be parsed: Line ...: No memory" during import in EXT:impexpClosed2021-08-28

Actions
#1

Updated by Frank Naegler over 4 years ago

  • Related to Task #83578: Add more UnitTests for GeneralUtility::xml2array() added
#2

Updated by Markus Klein about 4 years ago

#3

Updated by Benni Mack about 3 years ago

  • Category changed from Miscellaneous to System/Bootstrap/Configuration
#4

Updated by Alexander Nitsche 9 months ago

  • Has duplicate Feature #95021: Handle "XML could not be parsed: Line ...: No memory" during import in EXT:impexp added
#5

Updated by Gerrit Code Review 9 months ago

  • Status changed from New to Under Review

Patch set 1 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#6

Updated by Alexander Nitsche 9 months ago

  • Related to Story #92526: Concept for improved EXT:impexp added
#7

Updated by Matthias Druske 9 months ago

I can confirm this issue also vor TYPO3 10.4.X

Tracing back the impexp extension's inability to parse XML files bigger than ~10 MB seems to be a direct consequence of the limit hard coded into the used parsing method. Besides the primary hurdle this poses for site migration (10 MB are exceeded easily nowadays), it also seems the current method is very hungry for memory anyway.

In addition, besides GeneralUtility::xml2array / GeneralUtility::xml2arrayProcess, I assume, that
GeneralUtility::xml2tree runs into the same issue.

#8

Updated by Gerrit Code Review 9 months ago

Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#9

Updated by Gerrit Code Review 9 months ago

Patch set 3 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#10

Updated by Gerrit Code Review 9 months ago

Patch set 4 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#11

Updated by Gerrit Code Review 9 months ago

Patch set 5 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#12

Updated by Gerrit Code Review 9 months ago

Patch set 6 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#13

Updated by Gerrit Code Review 9 months ago

Patch set 7 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#14

Updated by Gerrit Code Review 9 months ago

Patch set 8 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#15

Updated by Gerrit Code Review 9 months ago

Patch set 9 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#16

Updated by Gerrit Code Review 9 months ago

Patch set 10 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#17

Updated by Gerrit Code Review 9 months ago

Patch set 11 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#18

Updated by Gerrit Code Review 9 months ago

Patch set 12 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#19

Updated by Gerrit Code Review 9 months ago

Patch set 13 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#20

Updated by Gerrit Code Review 9 months ago

Patch set 14 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#21

Updated by Gerrit Code Review 9 months ago

Patch set 15 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#22

Updated by Gerrit Code Review 9 months ago

Patch set 16 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#23

Updated by Gerrit Code Review 9 months ago

Patch set 17 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#24

Updated by Gerrit Code Review 9 months ago

Patch set 18 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#25

Updated by Gerrit Code Review 9 months ago

Patch set 19 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#26

Updated by Gerrit Code Review 9 months ago

Patch set 20 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#27

Updated by Gerrit Code Review 9 months ago

Patch set 21 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#28

Updated by Gerrit Code Review 9 months ago

Patch set 22 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#29

Updated by Gerrit Code Review 9 months ago

Patch set 23 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#30

Updated by Gerrit Code Review 8 months ago

Patch set 24 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#31

Updated by Gerrit Code Review 8 months ago

Patch set 25 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#32

Updated by Gerrit Code Review 8 months ago

Patch set 26 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#33

Updated by Gerrit Code Review 8 months ago

Patch set 27 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#34

Updated by Gerrit Code Review 8 months ago

Patch set 28 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#35

Updated by Gerrit Code Review 8 months ago

Patch set 29 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#36

Updated by Gerrit Code Review 8 months ago

Patch set 30 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#37

Updated by Gerrit Code Review 8 months ago

Patch set 31 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#38

Updated by Gerrit Code Review 8 months ago

Patch set 32 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#39

Updated by Gerrit Code Review 8 months ago

Patch set 33 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#40

Updated by Gerrit Code Review 8 months ago

Patch set 34 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#41

Updated by Gerrit Code Review 8 months ago

Patch set 35 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#42

Updated by Gerrit Code Review 8 months ago

Patch set 36 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#43

Updated by Gerrit Code Review 8 months ago

Patch set 37 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#44

Updated by Gerrit Code Review 8 months ago

Patch set 38 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#45

Updated by Gerrit Code Review 8 months ago

Patch set 39 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#46

Updated by Alexander Nitsche 8 months ago

  • Related to Task #95398: Replace XML handling by GeneralUtility with XmlEncoder added
#47

Updated by Gerrit Code Review 8 months ago

Patch set 40 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#48

Updated by Gerrit Code Review 8 months ago

Patch set 41 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#49

Updated by Gerrit Code Review 7 months ago

Patch set 42 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#50

Updated by Gerrit Code Review 7 months ago

Patch set 43 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#51

Updated by Gerrit Code Review 7 months ago

Patch set 44 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#52

Updated by Gerrit Code Review 6 months ago

Patch set 45 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#53

Updated by Gerrit Code Review 4 months ago

Patch set 46 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#54

Updated by Gerrit Code Review 4 months ago

Patch set 47 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#55

Updated by Gerrit Code Review 4 months ago

Patch set 48 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#56

Updated by Gerrit Code Review 4 months ago

Patch set 49 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#57

Updated by Gerrit Code Review 4 months ago

Patch set 50 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#58

Updated by Alexander Nitsche 4 months ago

  • Related to Task #96635: Improve XML handling of XmlEncoder and XmlDecoder added
#59

Updated by Gerrit Code Review 4 months ago

Patch set 51 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#60

Updated by Gerrit Code Review 4 months ago

Patch set 52 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#61

Updated by Gerrit Code Review 4 months ago

Patch set 53 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#62

Updated by Gerrit Code Review 4 months ago

Patch set 54 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#63

Updated by Gerrit Code Review about 2 months ago

Patch set 55 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#64

Updated by Gerrit Code Review about 2 months ago

Patch set 56 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

#65

Updated by Gerrit Code Review about 1 month ago

Patch set 57 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70814

Also available in: Atom PDF