Bug #18463

Cannot import previously exported t3d file

Added by David Frster over 13 years ago. Updated almost 11 years ago.

Status:
Closed
Priority:
Should have
Assignee:
Category:
-
Target version:
-
Start date:
2008-03-17
Due date:
% Done:

0%

Estimated time:
TYPO3 Version:
4.2
PHP Version:
5.2
Tags:
Complexity:
Is Regression:
Sprint Focus:

Description

I just tried to copy a page tree from one installation to another. Both are running Typo3 4.2 from subversion. (up to date)

The import fails, displaying this error message:
Warning: fread() [function.fread]: Length parameter must be greater than 0 in /home/dave/projects/develmisc/typo3-mce/typo3-4.2/typo3/sysext/impexp/class.tx_impexp.php on line 2277

A brief look reveiled, that the argument passed to intval in the given line is an empty string.

I don't want to upload the exported file here but I can send it by private email. Contact me at

(issue imported from #M7882)


Files

7882_import_t3d.diff (908 Bytes) 7882_import_t3d.diff Administrator Admin, 2008-03-17 13:39
0007882.patch (2.76 KB) 0007882.patch Administrator Admin, 2008-03-17 23:53
backend_php.png (3.62 KB) backend_php.png Administrator Admin, 2008-03-17 23:59

Related issues

Related to TYPO3 Core - Bug #18451: <br /> and <link> tags not properly converted but instead escaped and displayed literally in (in conjunction with UTF-8, umlaut)Closed2008-03-15

Actions
#1

Updated by Steffen Kamper over 13 years ago

could you test the attached patch?

there is an error with your t3d-file. I also tested with 4.1 and was not able to read the file

#2

Updated by David Frster over 13 years ago

Well, if the file is broken, the bug is in the t3d export. I suggest fixing the export instead of supporting the import of broken files.

The export was made with the latest revision (3444) from the 4.2 branch, exporting the page tree starting at the site root.

#3

Updated by Steffen Kamper over 13 years ago

I exported and imported t3d in 4.2 without any problems.

#4

Updated by Oliver Hader over 13 years ago

Hi David,
thanks for sending me the export file. It seems like the file is corrupted. But the interesting thing is: Why?
The first file part is defined to have a length of 6065 bytes, but really has a length of 8734 bytes. The data is compressed using zlib, thus I don't see a relationship to unicode/multibyte characters. I'm going to do some more tests on this issue.
What is the exact PHP version and your zlib/bzip wrappers you're using?

#5

Updated by David Frster over 13 years ago

Here's some information from the phpinfo() page. What do you mean by wrappers?

PHP Version 5.2.4

ZLib Support enabled
Stream Wrapper support compress.zlib://
Stream Filter support zlib.inflate, zlib.deflate
Compiled Version 1.2.3
Linked Version 1.2.3

Directive Local Value Master Value
zlib.output_compression Off Off
zlib.output_compression_level -1 -1
zlib.output_handler no value no value

#6

Updated by Oliver Hader over 13 years ago

Do you have mbstring.func_overload enabled?
http://de2.php.net/manual/de/ref.mbstring.php#mbstring.overload

I guess so, since it delivers exactly the data in your export file:
strlen($data): 8733
mb_strlen($data, 'utf-8'): 6065

#7

Updated by Oliver Hader over 13 years ago

The attached patch does not fix or revert the PHP setting, but shows a warning in the install tool and adds a warning entry if a file seems to be corrupted in tx_impexp.

#8

Updated by David Frster over 13 years ago

Oliver, thank you very much tracking this down. The mbstring.overload setting is indeed enabled on the server.

The proper fix for this is not to rely on that the length of a string is it's byte count because this is not true for unicode character sets. (Supporting them is one goal of Typo 4.2, isn't it?)

I suggest introducing a byte_count function using mb_strlen($string, '8bit') to determine the byte count of a string. (When the mbstring extension is not available it's of course safe to fallback to strlen.) What you want is not the length of the string but the byte count!

The mbstring.overload setting is very important for UTF-8 setups as it makes sure, that things like input validation continue to work. (Otherwise length checks for input fields will fail mysteriously when special characters are entered.)

#9

Updated by Oliver Hader over 13 years ago

Hi,
yes, supporting UTF-8 is a goal which does not mean to enforce it. mbstring is not enabled in PHP5 by default - there are also some distributions that require to install php5-mbstring explicitly as module.
Thus, changing all occurrences of "strlen()" in the TYPO3 Core to "mb_strlen()" is a no-go. But, if a developer wants his extension to be working also in unicode environments, he could e.g. use the following in his classes (front-ent as example):

if (function_exists('mb_strlen')) {
$length = mb_strlen($string, $GLOBALS['TSFE']->renderCharset);
} else {
$length = strlen($string);
}

We could put those hybrid-functions to t3lib_div which will determine whether to use single-/multi-bytes - but that's all.

I'm running a lot of UTF-8 sites without any problems and without the need of setting "mbstring.func_overload".

#10

Updated by David Frster over 13 years ago

Thus, changing all occurrences of "strlen()" in the TYPO3 Core to "mb_strlen()" is a no-go.

I fully agree, that's exactly what the overload setting is for. Setting it requieres no further porting to utf8-enable an application.

The only special case is, when you need to cout the bytes a string uses (which is rarely needed, probably only in the importer).

I suggest introduce a byte_count function:

function byte_count($string) {
if (function_exists('mb_strlen')) {
// string may be encode with a multibyte character set
return mb_strlen($string, '8bit');
} else {
// string is probably 8bit encoded
return strlen($string);
}
}

Wherever you want the length of string you use strlen (which may have been overloaded to support your multibyte character encoding) and when you want the byte count you use byte_count. No furter references to any mb_ functions needed.

This works for all kind of setups, regardless of the installed extensions or used character sets.

As said before: The only thing that is needed, is to realize that the length of a string may not be it's byte count and thus treat it differently.

#11

Updated by David Frster over 13 years ago

The overload setting is needed for the following case:

A username may only be 8 characters long.
When someone enters 8 characters of which some a special characters, the UTF-8 representation will be more than 8 bytes and thus rejected.
With the overload setting the string length will be determined correctly and it works as expected.

#12

Updated by Steffen Kamper over 13 years ago

did you looked at t3lib_cs::strlen ?

#13

Updated by Martin Kutschker over 13 years ago

David, of which username are you talking of? Neither BE nor FE users are restricted to 8 characters/bytes. But both are restricted to ASCII.

#14

Updated by David Frster over 13 years ago

The comment was not about Typo itself but custom PHP code.

As the use of the mbstring overload setting is discouraged officially, this bug can be closed.

I suggest applied the above patch to warn when this setting is enabled.

#15

Updated by Christian Kuhn almost 11 years ago

Resolved, no change required as suggested by reporter.

Also available in: Atom PDF