TYPO3 Core

Currently, in the TYPO3 AJAX interface for the backend there is an option to encode the response that is sent back to the browser via JSON.

The php-internal function "json_encode" is used for sending back the returned array as a JSON string. However, this function (as documented) works only on UTF-8 encoded data. Test results have proven that if you send json_encode an array with a string that contains non-unicode encoded symbols (like Umlauts in iso-8859-1) that an empty result is returned from this function.

This basically means that before any data is sent back to the browser via the JSON format that it needs to be converted to Unicode, otherwise the response will be empty.

Solution:
Let's check the localCharset (e.g. with forceCharset) if it is unicode before converting the response array to a JSON string. Send the JSON encoded string as UTF-8 then.

Thus: every server call that is returned via JSON needs to be UTF8-encoded. Should probably update the documentation as well then. I will take care of this as well.

Steps to reproduce:
1. Make sure your TYPO3 Backend runs in a non-unicode environment
2. Look for a registered AJAX call where you return a json response
3. Make sure the data that is sent back to the browser from the server includes some umlauts or something non-standard.
4. See with Firebug what the response is with JSON. Without the patch, the response will be empty (there will be no error), with the patch, the response will be as it should be.

Please note:
This is different issue than we had before with the $_REQUEST parameters being encoded in a wrong format (as javascript always sends unicode to the server, and we need to convert them to the force charset). However this issue is related to sending back data FROM the server TO the browser.

Btw: MAYBE this is also related to the mystery of the "relogin does not work anymore" errors...

FAQ:
Why hasn't this been an issue before?
Well, most of the core-related AJAX calls so far deliver either non-problematic symbols or data in XML or plain text format, not JSON. Of course, this problem only occurs when you have a non-unicode environment running.

How did you stumble over this?
In the Advanced Frontend Editing we send quite a lot of data (the TCEforms :)) back to the browser with JSON. Although we'll probably change the way this is sent (using an iframe), it did not work on my system with the current configuration.
(issue imported from #M11361)