Bug #36238

Pagetree lost for non admin after upgrading to 4.5.15

Added by Philipp Kitzberger over 4 years ago. Updated over 4 years ago.

Status:Closed Start date:2012-04-17
Priority:Must have Due date:
Assigned To:- % Done:

0%

Category:Backend User Interface Spent time: -
Target version:-
TYPO3 Version:4.5 Complexity:
PHP Version: Is Regression:
Tags: Sprint Focus:

Description

after the upgrade to the security release 4.5.15 we encountered a strange behavior for non-admins. some parts of the pagetree cannot be applied to the pagetree, chrome console says:

ext-all-b0a2fcadbbd7a49a8783e32735395476.js:7
Uncaught TypeError: Cannot call method 'appendChild' of undefined

we tracked it down to the following conditions:
- logged in as admin user
- switching to non-admin
- this non-admin needs at least to db mounts of which the first is a folder
- as the non-admin we unfold the second element of the pagetree (the first one is a folder) and click to one of the subpages
- after this process (maybe after logging out a.k.a. switching back to admin) the be_users-record of this user contains extra data in the field uc:

...
s:8:"Pagetree";O:8:"stdClass":1:{s:9:"stateHash";O:8:"stdClass":15:{s:3:"oot";i:1;s:5:"e9-e9";i:1;s:16:"lastSelectedNode";s:7:"p771-67";s:5:"67-67";i:1;s:6:"89a-67";i:1;s:6:"882-67";i:1;s:6:"805-67";i:1;s:6:"7fb-67";i:1;s:6:"7c4-67";i:1;s:6:"77d-67";i:1;s:6:"771-67";i:1;s:6:"76b-67";i:1;s:6:"7ca-67";i:1;s:5:"69-67";i:1;s:5:"68-67";i:1;}}}}}

- re-switching to this user leads to the lost parts of the pagetree and the error message in the chrome console.

class.t3lib_tree_pagetree_node.php Magnifier (7.7 kB) Alexander Wende, 2012-04-18 14:56

Ext.ux.state.TreePanel.js Magnifier (4.1 kB) Alexander Wende, 2012-04-18 14:56

ExtDirect.StateProvider.js Magnifier (8.6 kB) Alexander Wende, 2012-04-18 14:56

24884_revert.patch Magnifier (6.5 kB) Stefan Galinski, 2012-04-18 15:13


Related issues

related to Core - Bug #24884: pagetree->expand branch destroys parts of BE_USER->uc Resolved 2011-01-28
related to Core - Bug #28687: pagetree broken due to js exception Resolved 2011-08-02
related to Core - Bug #36459: Trouble with pagetree after upgrade from 4.5.14 to 4.5.15 Resolved 2012-04-23
related to Core - Bug #36456: BE User Pagetree Problem after Upload in Filelist Closed 2012-04-23

History

#1 Updated by Stefan Galinski over 4 years ago

  • Status changed from New to Needs Feedback

Seems to be related to the backport of the stateHash performance improvement. Can you try to remove the temporary data of your user? Do you have the possibility to reproduce the bug with 4.6, 4.7, master too? At least you should try to revert to verify my initial assumption. Thanks in advance!

http://forge.typo3.org/issues/24884

#2 Updated by Philipp Kitzberger over 4 years ago

we deleted the bad part of the user configuration via database manually and after that the pagetree was accessible again...

going to reproduce that with 4.7 now...

#3 Updated by Philipp Kitzberger over 4 years ago

can't confirm this bug for 4.7, in this version the root page is visible in the page tree even for non admins. maybe this fixes it...

#4 Updated by Urs Braem over 4 years ago

I can confirm this issue with 4.5.15

Non-admin user was unable to expand one of several db_mountpoints in his pagetree.
I had the impression this was only when the page hat id=1, but I'm not sure of that.

In my case, this first showed up without switching users, he logged in normally with his user credentials.
But it was also reproduceable switching.

I reverted to 4.5.14 and it's fine again

#5 Updated by Stefan Galinski over 4 years ago

So this bug is only reproducable if you using the user switching functionality to an non-admin user? However it would be interesting if the bug was fixed with the virtual tree root feature that was introduced in 4.7. Philipp, can you try to reproduce this in 4.6 please?

#6 Updated by Urs Braem over 4 years ago

So this bug is only reproducable if you using the user switching functionality to an non-admin user?

My customer logged in directly as a non-admin user and then was unable to expand the principal mountpoint of his pagetree (with the ID 1 and set as site root).

#7 Updated by Philipp Kitzberger over 4 years ago

cant being reproduced with 4.6 trunk apparently

#8 Updated by Boris Hinzer over 4 years ago

I can also confirm this behavior on several updated TYPO3 installations with 4.5.15. No matter if the non-admin has logged in or was switched from admin mode.

#9 Updated by Christian Hernmarck over 4 years ago

I also can confirm - user (redaktor) logged in directly (and I tried it with switching from admin) - v4.5.15.

In firefox the error is:

Fehler: uncaught exception: [Exception... "Component returned failure code: 0x80004003 (NS_ERROR_INVALID_POINTER) [nsIDOMHTMLUListElement.appendChild]"  nsresult: "0x80004003 (NS_ERROR_INVALID_POINTER)"  location: "JS frame :: http://www.domain.tld/typo3temp/compressor/ext-all-7631e2d6781ffe856585eef1391429a6.js :: <TOP_LEVEL> :: line 7"  data: no]

As redactor:
When reloading the structure (green reload symbol) it dissappears completely. But you still can use the filter and search for a page - then the structure opens exactly to this page. The opened structure you can close and open again - but no other pages will be opened...

#10 Updated by Boris Hinzer over 4 years ago

As workaround you can go to the be_user table and clear the contents of the uc field of the user.

#11 Updated by Boris Hinzer over 4 years ago

Boris Hinzer wrote:

As workaround you can go to the be_user table and clear the contents of the uc field of the user.

It's NOT working - at least only once.

#12 Updated by Philipp Kitzberger over 4 years ago

Boris Hinzer wrote:

As workaround you can go to the be_user table and clear the contents of the uc field of the user.

that's only working as long as the uc field won't get written again, right?

#13 Updated by Stefan Galinski over 4 years ago

Can someone try to revert the patch #24884 please to test if that bugfix is the root of the problems?

Also I tried it myself now and could reproduce the issue that one of the visible mountpoints couldn't be expanded on click in very rare and unreproducable cases, but it was always fixed if you refresh the tree by clicking the green reload icon. Can someone create a reproducable step-by-step list or did everybody made the same experiences like me?

#14 Updated by Stefan Völker over 4 years ago

Can confirm this error.
User direct logged in, and also when admin switched to user.

#15 Updated by Ralph Brugger over 4 years ago

Can formin this too.
Several installations after Upgrade to 4.5.15
User direct logged in, and also when admin switched to user.

#16 Updated by Andreas Krüger over 4 years ago

Can confirm this error on different installations with 4.5.15.

Unfolded pagetree for non-admin users and logout = failed to restore pagetree status after next login

This does not affect Adminusers.

I can't reproduce this behaviour for "Superusers" which have access to more than one domain-root (can someone confirm this?)

#17 Updated by Alexander Wende over 4 years ago

Can confirm too.
My solution revert the patch #24884 like Stefan suggested.
I attached the patched files. Pls can someone make a patch out of my files?

#18 Updated by Stefan Völker over 4 years ago

Ok, I uploaded Alexander's files, and it works again.

#19 Updated by Stefan Galinski over 4 years ago

I have contacted the author - Joey - of the rootline patch. Maybe he has an idea or can directly fix this problem...

Also I attached a patch that reverts the new behaviour. You can patch your files with this command (remove the --dry-run if it doesn't fails):

patch -p1 -R --dry-run < 24884_revert.patch
</pre

#20 Updated by Ralph Brugger over 4 years ago

For me: I've uplodaded Alexander's files => works again

#21 Updated by Stefan Völker over 4 years ago

strange: after uploading Alexander's files, it worked.
After new login the error is there again.

#22 Updated by Stefan Völker over 4 years ago

ok. one step further (with Alexander's files):
if I am logged in as a backend user and reset my backend user configuration via "settings", logout, log in -> works.
If I logout and log in again -> not working :(

#23 Updated by Jo Hasenau over 4 years ago

A question to those who run into problems with that patch: Did you update the database?
The size of the original uc field in some cases is too small for the information that is written there, which is why the field has to be mediumtext instead of text.

#24 Updated by Jo Hasenau over 4 years ago

I just tested the switch to another user including a set of mounts attached to this user with a freshly updated 4.5.15 and everything is working as expected.

#25 Updated by Jo Hasenau over 4 years ago

And of course you have to reset the settings, since the page tree handling has been completely rewritten, which is why it now fits into the uc field AND will perform much better. So it won't match any of the former settings saved in that field.

I guess best practice would be to reset the uc field of each and every BE user by a simple MySQL query.

#26 Updated by Stefan Völker over 4 years ago

No, since my installation was patched from the hoster, the database wasn't updated.
I did the update now manually.

Now (after restoring the original files - means the 4.5.15 files) it works, if I reset the user settings (or clear the 'uc' in the database), but only for the first re-login.
After a 2nd login, the tree isn't working any more.

#27 Updated by Martin Muskulus over 4 years ago

I've got the same (original) problem. I've patched our 4.5.15 and did not update database, so uc is still of type text. It seems to solve the problem. The pagetree is restored correctly when using normal login and admin-switch.

little reminder: restart webserver or clear php-cache if used

#28 Updated by Jo Hasenau over 4 years ago

Actually the problems should have been there before as well, if it was about the size of the uc field only, since the original page tree data saved to that field was much larger than after the patch.

The point is to make sure, that everything will be updated, since there are lots of different things involved in the problem that made the patch necessary:

  1. The DB field size was too small for large page trees and therefor things were just cut while saving the data, kicking out even other settings
  2. The size of the data contained in JS was much too large, since the IDs where in there twice as decimal numbers (now they are there only once as HEX numbers with a boolean value)

This means you have to make sure, the DB field will have the necessary size and will be cleared and the PHP and JS files will not come from any kind of cache. Especially when only one of them uses the new version you will run into problems, since they won't match.

#29 Updated by Alexander Wende over 4 years ago

little notice: if you used my files and you updated your DB after update to 4.5.15 you have to set the uc field of your be_users table back to text manually.

I didn't update too and it seems to work.

#30 Updated by Andreas Kiessling over 4 years ago

Why isn't the DB update mentioned in the release notes?
http://wiki.typo3.org/TYPO3_4.5.15

#31 Updated by Jo Hasenau over 4 years ago

You should NOT set it back but leave it as mediumtext, since this was the major source of the problems that made the patch necessary.
The uc-field will contain tons of serialized arrays that will easily blow the maximum size of a text field.
Of course this depends on the size of your page trees as well as on the size of any other data your backend users depend on, but it still won't hurt if you leave it as mediumtext, since this will be 16MB instead of the former 64KB.

#32 Updated by Jo Hasenau over 4 years ago

Andreas Kiessling wrote:

Why isn't the DB update mentioned in the release notes?
http://wiki.typo3.org/TYPO3_4.5.15

Maybe because it is recommended to use the install tool DB compare anyway?
http://wiki.typo3.org/Upgrade

But it should be mentioned that the uc field should be cleared to avoid problems with the old data structures saved there.

I still can not reproduce the behaviour on our servers though, and since the same structures are working flawlessly for users of 4.6 or 4.7 as well, I guess the source of the problems can't be this patch only.

#33 Updated by Andreas Kiessling over 4 years ago

Jo Hasenau wrote:

Andreas Kiessling wrote:

Why isn't the DB update mentioned in the release notes?
http://wiki.typo3.org/TYPO3_4.5.15

Maybe because it is recommended to use the install tool DB compare anyway?

The release notes cleary state:

No database updates are necessary.

Changes to the db are usually not allowed for bugfix releases. In this case, an exception surely is required.

#34 Updated by Jochen Weiland over 4 years ago

Changing the size of the uc field from text to mediumtext alone does not solve the problem.

#35 Updated by Stefan Galinski over 4 years ago

The backport was requested nine months ago (see related ticket). Unfortunatly the comment in the release notes is wrong, but to be honest it's not really expectable that a change of a field to a variant that can store more bytes could be a root of troubles. Especially if the patch shortened the necessary size dramatically. I informed Olly about this missing information. Sorry for the troubles!

#36 Updated by Jo Hasenau over 4 years ago

Yes - but since it is definitely not the cause for the current problems but was the cause for former problems leading to that patch, at least this change should not be reverted.

I am currently investigating what happened to the original patch provided with #24844 and why it does not work with 4.5 but does wotk with 4.6 and 4.7 even though it was proved for 4.5 only at that time.

#37 Updated by Jo Hasenau over 4 years ago

After all I was able to nail it down.

This is the reason: #28687

And here is the fix: https://review.typo3.org/#/c/5958/

After all it seems that there was no proper connection between #24884 and #28687 so we didn't notice that the patch was buggy while backporting it to 4.5.15.

Since the follow up has been applied to TYPO3 versions 4.6+ already, people don't have problems with those.

#38 Updated by Jo Hasenau over 4 years ago

For those who prefer a manual patch:

Go to the file

/t3lib/js/extjs/components/pagetree/javascript/Ext.ux.state.TreePanel.js

and change line 96 from
if (pageNode.expanded === false) {

to
if (pageNode.expanded === false && pageNode.rendered == true) {

#39 Updated by Oliver Hader over 4 years ago

Thanks for digging into the problem, can somebody please summarize what caused the problem and what criteria are required that it occurs? Thanks...

#40 Updated by Jo Hasenau over 4 years ago

The problem is the same as described here: #28687

If there are mount points with no child pages, the page tree can't expand them.
This is why there has to be a check, if the node has been rendered at all, before expanding it.
If there is no check, this might lead to a partly broken page tree with nodes, that can not be expanded anymore.

This usually happens to normal editors only but not to admin users, since usually mountpoints are not empty, but they might be due to restrictions for editors.

Basically that's it, the rest of the thread at #28687 is just about trying to find the reasons.

Actually this had been fixed for 4.6 already, but the fix was never connected to the original patch at #24884, which is why Stefan and me did not notice that it was necessary to apply both of them for a proper result.

#41 Updated by Stefan Galinski over 4 years ago

I just pushed the not yet backported fix to Gerrit. Please review and vote for it. Thanks Joey for finding this missing relationship. :-)

http://review.typo3.org/10626

I will close this issue, after the backport is merged.

#42 Updated by Maik Matthias over 4 years ago

I can confirm this bug for 4.5.15 installations. Changing Line 96 in Ext.ux.state.TreePanel.js as mentioned by Jo Hasenau seems to fix it. Thanks!

#43 Updated by Kay Strobach over 4 years ago

can confirm this as well

#44 Updated by Alexander Wende over 4 years ago

can confirm too, thanks.

#45 Updated by Georg Schönweger over 4 years ago

IMO if a database compare is required this should be mentioned in the realase notes, same thing for clearing the uc field! Afaik this was never required for minor upgrades, so people won't do it.

#46 Updated by André Steiling over 4 years ago

Yes, changing line 96 fix it!

#47 Updated by Helmut Hummel over 4 years ago

Andreas Kiessling wrote:

Why isn't the DB update mentioned in the release notes?
http://wiki.typo3.org/TYPO3_4.5.15

Why could such a change go into a minor release anyway? We have the policy to not change the DB structure in minor releases!

#48 Updated by Helmut Hummel over 4 years ago

Hi,

Jo Hasenau wrote:

Maybe because it is recommended to use the install tool DB compare anyway?
http://wiki.typo3.org/Upgrade

This the recommendation for upgrading minor versions not for updates of bugfix versions!

But it should be mentioned that the uc field should be cleared to avoid problems with the old data structures saved there.

This is also a breaking change. Has Ernesto been involved when backporting that to 4.5?

#49 Updated by Jo Hasenau over 4 years ago

Sorry - but the term "backport" seems a bit misleading here.

Actually the patch has been made for 4.5 but it has been forgotten to be merged. http://forge.typo3.org/issues/24884#note-12

The size of that field is essential, since otherwise larger trees will be completely broken. Even if you skipped the rest of the patch, a size of 64KB would still have been not enough for the trees, since they were much larger before the patch.

So the change of that field is definitely a bugfix and not a backport of a feature and it was confirmed by one of the release managers already last year.

#50 Updated by Andreas Kiessling over 4 years ago

Patching Ext.ux.state.TreePanel.js seems to work for me too. Thanks a lot for investigating!

Clearing the uc field also means loosing a lot of other settings from the user, e.g. backend language, title lengths, copy settings.
We should distinguish between real "user settings" and "temporary data" like the page tree state that can be cleared without really loosing something.

#51 Updated by Jo Hasenau over 4 years ago

Clearing the whole field or the temporary settings was just meant as a workaround for those who run into problems even with the hotfix being applied.

So you don't have to clear it but you might want to.

#52 Updated by Andrea Herzog-Kienast over 4 years ago

Hi!
Same problem. Will there be an official patch soon?

Best,
Andrea

#53 Updated by Maik Matthias over 4 years ago

There is another problem which could be related. After every login of normal users the pagetree in the view module is completely closed. This problem isn´t fixed by the patch.

#54 Updated by Martin Muskulus over 4 years ago

Maik Matthias wrote:

There is another problem which could be related. After every login of normal users the pagetree in the view module is completely closed. This problem isn´t fixed by the patch.

I can't reproduce this. Normal user logs in and (in my case) templavoila's page module is opened with correct pagetree state every time. Switching to view module shows the same pagetree. If user sets view module as startmodule the correct pagetree is shown too. It makes no difference if i use adminswitch or log in as normal user.

#55 Updated by Jo Hasenau over 4 years ago

Same here - can't reproduce it - page tree opens in the exact same state as it was left at the logout. And even the last page that has been visited is loaded. Did you change the size of the uc field to mediumtext? If not, it might happen that parts of your settings are just cut off when the serialized arrays are written to the DB. So they will be missing at the next login.

BTW: This is the default behaviour of MySQL in such a case: You send content that is larger than 64KB to a field of the type text => anything beyond 64KB is just cut off but the rest is saved without further notice. - And this was the original problem that has been fixed with patch #24884

#56 Updated by Jose Antonio Guerra over 4 years ago

Andreas Kiessling wrote:

Patching Ext.ux.state.TreePanel.js seems to work for me too. Thanks a lot for investigating!

This also worked for me :)

#57 Updated by Stefan Galinski over 4 years ago

  • Status changed from Needs Feedback to Closed

The missing patch was merged and will be available in the next 4.5 release. So I will close this issue now. Thanks everybody for reporting, testing and reviewing!

#58 Updated by Peter Linzenkirchner over 4 years ago

Maik Matthias wrote:

There is another problem which could be related. After every login of normal users the pagetree in the view module is completely closed. This problem isn´t fixed by the patch.


It is the same in a few of my installations: every time when the user logs in, the pagetree ist completely closed.

#59 Updated by Stefan Galinski over 4 years ago

This problem was fixed with the addition of the new virtual tree mount in 4.7.

#60 Updated by Stephan Großberndt over 4 years ago

Target version is not set yet (4.5.16?). More and more customers call me complaining about this issue. Should I patch manually or is 4.5.16 around the corner?

#61 Updated by Kay Strobach over 4 years ago

patch manually! i patched all my installs in the meantime, but the download should be hidden as long as this is not fixed :P ... such a bug without a fix-release is a mess - sry guys

I mean we have a patch since 7 - in words seven days -.-

#62 Updated by Thomas Deinhamer over 4 years ago

Does this mean an update of the database is still needed? How would
this be done for e.g. 40 installations with a minimal effort?

Also available in: Atom PDF