Story #32558: As core developer, the new Logging API needs to be integrated in all core compontents
Avoid duplicate entries in deprecation log
Currently deprecated stuff is logged every time it is called and can flood the deprecation log to a huge size quickly.
This can cause problems with the file size directly and also it's difficult for an admin to get an overview over a huge deprecation log.
In the end a high percentage of the content is redundant and useless.
This should be avoided.
My proposal is: When a new deprecation message is about to be logged, it should be checked if this message (excluding date/time) already exists in the log.
If so the already existing line should be deleted and only the most current version should go into the log. This way the admin knows when the problem occured the last time.
(issue imported from #M15617)
#1 Updated by Ernesto Baschny over 9 years ago
But I fear that reading and writing the whole file over and over has a major performance impact.
I would rather log the information using the registry (DB!), using a md5 hash of the line to quickly find an existing entry in the registry. As information for a line I would expect:
first_hit = tstamp
last_hit = tstamp
counter = integer
log_line = the current deprecation line
And then of course some way to "display" the information, maybe a report for the Reports module. And an "ALERT" after login (yellow box), if this registry contains anything.
What do you think?
#3 Updated by Ernesto Baschny over 9 years ago
Yes, appending to an existing file is "blazing fast". Requires only two FS operations.
Reading the whole file, processing and rewriting the whole "5kb" of data will require at least several FS operations more, and especiall more "write" operations.
Making it DB-based will make it more scalable, by having proper INDEX for the hash column, for example to be able to find existing records very fast.
#5 Updated by Sebastian Michaelsen over 9 years ago
I doubt that I can get that ready for 4.5, but anyway I can have a look at.. maybe it's implemented quite fast..
The registry just stores key/value pairs, right? How would you store all the information in that table? Especially an index over md5 hashes would not be possible I think.
So, we'd need a new table I think. And everything should be stored in a sysext.
Anything to add?
#6 Updated by Ernesto Baschny over 9 years ago
Yes, a dedicated table sounds nice. Shouldn't be a sysext because deprecation logging is activated through TYPO3_CONF_VARS. Or do you want to make the sysext "required"?
Go for it! 3 years of deprecation logs is a huge amount of time and HUGE logs, so it would be cool to have this feature still in 4.5. Thanks!
#7 Updated by Sebastian Michaelsen over 9 years ago
Yes, I thought of a required sysext. The whole deprecation log stuff should go there and I would deprecate t3lib_div::deprecationLog() ^^.
What would you think about just storing the records on root level, define TCA configurations, making it available through the regular list module. Do we really need a dedicated module for that?
#12 Updated by Thorsten Kahler almost 9 years ago
- Priority changed from Should have to Could have
- PHP Version changed from 4.3 to 5.2
The deprecation log is only relevant for developers and integrators, so IMHO it has low priority. It should be disabled on production sites anyways so however it might be implemented, performance is not an issue here.
#21 Updated by Markus Klein about 7 years ago
@Guido: Please comment directly in Gerrit. You can use the same username/pwd combination as in forge.
With this change, the files won't get that big anymore, hence this shouldn't be a problem.
Furthermore, one will remove the deprecation log file anyway after upgrading. Additionally deprecation log is not enabled on production systems and the little time it takes to read the file can be tolerated on development systems.