Task #23489

Story #32558: As core developer, the new Logging API needs to be integrated in all core compontents

Avoid duplicate entries in deprecation log

Added by Sebastian Michaelsen over 9 years ago. Updated over 5 years ago.

Status:
Rejected
Priority:
Should have
Assignee:
-
Category:
Integration
Start date:
2010-09-01
Due date:
% Done:

0%

TYPO3 Version:
6.2

Description

Currently deprecated stuff is logged every time it is called and can flood the deprecation log to a huge size quickly.
This can cause problems with the file size directly and also it's difficult for an admin to get an overview over a huge deprecation log.
In the end a high percentage of the content is redundant and useless.
This should be avoided.

My proposal is: When a new deprecation message is about to be logged, it should be checked if this message (excluding date/time) already exists in the log.
If so the already existing line should be deleted and only the most current version should go into the log. This way the admin knows when the problem occured the last time.
(issue imported from #M15617)


Related issues

Related to TYPO3 Core - Bug #24227: Deprecation Log blocks system Closed 2010-11-29
Related to TYPO3 Core - Bug #24040: Handling deprecation log Closed 2010-11-13

History

#1 Updated by Ernesto Baschny over 9 years ago

Nice idea!

But I fear that reading and writing the whole file over and over has a major performance impact.

I would rather log the information using the registry (DB!), using a md5 hash of the line to quickly find an existing entry in the registry. As information for a line I would expect:

first_hit = tstamp
last_hit = tstamp
counter = integer
log_line = the current deprecation line

And then of course some way to "display" the information, maybe a report for the Reports module. And an "ALERT" after login (yellow box), if this registry contains anything.

What do you think?

Cheers,
Ernesto

#2 Updated by Sebastian Michaelsen over 9 years ago

Is adding content to a 10MB file faster than reading and writing to a 5KB File? Just curious..

Apart from that I like your idea of making this database based.

#3 Updated by Ernesto Baschny over 9 years ago

Yes, appending to an existing file is "blazing fast". Requires only two FS operations.

Reading the whole file, processing and rewriting the whole "5kb" of data will require at least several FS operations more, and especiall more "write" operations.

Making it DB-based will make it more scalable, by having proper INDEX for the hash column, for example to be able to find existing records very fast.

#4 Updated by Ernesto Baschny almost 9 years ago

This feature has to be done for 4.6. Or do you have some time to tackle that issue still in time, Sebastian? I still think its an interesting approach.

#5 Updated by Sebastian Michaelsen almost 9 years ago

Hey Ernesto,

I doubt that I can get that ready for 4.5, but anyway I can have a look at.. maybe it's implemented quite fast..

The registry just stores key/value pairs, right? How would you store all the information in that table? Especially an index over md5 hashes would not be possible I think.

So, we'd need a new table I think. And everything should be stored in a sysext.

Anything to add?

#6 Updated by Ernesto Baschny almost 9 years ago

Yes, a dedicated table sounds nice. Shouldn't be a sysext because deprecation logging is activated through TYPO3_CONF_VARS. Or do you want to make the sysext "required"?

Go for it! 3 years of deprecation logs is a huge amount of time and HUGE logs, so it would be cool to have this feature still in 4.5. Thanks!

#7 Updated by Sebastian Michaelsen almost 9 years ago

Yes, I thought of a required sysext. The whole deprecation log stuff should go there and I would deprecate t3lib_div::deprecationLog() ^^.

What would you think about just storing the records on root level, define TCA configurations, making it available through the regular list module. Do we really need a dedicated module for that?

#8 Updated by Ernesto Baschny almost 9 years ago

Also an (easier) option, right. Let's start with that and see how it looks like. I was thinking about having it sorted by "count" or "last appearance" and linked from the Reports module as other warnings that we have.

#9 Updated by Sebastian Michaelsen almost 9 years ago

Just my thoughts.. :)

#10 Updated by Sebastian Michaelsen almost 9 years ago

Experiencing the first problem. There might be deprecation messages before the database connection is established. So writing to the DB fails.

I'll find a way around this, tomorrow.

#11 Updated by Xavier Perseguers over 8 years ago

  • Target version deleted (4.6.0-beta1)

#12 Updated by Thorsten Kahler over 8 years ago

  • Priority changed from Should have to Could have
  • PHP Version changed from 4.3 to 5.2

The deprecation log is only relevant for developers and integrators, so IMHO it has low priority. It should be disabled on production sites anyways so however it might be implemented, performance is not an issue here.

#13 Updated by Thorsten Kahler over 8 years ago

  • Tracker changed from Bug to Feature
  • Project changed from TYPO3 Core to Logging Project

Moved to project "TYPO3 v4 Logging improvements". I think it fit quite well there.

#14 Updated by Steffen Müller about 8 years ago

  • Parent task set to #32558

#15 Updated by Steffen Müller almost 8 years ago

  • Category set to Integration
  • Priority changed from Could have to Should have

#16 Updated by Steffen Müller over 6 years ago

  • Status changed from New to Accepted

#17 Updated by Gerrit Code Review over 6 years ago

Patch set 1 for branch master has been pushed to the review server.
It is available at https://review.typo3.org/19251

#18 Updated by Gerrit Code Review over 6 years ago

Patch set 2 for branch master has been pushed to the review server.
It is available at https://review.typo3.org/19251

#19 Updated by Guido S. over 6 years ago

With your patch you check the complete file. Is this maybe too unperformant if you have a bigger file?

Maybe you can remove the duplicate lines via Install Tool Upgrade Wizard for 6.1?

#20 Updated by Gerrit Code Review over 6 years ago

Patch set 3 for branch master has been pushed to the review server.
It is available at https://review.typo3.org/19251

#21 Updated by Markus Klein over 6 years ago

@Guido: Please comment directly in Gerrit. You can use the same username/pwd combination as in forge.

With this change, the files won't get that big anymore, hence this shouldn't be a problem.
Furthermore, one will remove the deprecation log file anyway after upgrading. Additionally deprecation log is not enabled on production systems and the little time it takes to read the file can be tolerated on development systems.

#22 Updated by Gerrit Code Review over 6 years ago

Patch set 4 for branch master has been pushed to the review server.
It is available at https://review.typo3.org/19251

#23 Updated by Gerrit Code Review over 5 years ago

  • Status changed from Accepted to Under Review

Patch set 5 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/19251

#24 Updated by Markus Klein over 5 years ago

Please set this to Accepted, the patch has been abandoned.

#25 Updated by Steffen Müller over 5 years ago

  • Status changed from Under Review to Rejected
  • TYPO3 Version set to 6.2

The debate about this issue neither lead to an agreement nor to an accepted patchset.
Closing this 3 years old issue as rejected.

Also available in: Atom PDF