Feature #25343

Scheduler task to delete old rows from database tables

Added by Xavier Perseguers almost 7 years ago. Updated over 4 years ago.

Status:
Closed
Priority:
Should have
Assignee:
-
Category:
Database API (Doctrine DBAL)
Target version:
-
Start date:
2011-03-16
Due date:
% Done:

100%

PHP Version:
Tags:
Complexity:
Sprint Focus:

Description

Often customer databases are filled up with lots of syslog entries. Those entries are useful but in most cases having the logs for the X past weeks/months would be sufficient.

Provide a scheduler task to remove older entries regularly.

(issue imported from #M17970)


Related issues

Related to TYPO3 Core - Feature #25338: Provide maintenance scheduler tasks Closed 2011-03-16
Related to TYPO3 Core - Bug #27632: Disabled empty "number of days" field in table garbage collection scheduler task Closed 2011-06-21

Associated revisions

Revision d8bdda72 (diff)
Added by Christian Kuhn over 6 years ago

[FEATURE] Add table garbage collection scheduler task

This feature was implemented during T3BOARD11 bug auction. It is
kindly sponsored with 100 beers by jweiland.net and is called the
'Schnaps Hans scheduler task'

Change-Id: Id7c58d28e4f6e82a708513a3ce9761baf36c06cd
Resolves: #25343
Releases: 4.6
Reviewed-on: http://review.typo3.org/1376
Reviewed-by: Christian Kuhn
Tested-by: Christian Kuhn

History

#1 Updated by Francois Suter over 6 years ago

Several tasks have been opened to clear various tables (cache, etc.). Why not trying to make it more general. I wrote an extension called "cachecleaner" which is designed to clean up arbitrary tables. It works as part of the CLI lowlevel cleaner for now, but could be adapted into a Scheduler task (I've been meaning to do that for a long time, actually).

http://typo3.org/extensions/repository/view/cachecleaner/current/

#2 Updated by Mr. Hudson over 6 years ago

Patch set 1 of change Id7c58d28e4f6e82a708513a3ce9761baf36c06cd has been pushed to the review server.
It is available at http://review.typo3.org/1376

#3 Updated by Christian Kuhn over 6 years ago

Pushed a patch for EXT:scheduler implementing the garbage collection as a scheduler task. The task deletes all database entries in sys_log table older than n days, which can be set by an additinonal field.

#4 Updated by Mr. Hudson over 6 years ago

Patch set 2 of change Id7c58d28e4f6e82a708513a3ce9761baf36c06cd has been pushed to the review server.
It is available at http://review.typo3.org/1376

#5 Updated by Mr. Hudson over 6 years ago

Patch set 3 of change Id7c58d28e4f6e82a708513a3ce9761baf36c06cd has been pushed to the review server.
It is available at http://review.typo3.org/1376

#6 Updated by Christian Kuhn over 6 years ago

  • Subject changed from Create scheduler task for removing old syslog entries to Scheduler task to garbage database tables

#7 Updated by Christian Kuhn over 6 years ago

  • Subject changed from Scheduler task to garbage database tables to Scheduler task to delete old rows from database tables

#8 Updated by Mr. Hudson over 6 years ago

Patch set 4 of change Id7c58d28e4f6e82a708513a3ce9761baf36c06cd has been pushed to the review server.
It is available at http://review.typo3.org/1376

#9 Updated by Christian Kuhn over 6 years ago

Patch set 4 is a revised version that handles more than just the sys_log table:

To prevent administrators from committing hazard to the database, tables that can be cleaned up must be registered in an own 'option' array (which extensions can add entries to). There a specific table and a time stamp field must be defined, the table will then be selectable in the task as additional field.

Per task only one table can be selected, there is no multi-select-option to simplify the logic a bit if configured tables 'vanish' from configuration if extensions are un-installed or configured otherwise (the according task will then throw an exception).

For the core tables: Currently only sys_log is configured as default table to be chosen. The following tables are candidates to be added to the configuration: sys_history, be_sessions, fe_sessions, fe_session_data. All of them already have some weird internal garbage collection implemented in the core, so that they are already garbage collected during fe/be calls sometimes. This could be cleaned up in an additional patch, which should add a registry handling to the garbage collection task that logs its last execution and only calls the currently built in core garbage collection if our new task did not run for a long time. Since there is no handling for this currently, I've not added those tables to the task, yet.

#10 Updated by Mr. Hudson over 6 years ago

Patch set 5 of change Id7c58d28e4f6e82a708513a3ce9761baf36c06cd has been pushed to the review server.
It is available at http://review.typo3.org/1376

#11 Updated by Mr. Hudson over 6 years ago

Patch set 6 of change Id7c58d28e4f6e82a708513a3ce9761baf36c06cd has been pushed to the review server.
It is available at http://review.typo3.org/1376

#12 Updated by Mr. Hudson over 6 years ago

Patch set 7 of change Id7c58d28e4f6e82a708513a3ce9761baf36c06cd has been pushed to the review server.
It is available at http://review.typo3.org/1376

#13 Updated by Mr. Hudson over 6 years ago

Patch set 8 of change Id7c58d28e4f6e82a708513a3ce9761baf36c06cd has been pushed to the review server.
It is available at http://review.typo3.org/1376

#14 Updated by Mr. Hudson over 6 years ago

Patch set 9 of change Id7c58d28e4f6e82a708513a3ce9761baf36c06cd has been pushed to the review server.
It is available at http://review.typo3.org/1376

#15 Updated by Christian Kuhn over 6 years ago

  • Status changed from New to Resolved
  • % Done changed from 0 to 100

#16 Updated by Steffen Gebert over 6 years ago

  • Target version changed from 4.6.0-beta1 to 1281

#17 Updated by Steffen Gebert over 6 years ago

  • TYPO3 Version changed from 4.5 to 4.6

#18 Updated by Xavier Perseguers almost 6 years ago

  • Status changed from Resolved to Closed

#19 Updated by Ernesto Baschny over 4 years ago

  • Target version deleted (1281)

Also available in: Atom PDF