Project

General

Profile

Actions

Feature #44210

closed

Add more effective ways to work with large index

Added by Stefan Neufeind over 11 years ago. Updated about 4 years ago.

Status:
Rejected
Priority:
Should have
Category:
Indexed Search
Target version:
-
Start date:
2012-12-24
Due date:
% Done:

0%

Estimated time:
PHP Version:
Tags:
Complexity:
Sprint Focus:
Needs Decision

Description

Situation:
  • large index with 3,9GB in index_fulltext and 3,1GB in index_rel
  • several websites, languages and therefor quite large
  • trying to reduce index by excluding certain branches from the index (and deleting that data)

Using the regular means to clear data from the index (using Info-module on a subtree) takes "ages" since quite a large number of individual queries are run and the index still is huge.

Actions #1

Updated by Oliver Hader about 11 years ago

  • Target version set to 2222
Actions #2

Updated by Oliver Hader about 11 years ago

  • Project changed from 1382 to TYPO3 Core
Actions #3

Updated by Oliver Hader about 11 years ago

  • Category set to Indexed Search
Actions #4

Updated by Oliver Hader about 11 years ago

  • Target version deleted (2222)
Actions #5

Updated by Mathias Schreiber over 9 years ago

  • Status changed from New to Needs Feedback
  • Assignee set to Mathias Schreiber

Any pointers what to do here?
Large datasets tend to get slow. That's normal.

Actions #6

Updated by Stefan Neufeind over 9 years ago

Maybe some kind of cleanup-task, maybe a scheduler-job or cli-job? So you'd be able to add certain excludes and it would look which parts can get mass-removed from the index-tables. Doing that in a cli-job would be much better because we don't risk timeouts and you'll save the manual searching/clicking to clean up certain parts.

Actions #7

Updated by Alexander Opitz almost 9 years ago

  • Status changed from Needs Feedback to New
Actions #8

Updated by Susanne Moog over 8 years ago

  • Sprint Focus set to PRC
Actions #9

Updated by Susanne Moog over 5 years ago

  • Status changed from New to Rejected

As there was no interest in picking up this topic in the last few years and there are existing extension solutions for handling bigger search needs (like elasticsearch or solr), this issue is closed for now.

Actions #10

Updated by Benni Mack about 4 years ago

  • Sprint Focus changed from PRC to Needs Decision
Actions

Also available in: Atom PDF