Feature #27357
Make static info tables uid's fixed
| Status: | Resolved | Start date: | 2011-06-10 | |
|---|---|---|---|---|
| Priority: | Should have | Due date: | ||
| Assignee: | Stanislas Rolland | % Done: | 0% |
|
| Category: | - | |||
| Target version: | - | |||
| Votes: | 0 |
Description
With r29148, the Spanish country zone "Albacete" was newly introduced. Unfortunately, this happened by putting it at uid 138, which was already occupied by Alicante, and renumbering all subsequent zones upto 164 ("Baleares"), which became 165 (that number was still free).
The Spanish country zones have been part of static_info_tables for at least 4.5 years - they were already present when Stucki did the initial import of the extension to subversion in 2006.
The problem now is that the data of all people who used the extension before this change is potentially corrupted. The data of all people who started using it afterwards will potentially become corrupted when changing it again. In this particular case I would go for implementing a check script in the extension that checks for potentially corrupted records (changed before the date and containing a value that is in the corrupted range). I don't think there will be many affected users, but nevertheless, this uid change should never happen in the first place.
History
Updated by Franz Holzinger almost 2 years ago
The uid of the country zone entries does not need to remain identical. Only the ISO2 and ISO3 number remain the same. A TYPO3 extension should not rely on the uid, it should instead use the ISO3 code to store the selected data.
Updated by Andreas Wolf almost 2 years ago
I beg to differ... If you store a relation in TYPO3, this is done by standard means in t3lib_TCEmain etc. These all use the uid as an identifier - a different way to generate identifiers is not used anywhere. I agree that this is odd for static data that has its own unique keys, but that's how the TYPO3 Core Engine (TCE) works... Maybe this will be changed in the near future, or at least we implement additional data storages (like triple stores), but currently we have to stick to the uid/pid scheme.
Updated by Franz Holzinger almost 2 years ago
It is better to pay attention how other extensions handle this.
E.g. sr_feuser_register uses these lines to enhance the $TCA for the table 'fe_users'.
'static_info_country' => Array (
'exclude' => 0,
'label' => 'LLL:EXT:sr_feuser_register/locallang_db.xml:fe_users.static_info_country',
'config' => Array (
'type' => 'input',
'size' => '5',
'max' => '3',
'eval' => '',
'default' => ''
)
),
'zone' => Array (
'exclude' => 0,
'label' => 'LLL:EXT:sr_feuser_register/locallang_db.xml:fe_users.zone',
'config' => Array (
'type' => 'input',
'size' => '20',
'max' => '40',
'eval' => 'trim',
'default' => ''
)
),
The same should be used within other extensions in order to have an easier data exchange.
The TCE does not support the dynamic display of the zones anyways. Each country will have different possible zones.
Updated by Andreas Wolf almost 2 years ago
Franz, what you say might be correct, but it misses the point: No programmer would expect a uid of static (!) data to change as long as the entity behind the record stays the same. So would no user, no matter if there's a hint in the manual or not. The uid is the unique identifier for a record, and for imported data, this has to stay the same, even if relations are (should be) handled via a totally different concept.
I also don't agree with you regarding dynamic display of zones - this is of course possible, using an itemProcFunc for select elements. Of course this is rather unpleasant, but if you want to edit these relations via the backend, I don't see an easier way. Additionally, you propose to throw away the standard relation handling of TYPO3 and use a different foreign key concept, right? If we want to do something like this, it should be discussed and implemented in the core, and not some extensions without proper documentation for programmers.
Updated by Franz Holzinger almost 2 years ago
Andreas, I am afraid that your wish with remaining uids cannot be handled. Sometimes new country zones will be added and others deleted. E.g. if more Spanish zones are added, then it would be bad to add them at the end of the zone list. They shall come together with the Spanish zones and not after Zypern zones. I cannot think that such a list would still be understandable if the zones of different countries are mixed up.
The foreign key concept with ISO codes for country zones is internationally defined. If TYPO3 is not able to follow the standards, then it should be improved. You would anyways get in trouble if you want to export records to foreign systems using the only TYPO3 internal uids and not rely on the country codes.
Updated by Andreas Wolf almost 2 years ago
I'm sorry, but that is really ridiculous. Why does it matter that we have consecutive uids for all zones of one country? After all, that's one reason we use relational databases - because we can sort by arbitrary columns in any order. If you inserted a new zone into the "first" country, you would really go for renumbering each and every record in all other countries? I can't believe that...
And btw., the short form of the country abbreviations is not as stable as you said - the abbreviations for all German states were changed back in 2009 from three to two characters. Even though this was maybe perfectly fine because the three characters were wrong in the first place - what should I as a developer use as a stable, reliable foreign key in my database tables? It seems to me that no field of static_info_tables is really immutable.
Regarding your statements about TYPO3 and standards: TYPO3 is of course able to follow standards. You are by no means limited in what you export from the system, and if the built-in methods (e.g. CSV export in the list module) don't suffice your needs, you are free to write your own export filters; and of course you would want to use an internationally recognized and standardized format for that. But that has got nothing to do with the way the data ist stored internally. There we have a clear convention: one entity, one uid, and entites are addressed via table:uid tuples.
The uid of standard records never changes (for the live workspace, i.e. the online website). I agree that we must make some compromises for doing the workspaces stuff, but there is a quite comfortable API for that. Everybody who does TYPO3 development knows about this basic paradigm, and so every extension, especially those providing services for other extensions, should follow it. Anyways, in the case discussed here damage has already occured. We should try avoiding this in the future.
Updated by Franz Holzinger almost 2 years ago
I disagree. This extension is not meant to introduce uids for country zones, but to make the international zone codes available.
zn_country_iso_2 char(2) DEFAULT '' NOT NULL,
zn_country_iso_3 char(3) DEFAULT '' NOT NULL,
zn_country_iso_nr int(11) unsigned DEFAULT '0' NOT NULL,
zn_code varchar(45) DEFAULT '' NOT NULL,
Only codes are used inside of the table fields of static_info_tables.
And there is another reason why the uids need changes:
Image a zone having a name in a special character, 'Ä'. 'Ä' should be sorted equally to 'A'.
Then the sorting would be wrong. So only the uids can be used to provide the sorting.
There are only some country zones available at the moment. So you should consider the zone feature still in an alpha phase where everything can be changed.
Surely a better update script should be added which will do updates instead of inserts.
Updated by Andreas Wolf almost 2 years ago
What you mention could easily be done using a correct collation, which defines the sort order for MySQL. I see that some problems might arise here for countries that require different collations, but I consider this to be rather unlikely for websites in a single language. If you use different languages, you might want to set the collation on a per-language base anyways.
The country zones have been part of static_info_tables for quite some years now, the extension itself is marked as stable in the TER and its version number 2.2.0 would have never made me guess that the available content might be unstable. It's clear that there's still information missing, after all, there's an awful lot of information out there that could be put into the extension. But nobody would expect the existing contents to change if there has been no change in the real world.
I really can't understand why you want to stick to using uids for sorting, which are totally inadequate for that purpose. After all, sorting etc. is exactly what relational databases are good at, without us needing to do their work. The uid is just a TYPO3 concept we put in the database - and our concept says that uids have to be stable.
Updated by Stanislas Rolland almost 2 years ago
Hi Andreas,
I think this is a non-issue.
Any developer using this extension should read the manual: «It is highly recommended not to use the uid of the records as reference! They may change with the next update. Use the ISO codes instead.»
There is nothing new here. This is how the extension was designed and it has been so since initial development by René Fritz. Static Info Tables Manager extension is also designed on the same basis.
Kind regards,
Stan
Updated by Andreas Wolf almost 2 years ago
I still don't see why the uids have to change. It might be convenient for some cases, e.g. the sorting Franz mentioned, but I don't see where that outweighs the fact that it is not in accordance with the concepts used in TYPO3.
OTOH, if the current policy is not changed, a description on how to best use static countries/zones in backend forms for other tables would be great!
Updated by Franz Holzinger almost 2 years ago
The uids change because it should be easy for the developer to edit the sql text files in a text editor and to have a good structure. Therefore the best solution for cthe person who edits the files is to be allowed to renumber the uids.
There is no other tool available then a text editor. Or if you export it from TYPO3, then it is maybe also restructured using the uids.
If a new entry comes then the sql file is better readable if the zones of one country are counted like 1,2, ... 7 and not like 1,600,8000,60. This would otherwise cause much confusion and time consumption to the person who needs to add new zones in the future.
If somebody writes a description about the usage of zones, then this should come into the manual.
Updated by Stanislas Rolland over 1 year ago
- Tracker changed from Bug to Feature
- Subject changed from Data corruption: Spanish country zones to Make static info tables uid's fixed
Updated by Stanislas Rolland over 1 year ago
I think this is a feature request as it clearly requires a significant redesign of the extension and of extension Static Info Tables Manager.
Updated by Stanislas Rolland 3 months ago
- Status changed from New to Resolved
- Assignee set to Stanislas Rolland
- Priority changed from -- undefined -- to Should have
This is now resolved in current trunk.
uid's will be fixed from now on.
Beware however that some records may be marked as deleted, and yet not removed from the tables.
Country zones are now linked to countries by means of an IRRE relation which will make it easy to maintain the tables.
Redundant iso codes will all be maintained by hooks on t3lib_tcemain (TYPO3/Core/DataHandling/DataHandler).