Bug #104827
closedCKEditor 5: custom transformation do not allow regular expressions due to escaping
100%
Description
CKEditor 5 ships with a "typing" plugin to allow transformations, like "1/2" to ½. This plugin also allows (for example) quote transformations. It ships by default with English quote transformations: "..."
will get “...”
- it does this by a RegExp evaluation. The built-in transformations (including the RegExp ones) work fine. TYPO3 allows to add custom transformations like so:
editor: config: typing: transformations: extra: - { from: 'A', to: 'B' }
For plain transformations this works perfectly fine, but if you want to add custom RegExp transformations (fully analog to the built-in RegExp transformations) this does not work. CKEditor 5 has a wrapper method for this:
editor: config: typing: transformations: extra: - { from: buildQuotesRegExp('#'), to: [ null, '<', null, '>' ] }
When the core converts this to JSON for the CKEditor options, it adds to much escaping for CKEditor to recognize this as a callback to the built-in buildQuotesRegExp
JS method. Also, using direct RegExp syntax in the "from" field also does not work for the same reason.
It does not matter if you insert such configuration via YAML or via PHP (in one of the CKEditor PHP events of ext:rte-ckeditor
), the escaping happens when the final JSON is generated.
The use case in my special case would be to add German quotations „"
. Adding French quotes, for example, should be another common use case.
Adding custom RegExp-enhanced transformations should be possible with TYPO3. If not possible via adjusting the JSON options generation, then there probably should be some special configuration endpoint that inserts the extra transformations into the JS CKEditor constructor as direct JS code.
Updated by S P about 1 month ago
- Related to Bug #102559: CKEditor: RegExp rules in editor.config.htmlSupport not working added
Updated by Garvin Hicking about 1 month ago · Edited
- Status changed from New to Needs Feedback
Could you give an example for what you try to use to make this more easy to investigate (actual yaml code)? And you seem to have debugged the relevant core classes, which responsible pieces of code did you find?
I believe you should be able to perform transformations with the HTMLParser event BeforeTransformTextForRichTextEditorEvent and AfterTransformTextForRichTextEditorEvent and alike (https://docs.typo3.org/m/typo3/reference-coreapi/main/en-us/ApiOverview/Events/Events/Core/Html/AfterTransformTextForPersistenceEvent.html) or the former hooks.
But I do agree this is harder to accomplish and should allow to utilize distinct CKEditor features.
Updated by Garvin Hicking about 1 month ago
It could be though that we cannot easily remove the escaping here, because then arbitrary javascript might get injected. So we may need very specific regexp passthrougu handling here, not allow full unescaped syntax use in the yaml parsing. This needs sensitive inspection.
Updated by S P about 1 month ago · Edited
GeneralUtility::jsonEncodeForHtmlAttribute()
is doing the JSON encoding.
As for actual YAML, just take the one from the ticket. You can exchange the characters in the line
- { from: buildQuotesRegExp('#'), to: [ null, '<', null, '>' ] }
if you like to try regular letters, like
- { from: buildQuotesRegExp('A'), to: [ null, 'B', null, 'B' ] }
it still won't work. The point is, that the single qoutes needed for teh argument of buildQuotesRegExp
are escaped. And if you write direct JS regExp (instead of calling buildQuotesRegExp()
) then you still need to pass to-be-replaced quote characters inside the regexp, which will get escaped in the JSON.
I know that you cannot change the escaping (or any) behaviour of GeneralUtility::jsonEncodeForHtmlAttribute()
because this is heavily used public API since forever. Instead, the code from ext:rte-ckeditor passing the custom config to CKEditor should not use GeneralUtility::jsonEncodeForHtmlAttribute()
but use something own, which is less strict at places where RegExp is allowed by CKEditor 5. Or you need to provide a "wrapper API" that will correctly transform these transformations from the YAML to the CKEditor API syntax.
The events you mention are PHP backend-side when persisting (and are v13-only). I am talking about the on-the-fly client-side JavaScript transformations of CKEditor, of the CKEditor 5 built-in plugin named typing
, so that the transforming already happens while typing.
Updated by Garvin Hicking about 1 month ago
Thank you. That helps a lot.
I think a good way would be to write a YAML meta config for this, like:
transformations: extra: fromRegex: '#' toRegex: { - null - ...
that way, a safe escaping can be done because the actual JS call is wrapped in API and no other "JS injection" can take place.
Hm, thinking about this.
Maybe it's better to actually provide an event, so that actual JS can be provided by an extension, so the whole editor.config array can be adjusted with pure JS.
That would allow more customization, because I believe passing such "dangerous" things in YAML is beyond scope of what a simple configuration layer only based on key/values is supposed to provide.
I haven't yet looked into possibilities to use JS for it, but will investigate. If nothing exists, this might actually become a feature then, not a bugfix, but that's up for discussion then.
Updated by Garvin Hicking about 1 month ago
I've checked a bit into it. The CKEditor is emitted with a LitElement, so you can't easily emit JavaScript from outside of that to "adjust" the options.
Having a PSR-14 event to manipulate the outcome of jsonEncodeForHtmlAttribute
(or patching a different method here) will actually not help, because the attribute "options" is interpreted as JSON, and not JavaScript. So you could not embed a javascript function returning a RegExp object at this place.
It would need to be patched in the ckeditor5.ts implementation.
I see two options, one very specific for your use case:
- Allow a custom YAML/JSON attribute like typing.transformations.extra.fromRegexp
- That field can hold a quoted regexp string like: typing.transformations.extra.fromRegexp = '/(^|\s)(["''])([^"'']*)(["''])$/';
(note that YAML double quoting of a single quote....)
- Parse that in ckeditor5.ts method firstUpdated
- Handle the input JSON/YAML of that specific key, transform it to a "real" RegExp object (and only that. No other custom JS)
- Pass that regExp object to the real @editor.config.typing.transformations.extra.from = realRegexpObject
The other option:
- Quite the same, but instead of "inventing" specific extra keys, make a general suffix like "from-asRegexpObject".
- In the ckeditor5.ts method firstUpdated
iterate/walk the whole this.options structure.
- Whenever a key matches "-asRegExpObject" at the end, convert the value to a RegExp object, and create a new config option key without the suffix, and remove the temporary object again
I think the latter makes more sense for forward compatibility. I'm trying to whip up a prototype for this.
Updated by Gerrit Code Review about 1 month ago
- Status changed from Needs Feedback to Under Review
Patch set 1 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/85910
Updated by Garvin Hicking about 1 month ago
I've pushed a rough draft of this, not sure yet though.
Updated by Gerrit Code Review about 1 month ago
Patch set 2 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/85910
Updated by Gerrit Code Review 30 days ago
Patch set 3 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/85910
Updated by Gerrit Code Review 29 days ago
Patch set 4 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/85910
Updated by Gerrit Code Review 26 days ago
Patch set 1 for branch 12.4 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/86054
Updated by Gerrit Code Review 26 days ago
Patch set 5 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/85910
Updated by Gerrit Code Review 26 days ago
Patch set 2 for branch 12.4 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/86054
Updated by Gerrit Code Review 26 days ago
Patch set 6 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/85910
Updated by Gerrit Code Review 26 days ago
Patch set 7 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/85910
Updated by Gerrit Code Review 26 days ago
Patch set 3 for branch 12.4 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/86054
Updated by Gerrit Code Review 26 days ago
Patch set 8 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/85910
Updated by Garvin Hicking 25 days ago
- Status changed from Under Review to Resolved
- % Done changed from 0 to 100
Applied in changeset e1b38313285fa50f4a8e44abf520dfa01da8720f.