TYPO3 Core

Suppose we have the following field declaration in our TCA (taken from the news extension):

 'categories' => [
    'exclude' => 1,
    'l10n_mode' => 'mergeIfNotBlank',
    'label' => $ll . 'tx_news_domain_model_news.categories',
    'config' => [
        'type' => 'select',
        'renderType' => 'selectTree',
        'treeConfig' => [
            'dataProvider' => \GeorgRinger\News\TreeProvider\DatabaseTreeDataProvider::class,
            'parentField' => 'parent',
            'appearance' => [
                'showHeader' => true,
                'allowRecursiveMode' => true,
                'expandAll' => true,
                'maxLevels' => 99,
            ],
        ],
        'MM' => 'sys_category_record_mm',
        'MM_match_fields' => [
            'fieldname' => 'categories',
            'tablenames' => 'tx_news_domain_model_news',
        ],
        'MM_opposite_field' => 'items',
        'foreign_table' => 'sys_category',
        'foreign_table_where' => ' AND (sys_category.sys_language_uid = 0 OR sys_category.l10n_parent = 0) ORDER BY sys_category.sorting',
        'size' => 10,
        'autoSizeMax' => 20,
        'minitems' => 0,
        'maxitems' => 99,
    ]
],

Let's say I have a news entry in the default langauge with some categories selected.
Now I translate that news to let's say french.

In my translation no categories are selected, which is correct because of mergeIfNotBlank.
But when I look at the categories of the translated Article from Fluid or my Extbase controller they're empty when they should inherit from the Article in the default language.

If I add some categories to my translated Article naturally those are shown in the frontend.

Btw: I have tried with both translated categories and with language set to [ALL]