nasty.
docu link: https://docs.typo3.org/typo3cms/TSconfigReference/PageTsconfig/TCEform/Index.html
docu example:
TCEFORM.tt_content.pi_flexform.my_ext_pi1.sDEF.myField.label = ...
All the other settings like addItems and friends are handled properly.
The issue is here that the label value is determined very late in the container structure, according code can be found in PaletteAndSingleContainer getSingleFieldLabel() and similar stuff is not done in FlexFormElementContainer. The problem is that the pageTsConfig / fieldTSconfig / fieldTSConfig (sic!) is not properly available for flex form fields in this area anymore, it is rather hidden in the "global" pageTsConfig and can not be accessed that easily anymore. I guess this could be the root cause for further different page ts config related issues with flex forms, too.
In this case it seems to be best to handle the full pageTsConfig mapping onto the tca / fake-tca-of-flex for the label stuff in one or multiple data providers instead of doing that within the containers. basically, getSingleFieldLabel() should be merged into or created as new provider and the result can probably just be set into the according tca stuff.
This would reduce the overall dependency of the container / element render-structure to page ts config, and that would be a good idea anyway.