TYPO3 Core

Yes, you will need to use a field of type "group" with internal_type "folder".

There is currently no example in core (at least in TYPO3 7.6), but you could use my extension:
https://github.com/fnagel/pluploadfe
and remove this hotfix: https://github.com/fnagel/pluploadfe/commit/2e8db2074785caf84337cd8abe663bd0f04c1929

ps: You described use case does not trigger any JS error in my TYPO3 7.6 instance (tested with latest Chrome)

I see the same problem in three different TYPO3 installations (all 7.6.4). One of them is a completely raw installation.
As soon as I click on one of the triangle icons to unfold this folders subfolders I get a JS error and the subfolders don't show:

JS error "Uncaught ReferenceError: Tree is not defined" in /typo3/index.php?route=%2Fwizard%2Frecord%2Fbrowse&token=bf3be288847dbc1f2d4198fe24af4fd37f1ad00e&mode=folder&bparams=data[tt_content]²⁷[pi_flexform][data][sDEF][lDEF][settings.folder][vDEF]||||:92

I can reproduce this error whether the folder dialog is called from the flexform of a plugin or from a TCA field.

Hi

I have the same problem with internal_type "folder"
Typo3 7.6.9 Chrome 49

I have the same problem while using alm_gallery in 7.6.9. The flexform config is as follows:

            <settings.foldergallery.flexform.folderPath>
                <TCEforms>
                    <label>LLL:EXT:alm_gallery/Resources/Private/Language/locallang_db.xml:tx_almgallery_foldergallery.folderPath</label>
                    <config>
                        <type>group</type>
                        <internal_type>folder</internal_type>
                        <maxitems>1</maxitems>
                        <size>1</size>
                      </config>
                </TCEforms>
            </settings.foldergallery.flexform.folderPath>

I just switched to a simple folder based gallery, because creating FAL file collections don't work in workspaces. I think this a bad situation, because when developing a gallery with workspaces in mind you can't let your users create file collections, it doesn't work in workspaces. So I thought, okay, let the user pick a folder. Then I ran into this bug. So as a workaround I used the logic that the core uses to create a file collection in a flexform and created an own extension where I generate a select with all the folders the user has access to. If anyone is interested in the flexform and the corresponding itemsProcFunc just message me.

Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/49226

Patch set 1 for branch TYPO3_7-6 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/49312

Verified in TYPO3 CMS 7.6.11: TCA type group with internal_type folder works again!

Thanks a lot Wouter and Benni!