TYPO3 Core

Just checked that since 4.7 it's possible to give multiple video/audio-sources and so at least the "multiformat-feature" is already possible there.
With 4.7 also "videojs" was added as a contrib-library to the TYPO3-core.

About the "specially cropped version of an image" I think that this has been achieved in version 8 (in a different way) with the "multiple cropping variants" (https://docs.typo3.org/typo3cms/extensions/core/8.7/Changelog/8.6/Feature-75880-ImplementMultipleCroppingVariantsInImageManipulationTool.html)

I was thinking of video-tags with multiple sources (separate files, but belonging together). Like here:

https://www.w3schools.com/html/html5_video.asp
<video width="320" height="240" controls>
<source src="movie.mp4" type="video/mp4">
<source src="movie.ogg" type="video/ogg">
Your browser does not support the video tag.
</video>

Of course it could be rendered different in the frontend, if need be.

What about creating a 1:n relation from sys_file_metadata to other sys_files. In sys_files a additional field for picking the resolution would be helpful and the mime_type and extension columns already provide the file format. Now one could choose variant files in the metadata once and would get those directly by choosing the base file in a media content element. Now things could be rendered with source tags like mentioned above or inserted with javascript or whatever else, because you get all of the data for all file variants passed into the fluid template.
I'm not so familiar with all the side effects that come with the relations, localization, workspaces, etc. but would this be a possible or good way to solve it?