Hey Torben,
thank you for your patch. Following, I refer to your patchset 2.
I consider it a good idea to take care about brute-forcing. Having a maximal number by IP for login attempts is good. Please make it configurable via install tool as planned (your note in your code). But please do not enable it by default! e.g. "maxAuthAttempts=-1"
Introducing a separate table is good; naming is not. It should be called sth. like sys_auth_attempts and should contain also the TYPO3 context (BE,FE). So we could also use it for Frontent Users, too.
The newly created class IpUtility suggests to contain generic IP related code. Your code is about blacklisting authentication requests by IP. IMHO naming should cover this by being more specific.
Additionally, all methods are static and by that, hard to test (unit tests). Furthermore, the methods retrieve the IP address on their own. However, the IP addresses should be at least method parameters.
Failed login attempts caused by blacklisting should not throw an exception. Please make sure, that such attempts are properly logged!