After many attempts following tutorials and forums, I eventually I tried a different web server (nginx) with the exact same result. This confirmed that it was not the web server or PHP settings since I made sure the libraries can be found and folders were also set in the environment paths.
Somebody recommended copying the libmysql.dll file into the C:\Windows\System32 folder, but that failed since the file never stayed there and no error message. PHP also did not recognise this file as a valid extension and registering the DLL with Windows did also not help.
Somehow the DLL file ended up mysteriously in the C:\Windows\SysWOW64 folder since my installation of Windows 7 is 64 bit and the same with MySQL. I suppose this is a useful Windows feature to keep some order.
In the end it came down to what none of the forums or tutorials said which is setting things like socket, port, etc. in the "my.ini" file which I had based on the medium example file. Running mysqld or as a service made no difference.