You could probably just try to install your new certificate and private key, reload your webserver config, and see if it works. But that’s not very convenient if you want to validate your private key and certificate beforehand.
So, how do you verify that a private key matches your certificate and that they’re valid?
Calculate MD5 hash of private key
$ openssl rsa -noout -modulus -in /path/to/your/private.key 2> /dev/null | openssl md5 (stdin)= 3a5a1682678d243b6b8337360b55ff10
Calculate MD5 hash of certificate
$ openssl x509 -noout -modulus -in /path/to/your/certificate.crt 2> /dev/null | openssl md5 (stdin)= 3a5a1682678d243b6b8337360b55ff10
Check if they match
The MD5 hash from the private key and the certificate should be the exact same. If they’re not, the private key can not be used together with the certificate and something in the CSR process has probably gone wrong. This can mean a wrong CSR was used, a wrong private key was stored, … Up to you to find out. ;-)