By far the easiest way to copy SSH authorized public keys is using the ssh-copy-id tool. On current versions of OpenSSH you just pass the port with -p, like you would with ssh:
# ssh-copy-id -p 1234 [email protected]
Older versions of the script didn’t support a -p flag and had a quirky syntax issue on a non-standard port, where you had to cram the host and port into a single quoted argument:
# ssh-copy-id "[email protected] -p 1234"
If you’re on anything reasonably recent, stick to the -p form above.