It’s not possible by default, but you can install the yum-utils package that provides tools to list the contents of a certain package.
$ yum -y install yum-utils
Now the repoquery tool is available, that allows you to look into (installed and not-yet-installed) packages.
$ repoquery --list varnish-libs-devel
/usr/include/varnish
/usr/include/varnish/shmlog.h
/usr/include/varnish/shmlog_tags.h
/usr/include/varnish/stat_field.h
/usr/include/varnish/stats.h
/usr/include/varnish/varnishapi.h
...
Very useful to combine with yum whatprovides */something searches to find exactly the package you need!
On modern RHEL/CentOS/Fedora (8 and up), yum has been replaced by dnf and repoquery is built in, so you can skip the yum-utils install and run it directly:
$ dnf repoquery --list varnish-libs-devel
The dnf whatprovides */something equivalent works there too.