TLS: file ldaprov1.crt does not end in [.0] - does not appear to be a
CA certificate directory file with a properly hashed file name -
skipping.
TLS: file cacert.pem does not end in [.0] - does not appear to be a
CA certificate directory file with a properly hashed file name -
skipping.
TLS: file ldaprov1.key does not end in [.0] - does not appear to be a
CA certificate directory file with a properly hashed file name -
skipping.

.....

I tell slapd to look for specific files but how come it is still
checking in a directory?

I don't know. What does /etc/openldap/ldap.conf say? Do you have a
~/.ldaprc or ~/ldaprc for the user "ldap"?

So even for slapd the setting TLS_CACERTDIR in /etc/openldap/ldap.conf
takes precedence over olcTLSCACertificateFile in cn=config? I set
/etc/openldap/ldap.conf for client only and did not mean it for slapd.

Now after I removed it from /etc/openldap/ldap.conf, ldapsearch -d 1 is
indicating the CA certificate not valid: