When using vault as a CA and generating the local signing cert, try to enable the PKI endpoint's auto-tidy feature with it set to tidy expired issuers.