Linux Archive

Linux Archive (http://www.linux-archive.org/)
-   Fedora Development (http://www.linux-archive.org/fedora-development/)
-   -   Building production machines out-of-place, regenerating certs when a machine's identity changes, etc. (http://www.linux-archive.org/fedora-development/458181-building-production-machines-out-place-regenerating-certs-when-machines-identity-changes-etc.html)

nodata 11-27-2010 02:15 PM

Building production machines out-of-place, regenerating certs when a machine's identity changes, etc.
 
On 26/11/10 23:47, Philip Prindeville wrote:
> I recently rebuilt a failing mail server (sendmail and cyrus-imapd), replacing the hardware and building the replacement machine offline (leaving the current server in place while I did so).
>
> This would seem normal enough to do, but had some unintended pitfalls that really should be more addressable.
>
> Because the new machine was built offline while the old machine was still in-place, all of the packages that generate certificates--and there are a fair number of them:
>
> sendmail
> cyrus-imapd
> openssl
> openssh
> httpd/mod_ssl
>
> did so incorrectly. That is, they ended up using localhost.localdomain as the identity of the machine... and once I had the machine in place, I brought it up to find that various clients were now complaining about invalid certs.
>
> So I had to dig through what had happened, why, and what needed to be done (or redone) to fix this.
>
> Here are some observations and their accompanying conclusions.
>
> * A lot of packages generate their certs silently as part of the %post scripting
>
> This is certainly true of mod_ssl and cyrus-imapd.
>
> * A lot of packages also provide default answers to the certificate fields:
>
> Running:
>
> # rpm -q --scripts mod_ssl
>
> will get you:
>
> ...
> at<< EOF | /usr/bin/openssl req -new -key /etc/pki/tls/private/localhost.key
> -x509 -days 365 -set_serial $RANDOM
> -out /etc/pki/tls/certs/localhost.crt 2>/dev/null
> --
> SomeState
> SomeCity
> SomeOrganization
> SomeOrganizationalUnit
> ${FQDN}
> root@${FQDN}
> EOF
> ...
>
>
> or
>
> # rpm -q --scripts cyrus-imapd
> ...
> /bin/cat<< EOF | make cyrus-imapd.pem
> --
> SomeState
> SomeCity
> SomeOrganization
> SomeOrganizationalUnit
> localhost.localdomain
> root@localhost.localdomain
> EOF
> ...
>
> etc. It seems to be fairly common.
>
> Openssl provides the script /etc/pki/tls/certs/make-dummy-cert, which contains the function answers():
>
>
> answers() {
> echo --
> echo SomeState
> echo SomeCity
> echo SomeOrganization
> echo SomeOrganizationalUnit
> echo localhost.localdomain
> echo root@localhost.localdomain
> }
>
> which is used to pipe form data to "openssl req" when generating a new certificate request.
>
>
> This seems to be duplicated in several places, and indeed some of the information could easily be captured statically at install time, and other data gleaned from the running system (such as the last two fields).
>
> * The certificate fields should be consistent
>
> The easiest way to do this is to have a standard file containing the first 5 fields above (C, ST, L, O, OU). This file could be populated by Anaconda/Kickstart during installation.
>
> It's presence would then be a requirement for other packages that dynamically create certs at install time.
>
> * A permanent or 'real' identity should be an RPM requirement
>
> Packages should have the ability to have the system configured as its ultimate identity before being installed.
>
> This could be accomplished by an RPM pseudo-requirement that some package generates (via Provides:) as part of its installation.
>
> This could be the same package that provides the certificate "seed" information, but would also check to make sure that hostname, the domain, etc. were proper values.
>
> * If a machine gets relocated, it should be able to re-seed its certificates
>
> This last one is perhaps the most obscure change.
>
> If I build a machine "offline", test it with a temporary identity, and decide its "ready for prime-time", I should be able to reboot it with its new identity, and manually re-run the RPM scripts (or some idempotent portion of them) that regenerates all of the identity-derived information... typically this would be based on hostname, domain, IP address, and the certificate seed information (C, ST, L, O, and OU).
>
> We'd need a %post subsection that could be run idempotently to generate new certificates, for instance.
>
> In the examples of the above packages (openssl, mod_ssl, cyrus-imapd, etc) new .crt, .key, or .pem files would be generated... the serial number incremented (where necessary), and the service restarted.
>
> * This affects a lot more than just the packages that generate certs and keys
>
> This also would affect rpm (and possibly yum as well), in addition to the Anaconda/Kickstart.
>
> It might even be necessary to have a new package that runs early at boot time that checks packages (services) for "refreshed" certificates, and disables (chkconfig service off) any services that have out-of-date certificates/keys.
>
>
> So how far into left field am I on this, anyway?
>
> Thanks,
>
> -Philip
>
>

I don't agree. If you are replacing a production machine, you take the
keys from the old machine and use them. If you don't want to do that,
you buy new, probably stronger, certificates that are also valid. I
think your case only covers self-signed certificates.
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Ralf Ertzinger 11-27-2010 02:44 PM

Building production machines out-of-place, regenerating certs when a machine's identity changes, etc.
 
Hi.

On Sat, 27 Nov 2010 16:15:47 +0100, nodata wrote

> I don't agree. If you are replacing a production machine, you take
> the keys from the old machine and use them. If you don't want to do
> that, you buy new, probably stronger, certificates that are also
> valid. I think your case only covers self-signed certificates.

I agree, usually the keys from the old machine are imported into the new.
I do, however, question the usefulness of generating self signed keys
for 'localhost' or 'localhost.localdomain'. Is there any valid use
case for these?
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

nodata 11-27-2010 07:09 PM

Building production machines out-of-place, regenerating certs when a machine's identity changes, etc.
 
On 27/11/10 16:44, Ralf Ertzinger wrote:
> Hi.
>
> On Sat, 27 Nov 2010 16:15:47 +0100, nodata wrote
>
>> I don't agree. If you are replacing a production machine, you take
>> the keys from the old machine and use them. If you don't want to do
>> that, you buy new, probably stronger, certificates that are also
>> valid. I think your case only covers self-signed certificates.
>
> I agree, usually the keys from the old machine are imported into the new.
> I do, however, question the usefulness of generating self signed keys
> for 'localhost' or 'localhost.localdomain'. Is there any valid use
> case for these?

Not normally, no.

localhost is a catchall for when either your hosts file is badly
configured or you didn't configure networking yet. So we're back to the
problem you mentioned of these things running from rpm scriptlets.

Maybe the sshd approach would be better - generate at first run of the
daemon?
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Philip Prindeville 11-27-2010 08:04 PM

Building production machines out-of-place, regenerating certs when a machine's identity changes, etc.
 
On 11/27/10 8:15 AM, nodata wrote:
> On 26/11/10 23:47, Philip Prindeville wrote:
>> I recently rebuilt a failing mail server (sendmail and cyrus-imapd), replacing the hardware and building the replacement machine offline (leaving the current server in place while I did so).
>>
>> This would seem normal enough to do, but had some unintended pitfalls that really should be more addressable.
>>
>> Because the new machine was built offline while the old machine was still in-place, all of the packages that generate certificates--and there are a fair number of them:
>>
>> sendmail
>> cyrus-imapd
>> openssl
>> openssh
>> httpd/mod_ssl
>>
>> did so incorrectly. That is, they ended up using localhost.localdomain as the identity of the machine... and once I had the machine in place, I brought it up to find that various clients were now complaining about invalid certs.
>>
>> So I had to dig through what had happened, why, and what needed to be done (or redone) to fix this.
>>
>> Here are some observations and their accompanying conclusions.
>>
>> * A lot of packages generate their certs silently as part of the %post scripting
>>
>> This is certainly true of mod_ssl and cyrus-imapd.
>>
>> * A lot of packages also provide default answers to the certificate fields:
>>
>> Running:
>>
>> # rpm -q --scripts mod_ssl
>>
>> will get you:
>>
>> ...
>> at<< EOF | /usr/bin/openssl req -new -key /etc/pki/tls/private/localhost.key
>> -x509 -days 365 -set_serial $RANDOM
>> -out /etc/pki/tls/certs/localhost.crt 2>/dev/null
>> --
>> SomeState
>> SomeCity
>> SomeOrganization
>> SomeOrganizationalUnit
>> ${FQDN}
>> root@${FQDN}
>> EOF
>> ...
>>
>>
>> or
>>
>> # rpm -q --scripts cyrus-imapd
>> ...
>> /bin/cat<< EOF | make cyrus-imapd.pem
>> --
>> SomeState
>> SomeCity
>> SomeOrganization
>> SomeOrganizationalUnit
>> localhost.localdomain
>> root@localhost.localdomain
>> EOF
>> ...
>>
>> etc. It seems to be fairly common.
>>
>> Openssl provides the script /etc/pki/tls/certs/make-dummy-cert, which contains the function answers():
>>
>>
>> answers() {
>> echo --
>> echo SomeState
>> echo SomeCity
>> echo SomeOrganization
>> echo SomeOrganizationalUnit
>> echo localhost.localdomain
>> echo root@localhost.localdomain
>> }
>>
>> which is used to pipe form data to "openssl req" when generating a new certificate request.
>>
>>
>> This seems to be duplicated in several places, and indeed some of the information could easily be captured statically at install time, and other data gleaned from the running system (such as the last two fields).
>>
>> * The certificate fields should be consistent
>>
>> The easiest way to do this is to have a standard file containing the first 5 fields above (C, ST, L, O, OU). This file could be populated by Anaconda/Kickstart during installation.
>>
>> It's presence would then be a requirement for other packages that dynamically create certs at install time.
>>
>> * A permanent or 'real' identity should be an RPM requirement
>>
>> Packages should have the ability to have the system configured as its ultimate identity before being installed.
>>
>> This could be accomplished by an RPM pseudo-requirement that some package generates (via Provides:) as part of its installation.
>>
>> This could be the same package that provides the certificate "seed" information, but would also check to make sure that hostname, the domain, etc. were proper values.
>>
>> * If a machine gets relocated, it should be able to re-seed its certificates
>>
>> This last one is perhaps the most obscure change.
>>
>> If I build a machine "offline", test it with a temporary identity, and decide its "ready for prime-time", I should be able to reboot it with its new identity, and manually re-run the RPM scripts (or some idempotent portion of them) that regenerates all of the identity-derived information... typically this would be based on hostname, domain, IP address, and the certificate seed information (C, ST, L, O, and OU).
>>
>> We'd need a %post subsection that could be run idempotently to generate new certificates, for instance.
>>
>> In the examples of the above packages (openssl, mod_ssl, cyrus-imapd, etc) new .crt, .key, or .pem files would be generated... the serial number incremented (where necessary), and the service restarted.
>>
>> * This affects a lot more than just the packages that generate certs and keys
>>
>> This also would affect rpm (and possibly yum as well), in addition to the Anaconda/Kickstart.
>>
>> It might even be necessary to have a new package that runs early at boot time that checks packages (services) for "refreshed" certificates, and disables (chkconfig service off) any services that have out-of-date certificates/keys.
>>
>>
>> So how far into left field am I on this, anyway?
>>
>> Thanks,
>>
>> -Philip
>>
>>
> I don't agree. If you are replacing a production machine, you take the
> keys from the old machine and use them. If you don't want to do that,
> you buy new, probably stronger, certificates that are also valid. I
> think your case only covers self-signed certificates.

The keys and certs generated by the RPM scripts are by definition self-signed, since they don't have enough information to be anything else.

-Philip

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Philip Prindeville 11-27-2010 08:05 PM

Building production machines out-of-place, regenerating certs when a machine's identity changes, etc.
 
On 11/27/10 1:09 PM, nodata wrote:
> On 27/11/10 16:44, Ralf Ertzinger wrote:
>> Hi.
>>
>> On Sat, 27 Nov 2010 16:15:47 +0100, nodata wrote
>>
>>> I don't agree. If you are replacing a production machine, you take
>>> the keys from the old machine and use them. If you don't want to do
>>> that, you buy new, probably stronger, certificates that are also
>>> valid. I think your case only covers self-signed certificates.
>> I agree, usually the keys from the old machine are imported into the new.
>> I do, however, question the usefulness of generating self signed keys
>> for 'localhost' or 'localhost.localdomain'. Is there any valid use
>> case for these?
> Not normally, no.
>
> localhost is a catchall for when either your hosts file is badly
> configured or you didn't configure networking yet. So we're back to the
> problem you mentioned of these things running from rpm scriptlets.
>
> Maybe the sshd approach would be better - generate at first run of the
> daemon?

There's no guarantee that the daemon is run while the machine is in a useful state... unless the script refuses to start while the hostname and domain name are unset...

-Philip

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


All times are GMT. The time now is 11:36 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.