Linux Archive

Linux Archive (http://www.linux-archive.org/)
-   Gentoo Portage Developer (http://www.linux-archive.org/gentoo-portage-developer/)
-   -   GLEP59: Change live Manifest2 hashes to SHA256, SHA512, WHIRLPOOL (http://www.linux-archive.org/gentoo-portage-developer/581877-glep59-change-live-manifest2-hashes-sha256-sha512-whirlpool.html)

"Robin H. Johnson" 09-30-2011 01:27 AM

GLEP59: Change live Manifest2 hashes to SHA256, SHA512, WHIRLPOOL
 
Change Manifest2 hashes to a more secure set as approved in GLEP59.
SHA512 and WHIRLPOOL are added, SHA1 and RMD160 are dropped.

SHA256 is now the lowest security hash, and must remain in Manifest
files for at least 1 year, otherwise older Portage installs will
complain that they do not support any of the hashes in the Manifest
files.

Future events:
After 2012/10/01:
- Change MANIFEST2_REQUIRED_HASH to WHIRLPOOL.
- Remove SHA256 from MANIFEST2_HASH_FUNCTIONS.
After SHA-3 is approved:
- Add new hashes to MANIFEST2_HASH_FUNCTIONS.

Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
---
pym/portage/const.py | 6 ++++--
1 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/pym/portage/const.py b/pym/portage/const.py
index 8b5f4ac..a42ebe8 100644
--- a/pym/portage/const.py
+++ b/pym/portage/const.py
@@ -109,10 +109,12 @@ EAPI = 4

HASHING_BLOCKSIZE = 32768
MANIFEST1_HASH_FUNCTIONS = ("MD5", "SHA256", "RMD160")
-MANIFEST2_HASH_FUNCTIONS = ("SHA1", "SHA256", "RMD160")
+MANIFEST2_HASH_FUNCTIONS = ("SHA256", "SHA512", "WHIRLPOOL")
+# FUTURE: Add SHA-3 when available; remove SHA256 after 2012/10/01

MANIFEST1_REQUIRED_HASH = "MD5"
-MANIFEST2_REQUIRED_HASH = "SHA1"
+MANIFEST2_REQUIRED_HASH = "SHA256"
+# FUTURE: Change to WHIRLPOOL after 2012/10/01

MANIFEST2_IDENTIFIERS = ("AUX", "MISC", "DIST", "EBUILD")
# ================================================== =========================
--
1.7.6

"Robin H. Johnson" 10-01-2011 07:40 AM

GLEP59: Change live Manifest2 hashes to SHA256, SHA512, WHIRLPOOL
 
From: "Robin H. Johnson" <robbat2@gentoo.org>

Change Manifest2 hashes to a more secure set as approved in GLEP59.
SHA512 and WHIRLPOOL are added, SHA1 and RMD160 are dropped.

SHA256 is now the lowest security hash, and must remain in Manifest
files for at least 1 year, otherwise older Portage installs will
complain that they do not support any of the hashes in the Manifest
files.

Future events:
After 2012/10/01:
- Change MANIFEST2_REQUIRED_HASH to WHIRLPOOL.
- Remove SHA256 from MANIFEST2_HASH_FUNCTIONS.
After SHA-3 is approved:
- Add new hashes to MANIFEST2_HASH_FUNCTIONS.

Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
---
pym/portage/const.py | 6 ++++--
1 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/pym/portage/const.py b/pym/portage/const.py
index 8b5f4ac..a42ebe8 100644
--- a/pym/portage/const.py
+++ b/pym/portage/const.py
@@ -109,10 +109,12 @@ EAPI = 4

HASHING_BLOCKSIZE = 32768
MANIFEST1_HASH_FUNCTIONS = ("MD5", "SHA256", "RMD160")
-MANIFEST2_HASH_FUNCTIONS = ("SHA1", "SHA256", "RMD160")
+MANIFEST2_HASH_FUNCTIONS = ("SHA256", "SHA512", "WHIRLPOOL")
+# FUTURE: Add SHA-3 when available; remove SHA256 after 2012/10/01

MANIFEST1_REQUIRED_HASH = "MD5"
-MANIFEST2_REQUIRED_HASH = "SHA1"
+MANIFEST2_REQUIRED_HASH = "SHA256"
+# FUTURE: Change to WHIRLPOOL after 2012/10/01

MANIFEST2_IDENTIFIERS = ("AUX", "MISC", "DIST", "EBUILD")
# ================================================== =========================
--
1.7.7

Zac Medico 10-02-2011 04:40 AM

GLEP59: Change live Manifest2 hashes to SHA256, SHA512, WHIRLPOOL
 
On 10/01/2011 12:40 AM, Robin H. Johnson wrote:
> diff --git a/pym/portage/const.py b/pym/portage/const.py
> index 8b5f4ac..a42ebe8 100644
> --- a/pym/portage/const.py
> +++ b/pym/portage/const.py
> @@ -109,10 +109,12 @@ EAPI = 4
>
> HASHING_BLOCKSIZE = 32768
> MANIFEST1_HASH_FUNCTIONS = ("MD5", "SHA256", "RMD160")
> -MANIFEST2_HASH_FUNCTIONS = ("SHA1", "SHA256", "RMD160")
> +MANIFEST2_HASH_FUNCTIONS = ("SHA256", "SHA512", "WHIRLPOOL")
> +# FUTURE: Add SHA-3 when available; remove SHA256 after 2012/10/01
>
> MANIFEST1_REQUIRED_HASH = "MD5"
> -MANIFEST2_REQUIRED_HASH = "SHA1"
> +MANIFEST2_REQUIRED_HASH = "SHA256"
> +# FUTURE: Change to WHIRLPOOL after 2012/10/01
>
> MANIFEST2_IDENTIFIERS = ("AUX", "MISC", "DIST", "EBUILD")
> # ================================================== =========================

If we control these hashes via metadata/layout.conf, then we can toggle
it atomically for all commiters. Otherwise, we'll have an annoying
period of time where different committers are committing different sets
of hashes, depending on their portage version.
--
Thanks,
Zac

Zac Medico 10-02-2011 06:14 AM

GLEP59: Change live Manifest2 hashes to SHA256, SHA512, WHIRLPOOL
 
On 10/01/2011 09:40 PM, Zac Medico wrote:
> On 10/01/2011 12:40 AM, Robin H. Johnson wrote:
>> diff --git a/pym/portage/const.py b/pym/portage/const.py
>> index 8b5f4ac..a42ebe8 100644
>> --- a/pym/portage/const.py
>> +++ b/pym/portage/const.py
>> @@ -109,10 +109,12 @@ EAPI = 4
>>
>> HASHING_BLOCKSIZE = 32768
>> MANIFEST1_HASH_FUNCTIONS = ("MD5", "SHA256", "RMD160")
>> -MANIFEST2_HASH_FUNCTIONS = ("SHA1", "SHA256", "RMD160")
>> +MANIFEST2_HASH_FUNCTIONS = ("SHA256", "SHA512", "WHIRLPOOL")
>> +# FUTURE: Add SHA-3 when available; remove SHA256 after 2012/10/01
>>
>> MANIFEST1_REQUIRED_HASH = "MD5"
>> -MANIFEST2_REQUIRED_HASH = "SHA1"
>> +MANIFEST2_REQUIRED_HASH = "SHA256"
>> +# FUTURE: Change to WHIRLPOOL after 2012/10/01
>>
>> MANIFEST2_IDENTIFIERS = ("AUX", "MISC", "DIST", "EBUILD")
>> # ================================================== =========================
>
> If we control these hashes via metadata/layout.conf, then we can toggle
> it atomically for all commiters. Otherwise, we'll have an annoying
> period of time where different committers are committing different sets
> of hashes, depending on their portage version.

I've applied the whole series, except for 5/5:

http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=f27473d04e6dee44983d1e5ac32 ea9d4d375b5a2
http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=f3b05d6eed63e19cdfa7f645cf0 190ee8019dd90
http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=8ac29097395f24ad331602d8e87 fdf105ebd972b
http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=faf87ba9877e3b5a7866c6649f9 56f15950e789a

--
Thanks,
Zac

"Robin H. Johnson" 10-02-2011 12:46 PM

GLEP59: Change live Manifest2 hashes to SHA256, SHA512, WHIRLPOOL
 
On Sat, Oct 01, 2011 at 09:40:13PM -0700, Zac Medico wrote:
> If we control these hashes via metadata/layout.conf, then we can toggle
> it atomically for all commiters. Otherwise, we'll have an annoying
> period of time where different committers are committing different sets
> of hashes, depending on their portage version.
How do you suggest doing it via layout.conf? I've kept SHA256 in both
sets for now, but if you could enforce new signatures including both
WHIRLPOOL and SHA256, that would be great.

--
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail : robbat2@gentoo.org
GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85

Zac Medico 10-02-2011 08:39 PM

GLEP59: Change live Manifest2 hashes to SHA256, SHA512, WHIRLPOOL
 
On 10/02/2011 05:46 AM, Robin H. Johnson wrote:
> On Sat, Oct 01, 2011 at 09:40:13PM -0700, Zac Medico wrote:
>> If we control these hashes via metadata/layout.conf, then we can toggle
>> it atomically for all commiters. Otherwise, we'll have an annoying
>> period of time where different committers are committing different sets
>> of hashes, depending on their portage version.
> How do you suggest doing it via layout.conf? I've kept SHA256 in both
> sets for now, but if you could enforce new signatures including both
> WHIRLPOOL and SHA256, that would be great.

How about if we put something like this in
gentoo-x86/metadata/layout.conf now:

manifest2-sha1 = true
manifest2-whirlpool = false

Then we'll patch portage so that by default it will disable SHA1 and
enable WHIRLPOOL, and the above settings will override the defaults.
After the patched portage is marked stable in a month or so, we'll send
an announcement to gentoo-announce, and remove the above settings from
layout.conf.
--
Thanks,
Zac


Mon Oct 3 00:30:02 2011
Return-path: <gentoo-dev+bounces-47970-tom=linux-archive.org@lists.gentoo.org>
Envelope-to: tom@linux-archive.org
Delivery-date: Sun, 02 Oct 2011 23:33:21 +0300
Received: from pigeon.gentoo.org ([208.92.234.80]:48421 helo=lists.gentoo.org)
by s2.java-tips.org with esmtps (TLSv1:AES256-SHA:256)
(Exim 4.69)
(envelope-from <gentoo-dev+bounces-47970-tom=linux-archive.org@lists.gentoo.org>)
id 1RASiu-00063W-VN
for tom@linux-archive.org; Sun, 02 Oct 2011 23:33:21 +0300
Received: from pigeon.gentoo.org (localhost [127.0.0.1])
by pigeon.gentoo.org (Postfix) with SMTP id D262021C14F;
Sun, 2 Oct 2011 20:42:18 +0000 (UTC)
X-Original-To: gentoo-dev@lists.gentoo.org
Delivered-To: gentoo-dev@lists.gentoo.org
Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183])
by pigeon.gentoo.org (Postfix) with ESMTP id CD8D621C137
for <gentoo-dev@lists.gentoo.org>; Sun, 2 Oct 2011 20:40:22 +0000 (UTC)
Received: from [192.168.178.24] (e178067176.adsl.alicedsl.de [85.178.67.176])
(using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits))
(No client certificate requested)
(Authenticated sender: chithanh)
by smtp.gentoo.org (Postfix) with ESMTPSA id E54AB1B401F
for <gentoo-dev@lists.gentoo.org>; Sun, 2 Oct 2011 20:40:21 +0000 (UTC)
Message-ID: <4E88CC32.9020708@gentoo.org>
Date: Sun, 02 Oct 2011 22:40:18 +0200
From: =?UTF-8?B?Q2jDrS1UaGFuaCBDaHJpc3RvcGhlciBOZ3V54buFbg==?=
<chithanh@gentoo.org>
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:6.0) Gecko/20110827 Firefox/6.0 SeaMonkey/2.3.1
Precedence: bulk
List-Post: <mailto:gentoo-dev@lists.gentoo.org>
List-Help: <mailto:gentoo-dev+help@lists.gentoo.org>
List-Unsubscribe: <mailto:gentoo-dev+unsubscribe@lists.gentoo.org>
List-Subscribe: <mailto:gentoo-dev+subscribe@lists.gentoo.org>
List-Id: Gentoo Linux mail <gentoo-dev.gentoo.org>
X-BeenThere: gentoo-dev@lists.gentoo.org
Reply-to: gentoo-dev@lists.gentoo.org
MIME-Version: 1.0
To: gentoo-dev@lists.gentoo.org
Subject: Re: [gentoo-dev] Re: [gentoo-commits] gentoo-x86 commit in net-im/qutecom:
metadata.xml ChangeLog qutecom-2.2_p20110210.ebuild
References: <20111001170259.E4D702004B@flycatcher.gentoo.org > <201110021420.47397.vapier@gentoo.org> <4E88C2DE.6000005@gentoo.org> <201110021611.03344.vapier@gentoo.org>
In-Reply-To: <201110021611.03344.vapier@gentoo.org>
X-Enigmail-Version: 1.3
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Mike Frysinger schrieb:
> the system is functioning wrongly because you're forcing users to needl=
essly=20
> upgrade/downgrade packages. in addition, packages in the tree aren't t=
he only=20
> things to be considered. if the user is building code that works fine =
against=20
> the latest stable, but your package forced it to downgrade, they might =
no=20
> longer build correctly.

Then the code is broken that is built outside portage and does not
function correctly with old linux-headers without doing any kind of
version check.

And again, downgrade of dependencies it is not against any rule which
would justify mask and removal.

Another example from the X.org packages, installing the proprietary
ATI/NVidia drivers will cause downgrades for xorg-server on ~arch
systems. Nobody in his right mind is proposing to treeclean them because
of this.

>> Not by surprise treecleaning of packages.
>=20
> as you were already shown, this wasn't really a surprise. it went thro=
ugh the=20
> normal announce process, albeit not the normal 30 day grace period.

The whole process was a surprise to me because the masking and
treecleaning happened while I was on 20 days of devaway. I leave the
away message for a day more in case anyone wants to verify.

And it was a surprise treecleaning because the mask and policy said 30
days, but the removal happened before the 30 days were over.

The second time the package was removed was even without mask or
announcement.

>>> further, when the newer version gets stabilized and then the older on=
es
>>> dropped, what then ? your package is broken.
>>
>> Yes, when the older one is dropped _that_ would be reason for
>> masking+removal. However I have not seen any plans of doing so. Actual=
ly
>> the current amd64 stable 2.6 versions are 35, 26 and 10 months old
>> respectively, I wouldn't expect that to happen any time soon.
>=20
> sorry, but that's irrelevant. the lack of tree-cleaning is more due to=
=20
> missing automatic generation of ChangeLog files. but if this is going =
to be a=20
> sticking point for you, i can simply clean the tree as soon as we get n=
ewer=20
> stable versions.

If the old versions and reverse dependencies are dropped in accordance
with
http://www.gentoo.org/proj/en/devrel/handbook/handbook.xml?part=3D2&chap=3D=
5#doc_chap7
then I won't complain.


Best regards,
Ch=C3=AD-Thanh Christopher Nguy=E1=BB=85n

Alec Warner 10-02-2011 08:46 PM

GLEP59: Change live Manifest2 hashes to SHA256, SHA512, WHIRLPOOL
 
On Sun, Oct 2, 2011 at 1:39 PM, Zac Medico <zmedico@gentoo.org> wrote:
> On 10/02/2011 05:46 AM, Robin H. Johnson wrote:
>> On Sat, Oct 01, 2011 at 09:40:13PM -0700, Zac Medico wrote:
>>> If we control these hashes via metadata/layout.conf, then we can toggle
>>> it atomically for all commiters. Otherwise, we'll have an annoying
>>> period of time where different committers are committing different sets
>>> of hashes, depending on their portage version.
>> How do you suggest doing it via layout.conf? I've kept SHA256 in both
>> sets for now, but if you could enforce new signatures including both
>> WHIRLPOOL and SHA256, that would be great.
>
> How about if we put something like this in
> gentoo-x86/metadata/layout.conf now:

Reminds me, I was going to do an analysis on -commit mails to track
portage versions; I'll do that now.

>
> * manifest2-sha1 = true
> * manifest2-whirlpool = false
>
> Then we'll patch portage so that by default it will disable SHA1 and
> enable WHIRLPOOL, and the above settings will override the defaults.
> After the patched portage is marked stable in a month or so, we'll send
> an announcement to gentoo-announce, and remove the above settings from
> layout.conf.
> --
> Thanks,
> Zac
>
>

"Robin H. Johnson" 10-02-2011 08:54 PM

GLEP59: Change live Manifest2 hashes to SHA256, SHA512, WHIRLPOOL
 
On Sun, Oct 02, 2011 at 01:39:41PM -0700, Zac Medico wrote:
> On 10/02/2011 05:46 AM, Robin H. Johnson wrote:
> > On Sat, Oct 01, 2011 at 09:40:13PM -0700, Zac Medico wrote:
> >> If we control these hashes via metadata/layout.conf, then we can toggle
> >> it atomically for all commiters. Otherwise, we'll have an annoying
> >> period of time where different committers are committing different sets
> >> of hashes, depending on their portage version.
> > How do you suggest doing it via layout.conf? I've kept SHA256 in both
> > sets for now, but if you could enforce new signatures including both
> > WHIRLPOOL and SHA256, that would be great.
> How about if we put something like this in
> gentoo-x86/metadata/layout.conf now:
Did you mean profiles/layout.conf? I just want to make sure no scripts
that pull from CVS and expect that dir to not exist don't break.

> manifest2-sha1 = true
> manifest2-whirlpool = false
Bikeshedding slightly, but can we figure something like a list or dict
instead? (Also gives us a chance to make the required hashes a list).
manifest2-hashes = ['SHA1', 'SHA256', 'RMD160']

> Then we'll patch portage so that by default it will disable SHA1 and
> enable WHIRLPOOL, and the above settings will override the defaults.
> After the patched portage is marked stable in a month or so, we'll send
> an announcement to gentoo-announce, and remove the above settings from
> layout.conf.
Sounds good to me. Hopefully I'll have more of the MetaManifest
prototype code in the next few days to go live around the same time.

--
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail : robbat2@gentoo.org
GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85

Zac Medico 10-02-2011 09:10 PM

GLEP59: Change live Manifest2 hashes to SHA256, SHA512, WHIRLPOOL
 
On 10/02/2011 01:54 PM, Robin H. Johnson wrote:
> On Sun, Oct 02, 2011 at 01:39:41PM -0700, Zac Medico wrote:
>> On 10/02/2011 05:46 AM, Robin H. Johnson wrote:
>>> On Sat, Oct 01, 2011 at 09:40:13PM -0700, Zac Medico wrote:
>>>> If we control these hashes via metadata/layout.conf, then we can toggle
>>>> it atomically for all commiters. Otherwise, we'll have an annoying
>>>> period of time where different committers are committing different sets
>>>> of hashes, depending on their portage version.
>>> How do you suggest doing it via layout.conf? I've kept SHA256 in both
>>> sets for now, but if you could enforce new signatures including both
>>> WHIRLPOOL and SHA256, that would be great.
>> How about if we put something like this in
>> gentoo-x86/metadata/layout.conf now:
> Did you mean profiles/layout.conf? I just want to make sure no scripts
> that pull from CVS and expect that dir to not exist don't break.

No, it's metadata/layout.conf. I didn't choose the location. We actually
inherited it from paludis about 1.5 years ago:


http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=f16aee82cefa95e9903fa46f448 d30f6d4350f64

We're also using it to control thin-manifest support, among other things
now:

https://bugs.gentoo.org/show_bug.cgi?id=333691

>> manifest2-sha1 = true
>> manifest2-whirlpool = false
> Bikeshedding slightly, but can we figure something like a list or dict
> instead? (Also gives us a chance to make the required hashes a list).
> manifest2-hashes = ['SHA1', 'SHA256', 'RMD160']

Well, booleans are simpler. Also, note that I designed them to be
removed from layout.conf eventually, which means that we will accumulate
less bloat in layout.conf over time.

>> Then we'll patch portage so that by default it will disable SHA1 and
>> enable WHIRLPOOL, and the above settings will override the defaults.
>> After the patched portage is marked stable in a month or so, we'll send
>> an announcement to gentoo-announce, and remove the above settings from
>> layout.conf.
> Sounds good to me. Hopefully I'll have more of the MetaManifest
> prototype code in the next few days to go live around the same time.

I'll see if I can get a layout.conf patch done today.
--
Thanks,
Zac

Brian Harring 10-02-2011 11:22 PM

GLEP59: Change live Manifest2 hashes to SHA256, SHA512, WHIRLPOOL
 
On Sun, Oct 02, 2011 at 02:10:09PM -0700, Zac Medico wrote:
> On 10/02/2011 01:54 PM, Robin H. Johnson wrote:
> > On Sun, Oct 02, 2011 at 01:39:41PM -0700, Zac Medico wrote:
> >> On 10/02/2011 05:46 AM, Robin H. Johnson wrote:
> >>> On Sat, Oct 01, 2011 at 09:40:13PM -0700, Zac Medico wrote:
> >>>> If we control these hashes via metadata/layout.conf, then we can toggle
> >>>> it atomically for all commiters. Otherwise, we'll have an annoying
> >>>> period of time where different committers are committing different sets
> >>>> of hashes, depending on their portage version.
> >>> How do you suggest doing it via layout.conf? I've kept SHA256 in both
> >>> sets for now, but if you could enforce new signatures including both
> >>> WHIRLPOOL and SHA256, that would be great.
> >> How about if we put something like this in
> >> gentoo-x86/metadata/layout.conf now:
> > Did you mean profiles/layout.conf? I just want to make sure no scripts
> > that pull from CVS and expect that dir to not exist don't break.
>
> No, it's metadata/layout.conf. I didn't choose the location. We actually
> inherited it from paludis about 1.5 years ago:
>
>
> http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=f16aee82cefa95e9903fa46f448 d30f6d4350f64
>
> We're also using it to control thin-manifest support, among other things
> now:
>
> https://bugs.gentoo.org/show_bug.cgi?id=333691
>
> >> manifest2-sha1 = true
> >> manifest2-whirlpool = false
> > Bikeshedding slightly, but can we figure something like a list or dict
> > instead? (Also gives us a chance to make the required hashes a list).
> > manifest2-hashes = ['SHA1', 'SHA256', 'RMD160']
>
> Well, booleans are simpler. Also, note that I designed them to be
> removed from layout.conf eventually, which means that we will accumulate
> less bloat in layout.conf over time.

Should use a space delimited list instead named hashes instead; those
being the hashes that should be generated, and that can be /used/.
Not in the list, not an acceptable hash (even if a manifest2 carries
that data).

If it's not set, then the pm defaults in a list; that default list
should be tracked somewhere (rather than just whatever the PM author
decides) also, although that's a seperate discussion.

Breaking it out into individual booleans isn't particularly great; we
use lists for masters, a tristate for use-manifest, etc. Having each
CHF controlled by a seperate boolean adds more toggles than is worth
it imo, and having the manifest2- prefix makes the parsing slightly
more complex while also making the key name a bit daft if we ever
switch to a manifest3. ;)

~harring


All times are GMT. The time now is 12:51 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.