Krzysztof Pawlik schrieb:
> On 30/04/12 10:39, Ulrich Mueller wrote:
>>>>>>> On Mon, 30 Apr 2012, Krzysztof Pawlik wrote:
>>
>>> On 30/04/12 10:12, Ulrich Mueller wrote:
>>>> If the eclass doesn't work with FEATURES="collision-protect" then it
>>>> needs to be fixed.
>>
>>> Long story short: older eclass compiled Python byte code in live
>>> file system, new one does it in src_install so .pyo/.pyc gets
>>> properly recorded,
>>
>> That's a good change.
>
>
>
>>> so the package *has to overwrite* files that are not owned by anyone
>>> (no package owns them).
>>
>> The usual approach to this problem was to have some cleanup code in
>> pkg_setup or pkg_preinst that would remove the old files.
>
> Interesting idea - I see two problems:
> - how I would get the list of files for already installed version (the file
> list can (and will) differ between versions)

Simple workaround in pkg_preinst (just example code, nothing to copy/paste):

if has_version < first_good_version ; then
for i in $(find ${D} -name *.{pyc,pyo}) ; do
[[ -e ${i/${D}/${ROOT}/} ]] && rm ${i/${D}/${ROOT}/}
done
fi

Alternatively, you could take all .py files of the installed versions
and blindly remove the pyo/pyc files for them in pkg_preinst (this
should also prevent leaving dead files around).


--

Thomas Sachau
Gentoo Linux Developer

>>>>> On Mon, 30 Apr 2012, Thomas Sachau wrote:

> Krzysztof Pawlik schrieb:
>> On 30/04/12 10:39, Ulrich Mueller wrote:
>>>>> If the eclass doesn't work with FEATURES="collision-protect"
>>>>> then it needs to be fixed.
>>>
>>>> Long story short: older eclass compiled Python byte code in live
>>>> file system, new one does it in src_install so .pyo/.pyc gets
>>>> properly recorded, so the package *has to overwrite* files that
>>>> are not owned by anyone (no package owns them).
>>>
>>> The usual approach to this problem was to have some cleanup code
>>> in pkg_setup or pkg_preinst that would remove the old files.
>>
>> Interesting idea - I see two problems:
>> - how I would get the list of files for already installed version
>> (the file list can (and will) differ between versions)

> Simple workaround in pkg_preinst (just example code, nothing to
> copy/paste):

> if has_version < first_good_version ; then
> for i in $(find ${D} -name *.{pyc,pyo}) ; do
> [[ -e ${i/${D}/${ROOT}/} ]] && rm ${i/${D}/${ROOT}/}
> done
> fi

This wouldn't work, because collision protection runs _before_
pkg_preinst.

However, one could imagine a (somewhat dirty) workaround:
- At the end of src_install, rename all *.pyc and *.pyo in ${D} to
names that don't collide with the ones in ${ROOT}, so that collision
protection won't be triggered.
- At the beginning of pkg_preinst, rename them back to their original
names.
- Run above code.

> Alternatively, you could take all .py files of the installed
> versions and blindly remove the pyo/pyc files for them in
> pkg_preinst (this should also prevent leaving dead files around).

s/preinst/setup/ otherwise the same argument as above would apply.

Ulrich

On Mon, Apr 30, 2012 at 03:36:17PM +0200, Ulrich Mueller wrote:
> >>>>> On Mon, 30 Apr 2012, Thomas Sachau wrote:
>
> > Krzysztof Pawlik schrieb:
> >> On 30/04/12 10:39, Ulrich Mueller wrote:
> >>>>> If the eclass doesn't work with FEATURES="collision-protect"
> >>>>> then it needs to be fixed.
> >>>
> >>>> Long story short: older eclass compiled Python byte code in live
> >>>> file system, new one does it in src_install so .pyo/.pyc gets
> >>>> properly recorded, so the package *has to overwrite* files that
> >>>> are not owned by anyone (no package owns them).
> >>>
> >>> The usual approach to this problem was to have some cleanup code
> >>> in pkg_setup or pkg_preinst that would remove the old files.
> >>
> >> Interesting idea - I see two problems:
> >> - how I would get the list of files for already installed version
> >> (the file list can (and will) differ between versions)
>
> > Simple workaround in pkg_preinst (just example code, nothing to
> > copy/paste):
>
> > if has_version < first_good_version ; then
> > for i in $(find ${D} -name *.{pyc,pyo}) ; do
> > [[ -e ${i/${D}/${ROOT}/} ]] && rm ${i/${D}/${ROOT}/}
> > done
> > fi
>
> This wouldn't work, because collision protection runs _before_
> pkg_preinst.
>
> However, one could imagine a (somewhat dirty) workaround:
> - At the end of src_install, rename all *.pyc and *.pyo in ${D} to
> names that don't collide with the ones in ${ROOT}, so that collision
> protection won't be triggered.
> - At the beginning of pkg_preinst, rename them back to their original
> names.
> - Run above code.
>
> > Alternatively, you could take all .py files of the installed
> > versions and blindly remove the pyo/pyc files for them in
> > pkg_preinst (this should also prevent leaving dead files around).
>
> s/preinst/setup/ otherwise the same argument as above would apply.

The checks above can misbehave.

Pretty sure I've already suggested this route for this specific issue,
but QA vars in the ebuild that portage can use for hints is a sane,
non-cluster-fucky way to deal with this. Something along the lines of
QA_COLLISION_PROTECT_IGNORE='*.pyc'

Still has the potential for things going boom, but warnings can be
setup for when collision-protect is on, and the portage version
doesn't look new enough (those being details; point is the general
approach).
~harring

On 01/05/12 14:36, Brian Harring wrote:
> On Mon, Apr 30, 2012 at 03:36:17PM +0200, Ulrich Mueller wrote:
>>>>>>> On Mon, 30 Apr 2012, Thomas Sachau wrote:
>>
>>> Krzysztof Pawlik schrieb:
>>>> On 30/04/12 10:39, Ulrich Mueller wrote:
>>>>>>> If the eclass doesn't work with FEATURES="collision-protect"
>>>>>>> then it needs to be fixed.
>>>>>
>>>>>> Long story short: older eclass compiled Python byte code in live
>>>>>> file system, new one does it in src_install so .pyo/.pyc gets
>>>>>> properly recorded, so the package *has to overwrite* files that
>>>>>> are not owned by anyone (no package owns them).
>>>>>
>>>>> The usual approach to this problem was to have some cleanup code
>>>>> in pkg_setup or pkg_preinst that would remove the old files.
>>>>
>>>> Interesting idea - I see two problems:
>>>> - how I would get the list of files for already installed version
>>>> (the file list can (and will) differ between versions)
>>
>>> Simple workaround in pkg_preinst (just example code, nothing to
>>> copy/paste):
>>
>>> if has_version < first_good_version ; then
>>> for i in $(find ${D} -name *.{pyc,pyo}) ; do
>>> [[ -e ${i/${D}/${ROOT}/} ]] && rm ${i/${D}/${ROOT}/}
>>> done
>>> fi
>>
>> This wouldn't work, because collision protection runs _before_
>> pkg_preinst.
>>
>> However, one could imagine a (somewhat dirty) workaround:
>> - At the end of src_install, rename all *.pyc and *.pyo in ${D} to
>> names that don't collide with the ones in ${ROOT}, so that collision
>> protection won't be triggered.
>> - At the beginning of pkg_preinst, rename them back to their original
>> names.
>> - Run above code.
>>
>>> Alternatively, you could take all .py files of the installed
>>> versions and blindly remove the pyo/pyc files for them in
>>> pkg_preinst (this should also prevent leaving dead files around).
>>
>> s/preinst/setup/ otherwise the same argument as above would apply.
>
> The checks above can misbehave.
>
> Pretty sure I've already suggested this route for this specific issue,
> but QA vars in the ebuild that portage can use for hints is a sane,
> non-cluster-fucky way to deal with this. Something along the lines of
> QA_COLLISION_PROTECT_IGNORE='*.pyc'
>
> Still has the potential for things going boom, but warnings can be
> setup for when collision-protect is on, and the portage version
> doesn't look new enough (those being details; point is the general
> approach).

I've sent an e-mail to gentoo-portage-dev@g.o with a patch that adds a new
variable that causes collision-protect/protect-owned to ignore specific file
patterns for unowned files. Behaviour is similar to already existing
COLLISION_IGNORE.

--
Krzysztof Pawlik <nelchael at gentoo.org> key id: 0xF6A80E46
desktop-misc, java, vim, kernel, python, apache...