FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo Development

 
 
LinkBack Thread Tools
 
Old 12-03-2010, 11:23 PM
Sebastian Pipping
 
Default Sane defaults for USE_PYTHON, patch to python eclass

Hello!


Current situation
=================
Without specifying USE_PYTHON in /etc/make.conf ebuilds based on the
python eclass will install packages for no more ABIs than the two active
versions on the 2.x and 3.x lines. To give an example: with Python 2.6,
2.7 and 3.1 installed and 2.7 set as the active 2.x Python version I
would get files installed for python 2.7 and 3.1, but not 2.6.

Is that a sane default? Especially when a new slot of Python arrives at
the Gentoo tree, you run into situations with two slots of Python 2.x
installed. To have packages functioning with both, you would need a
custom USE_PYTHON line like USE_PYTHON="2.6 2.7" - otherwise one of
these slots' Python will be very limited.

This problem is made worse by the fact that USE_PYTHON has almost no
documentation. This bug shows well, that the current behavior is a
surprising troublemaker:

https://bugs.gentoo.org/show_bug.cgi?id=347153


Proposed new situation
======================
If I have a version of Python installed, it should be usable well.
So USE_PYTHON is derived from the list of all available Python slots.
Excluded are ABIs restricted by an ebuild, say by a line like

RESTRICT_PYTHON_ABIS="3.*"

for software that does not build with Python 3.x.


Proposed code
=============
I would love to just pass a patch here - I have two files to diff
between - but (due to the algorithm diff works with) the patch is much
less clear as the relevant excerpt itself.

If you want a patch or the file version get my modified python.eclass
file from here:
http://hartwork.org/public/python.eclass

So now comes the excerpt (with tabs converted to double spaces just for
this mail). The code is meant to fill variable PYTHON_ABIS with content
like "2.6 2.7" in case of no USE_PYTHON line around.

================================================== =====================
debug-print 'USE_PYTHON not specified in make.conf, deriving default'
# - include all installed ABIs
# - but exclude those restricted by the ebuild

local PYTHON_ABI restricted_ABI restricted_ABIs support_ABI
python_versions

restricted_ABIs="${RESTRICT_PYTHON_ABIS// /$'
'}"
python_versions=("${_CPYTHON2_SUPPORTED_ABIS[@]}"
"${_CPYTHON3_SUPPORTED_ABIS[@]}")

for PYTHON_ABI in ${python_versions[@]} ; do
# ABI available?
local interpreter="${EPREFIX}"/usr/bin/python${PYTHON_ABI}
if [[ ! -x "${interpreter}" ]]; then
debug-print "Disabling ABI ${PYTHON_ABI} (interpreter
${interpreter} missing)"
continue
fi

# ABI restricted by ebuild?
support_ABI="1"
while read restricted_ABI; do
if [[ "${PYTHON_ABI}" == ${restricted_ABI} ]]; then
support_ABI="0"
break
fi
done <<< "${restricted_ABIs}"
if [[ "${support_ABI}" != "1" ]]; then
debug-print "Disabling ABI ${PYTHON_ABI} (due to restrictions from
the ebuild)"
continue
fi

debug-print "Enabling ABI ${PYTHON_ABI}"
export PYTHON_ABIS+="${PYTHON_ABIS:+ }${PYTHON_ABI}"
done

export PYTHON_ABIS="${PYTHON_ABIS% }"
debug-print "PYTHON_ABIS now is '${PYTHON_ABIS}'"
================================================== =====================

Please review the proposal and above code. Thanks in advance.
ECLASS_DEBUG_OUTPUT=on may be of use when testing.

Best,



Sebastian
 
Old 12-04-2010, 02:38 PM
Thomas Sachau
 
Default Sane defaults for USE_PYTHON, patch to python eclass

Am 04.12.2010 01:23, schrieb Sebastian Pipping:
> Hello!
>
>
> Current situation
> =================
> Without specifying USE_PYTHON in /etc/make.conf ebuilds based on the
> python eclass will install packages for no more ABIs than the two active
> versions on the 2.x and 3.x lines. To give an example: with Python 2.6,
> 2.7 and 3.1 installed and 2.7 set as the active 2.x Python version I
> would get files installed for python 2.7 and 3.1, but not 2.6.
>
> Is that a sane default? Especially when a new slot of Python arrives at
> the Gentoo tree, you run into situations with two slots of Python 2.x
> installed. To have packages functioning with both, you would need a
> custom USE_PYTHON line like USE_PYTHON="2.6 2.7" - otherwise one of
> these slots' Python will be very limited.
>
> This problem is made worse by the fact that USE_PYTHON has almost no
> documentation. This bug shows well, that the current behavior is a
> surprising troublemaker:
>
> https://bugs.gentoo.org/show_bug.cgi?id=347153

I think, the complete python code, behaviour and eclass is way too complicated to easily manage or
even understand it. E.g. why do we need 2 active versions by default? If i set e.g. python-2.6 as
default, i want everything to be installed for that and using it by default, unless i specify
something different. Nothing for python-2.7, nothing for python-3.1 or any other python version.

And USE_PYTHON is the wrong way to handle this. It is invisible, it only controls global settings
and together with other parts of the python eclass, it hides the dependency tree, so with
USE_PYTHON="2.6 2.7", all packages depend on both python versions, but the package manager does not
know about it.

I propose a package manager based solution just like the one i proposed and prepared for different
arches and crosscompilation. It is visible, easy to see and understand, allows per package control
and does leave the control to the package manager including the informations like the dependency tree.

>
>
> Proposed new situation
> ======================
> If I have a version of Python installed, it should be usable well.
> So USE_PYTHON is derived from the list of all available Python slots.
> Excluded are ABIs restricted by an ebuild, say by a line like
>
> RESTRICT_PYTHON_ABIS="3.*"
>
> for software that does not build with Python 3.x.

I am against this proposal. I already masked python-3* on my system, because no package actually
needs it, but the current behaviour installs completly useless code for those versions. Same would
happen for all other, not active versions of python. One default active version should be enough, if
i want more, i can select and set it.

--
Thomas Sachau

Gentoo Linux Developer
 
Old 12-04-2010, 07:23 PM
Sebastian Pipping
 
Default Sane defaults for USE_PYTHON, patch to python eclass

On 12/04/10 16:38, Thomas Sachau wrote:
> I think, the complete python code, behaviour and eclass is way too complicated to easily manage or
> even understand it. E.g. why do we need 2 active versions by default? If i set e.g. python-2.6 as
> default, i want everything to be installed for that and using it by default, unless i specify
> something different. Nothing for python-2.7, nothing for python-3.1 or any other python version.

Can we take that to another thread? This thread is about a small fix to
a problem of the current eclass design, not about starting from scratch.
We can still do that later if needed.


> And USE_PYTHON is the wrong way to handle this. It is invisible, it only controls global settings
> and together with other parts of the python eclass, it hides the dependency tree, so with
> USE_PYTHON="2.6 2.7", all packages depend on both python versions, but the package manager does not
> know about it.

If that's the case, we may be able to fix it. Can you elabortate how
dependencies are hidden and what part of the eclass is guilty? Maybe
that deserves another thread, too?


> I propose a package manager based solution just like the one i proposed and prepared for different
> arches and crosscompilation. It is visible, easy to see and understand, allows per package control
> and does leave the control to the package manager including the informations like the dependency tree.

I am not objecting better fixes later. It still may be a good idea, to
fix this problem before starting from scratch.


>> Proposed new situation
>> ======================
>> If I have a version of Python installed, it should be usable well.
>> So USE_PYTHON is derived from the list of all available Python slots.
>> Excluded are ABIs restricted by an ebuild, say by a line like
>>
>> RESTRICT_PYTHON_ABIS="3.*"
>>
>> for software that does not build with Python 3.x.
>
> I am against this proposal. I already masked python-3* on my system, because no package actually
> needs it, but the current behaviour installs completly useless code for those versions.

Does it? I assume when you say you masked it you also mean that you
don't have Python 3.x installed?

Packages are installed against an ABI iff:
1) The ABI is know to the python eclass - granted
2) The build is not restricting the ABI
3) The related version of python is actually installed

Let's do this check: Please run this command:

# equery f dev-python/pyinotify | fgrep site-packages

If you don't have any Python 3.x installed you should see no files for it.


> Same would
> happen for all other, not active versions of python.

See above, please re-check if that happens for you.


> One default active version should be enough, if
> i want more, i can select and set it.

Please elaborate in more detail on that so I'm sure I reply to the
comment you actually had in mind.

Best,



Sebastian
 
Old 12-06-2010, 05:05 PM
Thomas Sachau
 
Default Sane defaults for USE_PYTHON, patch to python eclass

Am 04.12.2010 21:23, schrieb Sebastian Pipping:
> On 12/04/10 16:38, Thomas Sachau wrote:
>>> Proposed new situation
>>> ======================
>>> If I have a version of Python installed, it should be usable well.
>>> So USE_PYTHON is derived from the list of all available Python slots.
>>> Excluded are ABIs restricted by an ebuild, say by a line like
>>>
>>> RESTRICT_PYTHON_ABIS="3.*"
>>>
>>> for software that does not build with Python 3.x.
>>
>> I am against this proposal. I already masked python-3* on my system, because no package actually
>> needs it, but the current behaviour installs completly useless code for those versions.
>
> Does it? I assume when you say you masked it you also mean that you
> don't have Python 3.x installed?

Exactly.

>
> Packages are installed against an ABI iff:
> 1) The ABI is know to the python eclass - granted
> 2) The build is not restricting the ABI
> 3) The related version of python is actually installed
>
> Let's do this check: Please run this command:
>
> # equery f dev-python/pyinotify | fgrep site-packages
>
> If you don't have any Python 3.x installed you should see no files for it.
>
>
>> Same would
>> happen for all other, not active versions of python.
>
> See above, please re-check if that happens for you.
>
>
>> One default active version should be enough, if
>> i want more, i can select and set it.
>
> Please elaborate in more detail on that so I'm sure I reply to the
> comment you actually had in mind.

You only need one active python around, in this case, python-2.6.

But you also get newer versions, like python-2.7 and python-3.1. While there is no big issue with
python-2.7 (it is masked, until most packages depending on it are fixed), since when it gets stable,
you can select it, update your depending packages, remove python-2.6 and be done with it. And as
long, as you dont select python-2.7, you wont get anything into the site-packages dir of python-2.7.

But with python-3*, it is different. You dont select it, it cannot even be the main or only python
version around. But by default, all packages working with it additionally install their files for
python-3*. This is some black magic behind the scene and invisible to the user. Imho nothing should
be installed for anything else except the main selected version, unless you request it. So while you
want to enable the installation for all versions currently installed, i request the other way round:
Only install for the current main active version and nothing else by default.

--
Thomas Sachau

Gentoo Linux Developer
 
Old 12-07-2010, 07:30 AM
Arfrever Frehtes Taifersar Arahesis
 
Default Sane defaults for USE_PYTHON, patch to python eclass

2010-12-04 01:23:41 Sebastian Pipping napisaƂ(a):
> Current situation
> =================
> Without specifying USE_PYTHON in /etc/make.conf ebuilds based on the
> python eclass will install packages for no more ABIs than the two active
> versions on the 2.x and 3.x lines. To give an example: with Python 2.6,
> 2.7 and 3.1 installed and 2.7 set as the active 2.x Python version I
> would get files installed for python 2.7 and 3.1, but not 2.6.
>
> Is that a sane default? Especially when a new slot of Python arrives at
> the Gentoo tree, you run into situations with two slots of Python 2.x
> installed. To have packages functioning with both, you would need a
> custom USE_PYTHON line like USE_PYTHON="2.6 2.7" - otherwise one of
> these slots' Python will be very limited.
>
> This problem is made worse by the fact that USE_PYTHON has almost no
> documentation. This bug shows well, that the current behavior is a
> surprising troublemaker:
>
> https://bugs.gentoo.org/show_bug.cgi?id=347153
>
>
> Proposed new situation
> ======================
> If I have a version of Python installed, it should be usable well.
> So USE_PYTHON is derived from the list of all available Python slots.

Please don't change current algorithm.
- Average users need packages built with support for at most 1 version of Python 2 and at most
1 version of Python 3.
- Only people, who want to explicitly test given modules/scripts with multiple Python versions,
would need dependencies built with support for more Python versions.
- You might break something outside of python.eclass.

There is a plan (and patches) to introduce usage of USE flags for selection of requested
Python versions. It will be available when a future EAPI supports dots in names of USE flags.

--
Arfrever Frehtes Taifersar Arahesis
 

Thread Tools




All times are GMT. The time now is 07:54 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright ©2007 - 2008, www.linux-archive.org