FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo Development

 
 
LinkBack Thread Tools
 
Old 02-29-2012, 04:39 PM
Krzysztof Pawlik
 
Default New eclass for Python

On 29/02/12 09:11, Dirkjan Ochtman wrote:
> On Tue, Feb 28, 2012 at 22:13, Krzysztof Pawlik <nelchael@gentoo.org> wrote:
>> If there are no objections then during the weekend (March 3, 4) I will add this
>> to portage (after finishing remaining TODO items, PyPy requires 4G of RAM(!!)).
>
> Can we perhaps just name it python-r2 rather than python-distutils-ng?
> Seems descriptive enough...

Yes and no - it's named python-distutils because main focus was this
combination, it can work for other cases too, it's just that it combines
functionality from both old eclasses. Besides I like the name, but I can be
convinced to name it python-r2.eclass.

--
Krzysztof Pawlik <nelchael at gentoo.org> key id: 0xF6A80E46
desktop-misc, java, vim, kernel, python, apache...
 
Old 02-29-2012, 05:03 PM
Jeroen Roovers
 
Default New eclass for Python

On Wed, 29 Feb 2012 18:38:22 +0100
Krzysztof Pawlik <nelchael@gentoo.org> wrote:

> On 29/02/12 08:49, "Paweł Hajdan, Jr." wrote:
> > This is awesome! Compare that to over 3000 LOC of python.eclass.
>
> Count distutils.eclass too:
>
> $ wc -l python-distutils-ng.eclass python.eclass distutils.eclass
> 364 python-distutils-ng.eclass
> 3168 python.eclass
> 592 distutils.eclass

cvs/gentoo-x86/eclass $ cloc --by-file
--force-lang="Bourne Again Shell",eclass python.eclass
distutils .eclass python-distutils-ng.eclass
3 text files.
3 unique files.
0 files ignored.

http://cloc.sourceforge.net v 1.55 T=0.5 s (6.0 files/s, 8294.0
lines/s)
-----------------------------------------------
File blank comment code
-----------------------------------------------
python.eclass 396 288 2494
distutils.eclass 111 86 395
python-distutils-ng.eclass 51 110 216
-----------------------------------------------
SUM: 558 484 3105
-----------------------------------------------


jer
 
Old 02-29-2012, 05:39 PM
Mike Gilbert
 
Default New eclass for Python

On Wed, Feb 29, 2012 at 12:39 PM, Krzysztof Pawlik <nelchael@gentoo.org> wrote:
> On 29/02/12 09:11, Dirkjan Ochtman wrote:
>> On Tue, Feb 28, 2012 at 22:13, Krzysztof Pawlik <nelchael@gentoo.org> wrote:
>>> If there are no objections then during the weekend (March 3, 4) I will add this
>>> to portage (after finishing remaining TODO items, PyPy requires 4G of RAM(!!)).
>>
>> Can we perhaps just name it python-r2 rather than python-distutils-ng?
>> Seems descriptive enough...
>
> Yes and no - it's named python-distutils because main focus was this
> combination, it can work for other cases too, it's just that it combines
> functionality from both old eclasses. Besides I like the name, but I can be
> convinced to name it python-r2.eclass.
>

The distutils-based packages are a good starting point. The name is a
good fit for that.

Looking forward, I think it might make sense to move some of the
functionality to a "core" python eclass that would just be for utility
functions -- similar to the python/distutils split we have now. This
would be used in ebuilds that are not primarily based around distutils
(setup.py).
 
Old 02-29-2012, 06:51 PM
Alexandre Rostovtsev
 
Default New eclass for Python

On Tue, 2012-02-28 at 22:13 +0100, Krzysztof Pawlik wrote:
> Hello,
>
> After some work during weekend on Python packages I've decided to start a
> rewrite of Python/distutils eclass for installing Python packages. My main goal
> was simplicity and functionality similar to ruby-ng.eclass (thanks Ruby team for
> your great work!). Python team members already contributed comments and
> suggestions and helped me to make the eclass better, thank you!
>
> Highlights:
> - *SIMPLE*next
> - uses PYTHON_TARGETS use-expand (no more python-updater, whoooo!)
> - EAPI4 required, uses REQUIRED_USE
> - <400 lines of code including documentation
> - should work for >95% of packages (my educated guess)
> - did I mention it's *SIMPLE*?
> - easy to maintain & read so it's also easy to use
>
> Important thing: I'm not aiming at having 100% functionality of current
> python.eclass+distutils.eclass in the new one, I think that simplicity is more
> important that supporting every possible, obscure case that's out there.
>
> I'm attaching the eclass itself and two ebuilds using it, code is also available
> in my overlay at http://git.overlays.gentoo.org/gitweb/?p=dev/nelchael.git;a=summary
>
> If there are no objections then during the weekend (March 3, 4) I will add this
> to portage (after finishing remaining TODO items, PyPy requires 4G of RAM(!!)).

The proposed eclass omits three features from python.eclass which are
heavily used in the gnome stack.

First, it does not set EPYTHON, which is a problem for the many packages
whose build systems execute /usr/bin/python and assume that it's a
generic python2 or the same version of python2.x for which the package
is being built.

Second, there doesn't seem to be any support for packages that do not
install in python's site-packages and do not allow multiple python ABIs.
If I have, for example, a package that installs python modules
in /usr/lib/appname or /usr/share/appname, how can I specify that
PYTHON_TARGETS="python2.6" or "python2.7" or "python3.2" is allowed, but
something like PYTHON_TARGETS="python2.7 python3.2" is not?

Third, python-distutils-ng_doscript() installs only one file at a time
(no built-in support for multiple files at a time or for recursing
through a directory tree), installs it only in /usr/bin (a package might
want it in e.g. /usr/libexec or /usr/share/appname/scripts), and always
creates scriptname-IMPLEMENTATION (polluting tab-completion in /usr/bin
for the common case of programs that do not support multiple python
ABIs). As a result, it is far less useful than python.eclass's
python_convert_shebangs().

-Alexandre.
 
Old 02-29-2012, 07:24 PM
Krzysztof Pawlik
 
Default New eclass for Python

On 29/02/12 20:51, Alexandre Rostovtsev wrote:
> On Tue, 2012-02-28 at 22:13 +0100, Krzysztof Pawlik wrote:
>> Hello,
>>
>> After some work during weekend on Python packages I've decided to start a
>> rewrite of Python/distutils eclass for installing Python packages. My main goal
>> was simplicity and functionality similar to ruby-ng.eclass (thanks Ruby team for
>> your great work!). Python team members already contributed comments and
>> suggestions and helped me to make the eclass better, thank you!
>>
>> Highlights:
>> - *SIMPLE*next
>> - uses PYTHON_TARGETS use-expand (no more python-updater, whoooo!)
>> - EAPI4 required, uses REQUIRED_USE
>> - <400 lines of code including documentation
>> - should work for >95% of packages (my educated guess)
>> - did I mention it's *SIMPLE*?
>> - easy to maintain & read so it's also easy to use
>>
>> Important thing: I'm not aiming at having 100% functionality of current
>> python.eclass+distutils.eclass in the new one, I think that simplicity is more
>> important that supporting every possible, obscure case that's out there.
>>
>> I'm attaching the eclass itself and two ebuilds using it, code is also available
>> in my overlay at http://git.overlays.gentoo.org/gitweb/?p=dev/nelchael.git;a=summary
>>
>> If there are no objections then during the weekend (March 3, 4) I will add this
>> to portage (after finishing remaining TODO items, PyPy requires 4G of RAM(!!)).
>
> The proposed eclass omits three features from python.eclass which are
> heavily used in the gnome stack.

Correct me if I'm wrong, but Gnome doesn't use standard distutils?

Regardless of this eclass you can still use python.eclass, it not going anywhere
anytime soon (just like I wrote in one of previous e-mails).

> First, it does not set EPYTHON, which is a problem for the many packages
> whose build systems execute /usr/bin/python and assume that it's a
> generic python2 or the same version of python2.x for which the package
> is being built.

Setting EPYTHON is not a problem -- just another variable.

> Second, there doesn't seem to be any support for packages that do not
> install in python's site-packages and do not allow multiple python ABIs.
> If I have, for example, a package that installs python modules
> in /usr/lib/appname or /usr/share/appname, how can I specify that
> PYTHON_TARGETS="python2.6" or "python2.7" or "python3.2" is allowed, but
> something like PYTHON_TARGETS="python2.7 python3.2" is not?

You're correct, note that I've stressed that this eclass is mainly for
distutils-based packages. I'm not using Gnome, so can you provide some package
examples that I can look at?

<personal opinion>
If package decides to use given language then please, please play by the rules
set by the rest of world (Ruby -> gems, Python -> distutils, Perl -> CPAN, PHP
-> PEAR).

I don't like installing Python code outside of site-packages, the only exception
to that rule is portage (at least for now).
</personal opinion>

> Third, python-distutils-ng_doscript() installs only one file at a time
> (no built-in support for multiple files at a time or for recursing
> through a directory tree),

Yes, that why bash has `for' loop

> installs it only in /usr/bin (a package might
> want it in e.g. /usr/libexec or /usr/share/appname/scripts)

Yes, I might add --destdir (or something like that) later on.

> and always
> creates scriptname-IMPLEMENTATION (polluting tab-completion in /usr/bin
> for the common case of programs that do not support multiple python
> ABIs). As a result, it is far less useful than python.eclass's
> python_convert_shebangs().

Yes, that "pollutes" this name space, but I'm not buying this argument. Do you
know how CVS and .svn "pollute" my tab-completion list? I even set FIGNORE so I
can live with it.

I'd be happy to hear how to solve this - what prefix or suffix to use? One way
would be quite trivial: if only one implementation is enabled do not create
script-${impl}, go with single file, does that sound good?

--
Krzysztof Pawlik <nelchael at gentoo.org> key id: 0xF6A80E46
desktop-misc, java, vim, kernel, python, apache...
 
Old 02-29-2012, 08:08 PM
"Andreas K. Huettel"
 
Default New eclass for Python

Am Mittwoch 29 Februar 2012, 21:24:49 schrieb Krzysztof Pawlik:
> > Second, there doesn't seem to be any support for packages that do not
> > install in python's site-packages and do not allow multiple python ABIs.
> > If I have, for example, a package that installs python modules
> > in /usr/lib/appname or /usr/share/appname, how can I specify that
> > PYTHON_TARGETS="python2.6" or "python2.7" or "python3.2" is allowed, but
> > something like PYTHON_TARGETS="python2.7 python3.2" is not?
>
> You're correct, note that I've stressed that this eclass is mainly for
> distutils-based packages. I'm not using Gnome, so can you provide some
> package examples that I can look at?
>
> <personal opinion>
> If package decides to use given language then please, please play by the
> rules set by the rest of world (Ruby -> gems, Python -> distutils, Perl ->
> CPAN, PHP -> PEAR).
>
> I don't like installing Python code outside of site-packages, the only
> exception to that rule is portage (at least for now).
> </personal opinion>

We will hit the same problem with KDE (actually we already hit it): it has
various types of scripting support, and each installs a KDE library linked to
whatever language interpreter.

(Now, that library- is it a Python/Ruby library or a KDE library? Because it
is at the proper place for KDE stuff

It's not just about calling an external language but also about embedding the
interpreter for in-app scripting... and KDE rather heavily relies on python.

--

Andreas K. Huettel
Gentoo Linux developer
dilfridge@gentoo.org
http://www.akhuettel.de/
 
Old 02-29-2012, 08:19 PM
Krzysztof Pawlik
 
Default New eclass for Python

On 29/02/12 22:08, Andreas K. Huettel wrote:
> Am Mittwoch 29 Februar 2012, 21:24:49 schrieb Krzysztof Pawlik:
>>> Second, there doesn't seem to be any support for packages that do not
>>> install in python's site-packages and do not allow multiple python ABIs.
>>> If I have, for example, a package that installs python modules
>>> in /usr/lib/appname or /usr/share/appname, how can I specify that
>>> PYTHON_TARGETS="python2.6" or "python2.7" or "python3.2" is allowed, but
>>> something like PYTHON_TARGETS="python2.7 python3.2" is not?
>>
>> You're correct, note that I've stressed that this eclass is mainly for
>> distutils-based packages. I'm not using Gnome, so can you provide some
>> package examples that I can look at?
>>
>> <personal opinion>
>> If package decides to use given language then please, please play by the
>> rules set by the rest of world (Ruby -> gems, Python -> distutils, Perl ->
>> CPAN, PHP -> PEAR).
>>
>> I don't like installing Python code outside of site-packages, the only
>> exception to that rule is portage (at least for now).
>> </personal opinion>
>
> We will hit the same problem with KDE (actually we already hit it): it has
> various types of scripting support, and each installs a KDE library linked to
> whatever language interpreter.
>
> (Now, that library- is it a Python/Ruby library or a KDE library? Because it
> is at the proper place for KDE stuff
>
> It's not just about calling an external language but also about embedding the
> interpreter for in-app scripting... and KDE rather heavily relies on python.

I agree with last sentence - it's about embedding Python/Ruby/Foo, *this eclass
is about installing packages for Python*. Compiling against an implementation is
something completely different -- for example try building against Jython.

--
Krzysztof Pawlik <nelchael at gentoo.org> key id: 0xF6A80E46
desktop-misc, java, vim, kernel, python, apache...
 
Old 02-29-2012, 08:57 PM
Alexandre Rostovtsev
 
Default New eclass for Python

On Wed, 2012-02-29 at 21:24 +0100, Krzysztof Pawlik wrote:
> On 29/02/12 20:51, Alexandre Rostovtsev wrote:
> > The proposed eclass omits three features from python.eclass which are
> > heavily used in the gnome stack.
>
> Correct me if I'm wrong, but Gnome doesn't use standard distutils?

Gnome is mostly written in C and therefore uses standard autotools

> > Second, there doesn't seem to be any support for packages that do not
> > install in python's site-packages and do not allow multiple python ABIs.
> > If I have, for example, a package that installs python modules
> > in /usr/lib/appname or /usr/share/appname, how can I specify that
> > PYTHON_TARGETS="python2.6" or "python2.7" or "python3.2" is allowed, but
> > something like PYTHON_TARGETS="python2.7 python3.2" is not?
>
> You're correct, note that I've stressed that this eclass is mainly for
> distutils-based packages. I'm not using Gnome, so can you provide some package
> examples that I can look at?
>
> <personal opinion>
> If package decides to use given language then please, please play by the rules
> set by the rest of world (Ruby -> gems, Python -> distutils, Perl -> CPAN, PHP
> -> PEAR).
>
> I don't like installing Python code outside of site-packages, the only exception
> to that rule is portage (at least for now).
> </personal opinion>

Some non-python packages allow python-based plugins. Obviously these
plugins live in the package's plugin directory (not in python's
site-packages) and use the package's main build system (not distutils),
and multiple python ABIs cannot be supported because that would result
in colliding plugins. Typical examples are app-editors/gedit,
media-gfx/gimp, media-sound/rhythmbox, or media-video/totem.

Some packages install a C library that links to a specific version of
libpython or that defines a particular python version string at compile
time, making it impossible to use the package with multiple python ABIs.
Examples I know are dev-libs/libpeas and dev-python/nautilus-python.

And then there are packages which could support e.g. multiple python2
ABIs in theory, but doing so in practice would require a fair bit of
patching, taking substantial effort with no real benefit for end users.
An example that springs to mind here is gnome-extra/zeitgeist.

> I'd be happy to hear how to solve this - what prefix or suffix to use? One way
> would be quite trivial: if only one implementation is enabled do not create
> script-${impl}, go with single file, does that sound good?

That would be the ideal solution.

-Alexandre.
 
Old 03-01-2012, 05:42 PM
Krzysztof Pawlik
 
Default New eclass for Python

On 29/02/12 22:57, Alexandre Rostovtsev wrote:
> On Wed, 2012-02-29 at 21:24 +0100, Krzysztof Pawlik wrote:
>> On 29/02/12 20:51, Alexandre Rostovtsev wrote:
>>> The proposed eclass omits three features from python.eclass which are
>>> heavily used in the gnome stack.
>>
>> Correct me if I'm wrong, but Gnome doesn't use standard distutils?
>
> Gnome is mostly written in C and therefore uses standard autotools

Ok, thank you for this information.

>>> Second, there doesn't seem to be any support for packages that do not
>>> install in python's site-packages and do not allow multiple python ABIs.
>>> If I have, for example, a package that installs python modules
>>> in /usr/lib/appname or /usr/share/appname, how can I specify that
>>> PYTHON_TARGETS="python2.6" or "python2.7" or "python3.2" is allowed, but
>>> something like PYTHON_TARGETS="python2.7 python3.2" is not?
>>
>> You're correct, note that I've stressed that this eclass is mainly for
>> distutils-based packages. I'm not using Gnome, so can you provide some package
>> examples that I can look at?
>>
>> <personal opinion>
>> If package decides to use given language then please, please play by the rules
>> set by the rest of world (Ruby -> gems, Python -> distutils, Perl -> CPAN, PHP
>> -> PEAR).
>>
>> I don't like installing Python code outside of site-packages, the only exception
>> to that rule is portage (at least for now).
>> </personal opinion>
>
> Some non-python packages allow python-based plugins. Obviously these
> plugins live in the package's plugin directory (not in python's
> site-packages) and use the package's main build system (not distutils),
> and multiple python ABIs cannot be supported because that would result
> in colliding plugins. Typical examples are app-editors/gedit,
> media-gfx/gimp, media-sound/rhythmbox, or media-video/totem.
>
> Some packages install a C library that links to a specific version of
> libpython or that defines a particular python version string at compile
> time, making it impossible to use the package with multiple python ABIs.
> Examples I know are dev-libs/libpeas and dev-python/nautilus-python.
>
> And then there are packages which could support e.g. multiple python2
> ABIs in theory, but doing so in practice would require a fair bit of
> patching, taking substantial effort with no real benefit for end users.
> An example that springs to mind here is gnome-extra/zeitgeist.

I see - so it's the same case as with KDE (like Andreas wrote) - it's actually
not a "python package" but rather an embedded Python. That's very different case
than I'm targeting with this eclass (at least for now).

>> I'd be happy to hear how to solve this - what prefix or suffix to use? One way
>> would be quite trivial: if only one implementation is enabled do not create
>> script-${impl}, go with single file, does that sound good?
>
> That would be the ideal solution.

Ok, I've implemented this solution, now if one implementation is enabled there
will be only one script, no foo-IMPL mangling. I've added also ability to
install the script in other directory than /usr/bin. Hope this solves this case.

Thank you for your comments and suggestions

--
Krzysztof Pawlik <nelchael at gentoo.org> key id: 0xF6A80E46
desktop-misc, java, vim, kernel, python, apache...
 
Old 03-03-2012, 06:59 AM
Arfrever Frehtes Taifersar Arahesis
 
Default New eclass for Python

2012-02-28 22:13:36 Krzysztof Pawlik napisał(a):
> - uses PYTHON_TARGETS use-expand

You cannot painlessly use USE flags for this purpose in gentoo-x86 without support for use.unsatisfiable.

--
Arfrever Frehtes Taifersar Arahesis
 

Thread Tools




All times are GMT. The time now is 09:43 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org