FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Ubuntu > Ubuntu Development

 
 
LinkBack Thread Tools
 
Old 02-20-2009, 03:00 PM
Matthias Klose
 
Default python2.6 related changes

python2.6 is now in the archive and is added as a supported python version,
python2.4 is dropped as a supported python version.

Starting with python2.6, the default installation points to
/usr/local/lib/pythonX.Y/dist-packages, while files for packages included in the
distribution are found in /usr/lib/pythonX.Y/dist-packages. This is to avoid
overwriting files installed by local admins/users. The python packaging helpers
are modified to do the right thing with files in these locations; for new
packages please add for distutils based installations (setup.py --install) the
extra option --install-layout=deb. This option is recognized for python2.4 and
python2.5 as well, and ignored.

The option to create the symlinks from sys.path to the shared area in the
preinst is removed (to shorten the non-availability of a module during the
installation). Instead dh_pycentral can now include the symlinks directly into
the package (use DH_PYCENTRAL=include-links dh_pycentral).

The pkgbinarymangler is modified to reject files located in
/usr/lib/pythonX.Y/site-packages.

Please assign bug reports caused by build failures to me.

Matthias

Details:

Local installation path
-----------------------

When installing Python modules using distutils, the resulting files
end up in the same location wether they are installed by a Debian
package, or by a local user or administrator, unless the installation
path is overwritten on the command line. Compare this with most
software based on autoconf, where an explicit prefix has to be
provided for the packaging, while the default install installs into
/usr/local. For new Python versions packaged in Debian this will
change so that an installation into /usr (not /usr/local) requires an
extra option to distutils install command (--install-layout=deb). To
avoid breaking the packaging of existing code the distutils install
command for 2.4 and 2.5 will just accept this option and ignore it.
For the majority of packages we won't see changes in the packaging,
provided that the python packaging helpers can find the files in the
right location and move it to the expected target path.

A second issue raised by developers was the clash of modules and
extensions installed by a local python installation (with default
prefix /usr/local) with the modules provided by Debian packages
(/usr/local/lib/pythonX.Y/site-packages shared by the patched "system"
python and the locally installed python. To avoid this clash the
directory `site-packages' should be renamed to `dist-packages' in
both locations:

- /usr/lib/pythonX.Y/dist-packages (installation location for code
packaged for Debian)
- /usr/local/lib/pythonX.Y/dist-packages (installation location
for locally installed code using distutils install without
options).

The path /usr/lib/pythonX.Y/site-packages is not found on sys.path
anymore.

About the name: Discussed this with Barry Warsaw and Martin v. Loewis,
and we came to the conclusion that using the same directory name for
both locations would be the most consistent way.

This change should make the request to conditionalize the inclusion of
/usr/local/lib/pythonX.Y/site-packages into sys.path obsolete.

If needed we can provide a symlink /usr/lib/pythonX.Y/site-packages
pointing to dist-packages.

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
 
Old 02-21-2009, 08:47 AM
Marius Gedminas
 
Default python2.6 related changes

On Fri, Feb 20, 2009 at 05:00:14PM +0100, Matthias Klose wrote:
> python2.6 is now in the archive and is added as a supported python version,
> python2.4 is dropped as a supported python version.

But it will still be installable from universe, right?

> Starting with python2.6, the default installation points to
> /usr/local/lib/pythonX.Y/dist-packages, while files for packages
> included in the distribution are found in
> /usr/lib/pythonX.Y/dist-packages. This is to avoid overwriting files
> installed by local admins/users. The python packaging helpers are
> modified to do the right thing with files in these locations; for new
> packages please add for distutils based installations (setup.py
> --install) the extra option --install-layout=deb. This option is
> recognized for python2.4 and python2.5 as well, and ignored.

This sounds sane and welcome.

Does upstream Python 2.6 know about --install-layout=deb, or is this
implemented by a Debian/Ubuntu patch to distutils? I assume the latter,
but would welcome the clarification.

I have a couple of questions about setuptools, which is pretty popular
despite not being mature enough to be included in the standard library.

Will users be able to run

sudo easy_install packagename

and get that package installed into
/usr/local/lib/pythonX.Y/dist-packages? Previously the package would be
installed into /usr/lib/pythonX.Y/site-packages, possibly overwriting
Debian-packaged files.

Will users be able to run

sudo apt-get install python-virtualenv
virtualenv /tmp/sandbox

and be able to use Python packages installed into
/usr/{,local/}lib/pythonX.Y/dist-packages from the interpreter in
/tmp/sandbox/bin/python? I ask because currently virtualenv creates a
symlink forest in /tmp/sandbox/lib/python2.x/site-packages and probably
needs to be explicitly taught about the new dist-packages directories.

And another question, probably not related directly to the site-packages to
dist-packages switch, but it also concerns setuptools and the default
sys.path: Will users ever be able to run easy_install without sudo and
be able to install additional Python packages somewhere in their $HOME?
I can buy an argument that this last thing should not be made to work,
especially if locally-installed packages override the ones in /usr and
cause compatibility issues for system-installed software that just
happens to be written in Python.

Regards,
Marius Gedminas
--
Hi. I'm the "I love you" .signature virus. You have been infected.
Please panic immediately.
--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
 
Old 02-21-2009, 03:09 PM
Matthias Klose
 
Default python2.6 related changes

Marius Gedminas schrieb:
> On Fri, Feb 20, 2009 at 05:00:14PM +0100, Matthias Klose wrote:
>> python2.6 is now in the archive and is added as a supported python version,
>> python2.4 is dropped as a supported python version.
>
> But it will still be installable from universe, right?

yes, to be decided if it will stay in main or universe. If we want to build
extensions for 2.4 from the same source of the extension, then it has to stay in
main. to demote it, we would need to copy all source packages which build with
2.4 and upload those to universe. There is a set of packages for 2.4 needed to
support zope2.x and plone.

>> Starting with python2.6, the default installation points to
>> /usr/local/lib/pythonX.Y/dist-packages, while files for packages
>> included in the distribution are found in
>> /usr/lib/pythonX.Y/dist-packages. This is to avoid overwriting files
>> installed by local admins/users. The python packaging helpers are
>> modified to do the right thing with files in these locations; for new
>> packages please add for distutils based installations (setup.py
>> --install) the extra option --install-layout=deb. This option is
>> recognized for python2.4 and python2.5 as well, and ignored.
>
> This sounds sane and welcome.
>
> Does upstream Python 2.6 know about --install-layout=deb, or is this
> implemented by a Debian/Ubuntu patch to distutils? I assume the latter,
> but would welcome the clarification.

No, currently not upstream.

> I have a couple of questions about setuptools, which is pretty popular
> despite not being mature enough to be included in the standard library.
>
> Will users be able to run
>
> sudo easy_install packagename
>
> and get that package installed into
> /usr/local/lib/pythonX.Y/dist-packages?

Yes.

> Previously the package would be
> installed into /usr/lib/pythonX.Y/site-packages, possibly overwriting
> Debian-packaged files.
>
> Will users be able to run
>
> sudo apt-get install python-virtualenv
> virtualenv /tmp/sandbox
>
> and be able to use Python packages installed into
> /usr/{,local/}lib/pythonX.Y/dist-packages from the interpreter in
> /tmp/sandbox/bin/python? I ask because currently virtualenv creates a
> symlink forest in /tmp/sandbox/lib/python2.x/site-packages and probably
> needs to be explicitly taught about the new dist-packages directories.

I will have a look at virtualenv. afair virtualenv does download it's own copy
of setuptools.

> And another question, probably not related directly to the site-packages to
> dist-packages switch, but it also concerns setuptools and the default
> sys.path: Will users ever be able to run easy_install without sudo and
> be able to install additional Python packages somewhere in their $HOME?

should be possible, if you install using the --prefix option.

Matthias


--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
 
Old 02-22-2009, 01:04 PM
Morten Kjeldgaard
 
Default python2.6 related changes

On 21/02/2009, at 17.09, Matthias Klose wrote:

>> I have a couple of questions about setuptools, which is pretty
>> popular
>> despite not being mature enough to be included in the standard
>> library.
>>
>> Will users be able to run
>>
>> sudo easy_install packagename
>>
>> and get that package installed into
>> /usr/local/lib/pythonX.Y/dist-packages?
>
> Yes.

I probably misunderstood something, because I don't see how just
renaming "site-packages" to "dist-packages" will accomplish anything.

It's not that "dist-packages" is reserved for dpkg installed modules,
while "site-packages" is for local installations?

-- Morten


--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
 
Old 02-22-2009, 01:18 PM
Matthias Klose
 
Default python2.6 related changes

Morten Kjeldgaard schrieb:
> On 21/02/2009, at 17.09, Matthias Klose wrote:
>
>>> I have a couple of questions about setuptools, which is pretty popular
>>> despite not being mature enough to be included in the standard library.
>>>
>>> Will users be able to run
>>>
>>> sudo easy_install packagename
>>>
>>> and get that package installed into
>>> /usr/local/lib/pythonX.Y/dist-packages?
>>
>> Yes.
>
> I probably misunderstood something, because I don't see how just
> renaming "site-packages" to "dist-packages" will accomplish anything.
>
> It's not that "dist-packages" is reserved for dpkg installed modules,
> while "site-packages" is for local installations?

/usr/local/lib/pythonX.Y/site-packages is the standard site dir for a python
installation done in /usr/local (without options). Debian does include
/usr/local/lib/pythonX.Y/site-packages in sys.path as well. Many (upstream)
developers complain that two python installation should not get into it's way.
Therefore having two differently named site dirs in /usr/local (if you do want
to have only one, symlink dist-packages to site-packages in /usr/local).

The default install (sudo easy_install ...) now goes into
/usr/local/lib/pythonX.Y/dist-packages, and doesn't do anything anymore in
/usr/lib, overwriting the distribution files. If you do want to install for a
Debian package, you need to call --install --install-layout=deb or move the
files from <tempdir>//usr/local/lib/pythonX.Y/dist-packages into the final
location yourself.

Matthias

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
 
Old 02-22-2009, 01:45 PM
Vincent Untz
 
Default python2.6 related changes

Le dimanche 22 février 2009, à 15:18 +0100, Matthias Klose a écrit :
> Morten Kjeldgaard schrieb:
> > I probably misunderstood something, because I don't see how just
> > renaming "site-packages" to "dist-packages" will accomplish anything.
> >
> > It's not that "dist-packages" is reserved for dpkg installed modules,
> > while "site-packages" is for local installations?
>
> /usr/local/lib/pythonX.Y/site-packages is the standard site dir for a python
> installation done in /usr/local (without options). Debian does include
> /usr/local/lib/pythonX.Y/site-packages in sys.path as well. Many (upstream)
> developers complain that two python installation should not get into it's way.

If upstream developers complain about it, shouldn't this be fixed
upstream? This would make it work in all distros/OS...

Vincent

--
Les gens heureux ne sont pas pressés.

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
 
Old 02-22-2009, 04:02 PM
Matthias Klose
 
Default python2.6 related changes

Vincent Untz schrieb:
> Le dimanche 22 février 2009, à 15:18 +0100, Matthias Klose a écrit :
>> Morten Kjeldgaard schrieb:
>>> I probably misunderstood something, because I don't see how just
>>> renaming "site-packages" to "dist-packages" will accomplish anything.
>>>
>>> It's not that "dist-packages" is reserved for dpkg installed modules,
>>> while "site-packages" is for local installations?
>> /usr/local/lib/pythonX.Y/site-packages is the standard site dir for a python
>> installation done in /usr/local (without options). Debian does include
>> /usr/local/lib/pythonX.Y/site-packages in sys.path as well. Many (upstream)
>> developers complain that two python installation should not get into it's way.
>
> If upstream developers complain about it, shouldn't this be fixed
> upstream? This would make it work in all distros/OS...

The upstreams did complain about the inclusion of a path in /usr/local to
sys.path in Debian's python. Here expectations/practice of Debian users/devels
clash with expections from others.

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
 
Old 02-25-2009, 12:54 PM
Matthias Klose
 
Default python2.6 related changes

Matthias Klose schrieb:
> python2.6 is now in the archive and is added as a supported python version,
> python2.4 is dropped as a supported python version.

main is now built with python2.6 as an optional python version; there are a few
remaining packages which need to be rebuilt after 2.6 becomes the default. You
can find the remaining packages in

https://edge.launchpad.net/~pythoneers/+archive/ppa

After an update, it should be possible to remove python2.5. Note, that universe
still needs some work, but usually these are easy things:

- .install files in debian/ referencing usr/lib/python*/site-packages. Make
this usr/lib/python*/*-packages.

- debian/rules referencing site-packages. To help selecting the correct name
in shell snippets or wildcard rules, see some helper macros in
/usr/share/python/python.mk.

- distutils based installations (setup.py install) go into /usr/local by
default now. Pass --install-layout=deb, or use the macro in python.mk.
The packaging helper tools should handle the easy cases where one single
binary package is built.

Currently running a desktop installed without python2.5. I would be interested
in feedback, but be prepared to find some edges, which I didn't find yet.

Matthias

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
 
Old 03-04-2009, 06:16 PM
Mario Limonciello
 
Default python2.6 related changes

Hi Matthias:

On Fri, Feb 20, 2009 at 10:00, Matthias Klose <doko@ubuntu.com> wrote:

<snip>*If needed we can provide a symlink /usr/lib/pythonX.Y/site-packages


pointing to dist-packages.



Can you please add this?* For the workaround in python-xml, upstreams had to add in a diff such as this:

27
# This is done for Ubuntu, they are removing PyXML.28
alt_path =*'/usr/lib/python%s/site-packages/oldxml'*%*sys.version[:3]
29if*os.path.exists(alt_path):
30* * * * sys.path.append(alt_path)

So it would be most beneficial to not have to bug the upstreams again to change that.
--
Mario Limonciello
superm1@gmail.com
Sent from: Bronx NY United States.
--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
 
Old 03-04-2009, 06:43 PM
Matthias Klose
 
Default python2.6 related changes

Mario Limonciello schrieb:
> Hi Matthias:
>
> On Fri, Feb 20, 2009 at 10:00, Matthias Klose <doko@ubuntu.com> wrote:
>
>> <snip>
>
>
>
>> If needed we can provide a symlink /usr/lib/pythonX.Y/site-packages
>> pointing to dist-packages.
>>
>> Can you please add this? For the workaround in python-xml, upstreams had
> to add in a diff such as this:

I don't think this is the right approach. python-xml isn't maintained for years.
We did move it outside sys.path to get rid of it. At this point, I really would
suggest that mythvideo is patched to use the xml included in the python core
library (preferred), or include it's own copy of this unmaintained code.

Afaik, pyxml isn't packaged anymore by other distributions as well.

Matthias


--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
 

Thread Tools




All times are GMT. The time now is 01:23 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright ©2007 - 2008, www.linux-archive.org