Linux Archive

Linux Archive (http://www.linux-archive.org/)
-   Fedora Development (http://www.linux-archive.org/fedora-development/)
-   -   Python Packages + Multiple Sources (http://www.linux-archive.org/fedora-development/462823-python-packages-multiple-sources.html)

BJ Dierkes 12-08-2010 01:39 AM

Python Packages + Multiple Sources
 
Hello all,

I've been trying to figure the best way to handle this situation for the better part of two days... and so I figured I'd bring it on list and hopefully I can clarify this all. I have several projects that relate to this question, however I will reference 'cement' as an example. Basically, cement is a CLI Application Framework for Python [1]. I am also the upstream maintainer, so I have the ability to modify upstream or downstream. Currently, cement consists of three separate sources:

cement
cement.devtools
cement.test

All three pieces follow each release meaning, when 0.8.12 (current stable) was released... new tarbals were released for all three. The reason for separate tarbals is primarily for maintaining releases via PyPi [2]. I need all three pieces to be separate so that users can 'easy_install cement', without pulling in a dozen dependencies for cement.devtools or cement.test. I don't have the luxury of creating 'subpackages' in PyPi, so I have to break up the sources.

That said, in the Fedora world... it is a bit excessive to maintain three separate RPM packages (for all Fedora/EPEL releases)... when all three packages are related and could easily role up under a single package, with subpackages. I have the same issue with 'rosendale' [3] which is/will be a set of plugins for applications built on cement. The same situation exists, in PyPi I need separate source tarbals because users need to be able to 'easy_install rosendale.some_plugin'. In Fedora world, maintaining a single package set for 'rosendale' would be ideal and make more sense because I can do sub packages for each plugin and not have to maintain 20 different package sets.

What I would like to see is if this type of situation would lend itself to making an exception to the FPG regarding 'one source per package'. I assume the section 'Bundling of multiple projects' [4] is relevant, though it is pretty vague. I guess what I'm looking for is for someone with more time in the community to give some advice on this situation. Ideally, I would like to be able to maintain a single package set for say 'cement', but with Source0 (cement), Source1 (cement.devtools), Source2 (cement.test).

Or would it be recommended to have a separate tarbal like 'cement-all-0.8.12.tar.gz' which would include all parts of the project, and use that as Source0?

Thanks for your time.

References:

[1] http://builtoncement.org
[2] http://pypi.python.org
[3]http://builtoncement.org/rosendale/1.0/doc/
[4] http://fedoraproject.org/wiki/PackagingGuidelines#Bundling_of_multiple_projects

---
derks



--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Jeff Spaleta 12-08-2010 02:25 AM

Python Packages + Multiple Sources
 
On Wed, Dec 8, 2010 at 2:39 AM, BJ Dierkes <wdierkes@5dollarwhitebox.org> wrote:
> Hello all,
>

Just to be clear... PyPI has an implied "one source" requirement
embedded in its repository structure and you have optimized your
upstream project release structure to meet PyPI's implied requirement.

Question does PyPI handle dependency tracking?

If I easy_install cement.devtools does cement get installed automagically?


-jef
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Stanislav Ochotnicky 12-08-2010 07:55 AM

Python Packages + Multiple Sources
 
On 12/08/2010 04:25 AM, Jeff Spaleta wrote:
> On Wed, Dec 8, 2010 at 2:39 AM, BJ Dierkes <wdierkes@5dollarwhitebox.org> wrote:
>> Hello all,
>>
>
> Just to be clear... PyPI has an implied "one source" requirement
> embedded in its repository structure and you have optimized your
> upstream project release structure to meet PyPI's implied requirement.
>
> Question does PyPI handle dependency tracking?
>
> If I easy_install cement.devtools does cement get installed automagically?

Yes, setup function from python setuptools has named argument
'install_requires' that causes dependencies to be pulled in during
install time. Assuming cement.devtools has install_requires = ['cement']
this will work as you'd expect

--
Stanislav Ochotnicky <sochotnicky@redhat.com>
Associate Software Engineer - Base Operating Systems Brno

PGP: 71A1677C
Red Hat Inc. http://cz.redhat.com

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

BJ Dierkes 12-08-2010 05:06 PM

Python Packages + Multiple Sources
 
On Dec 8, 2010, at 2:55 AM, Stanislav Ochotnicky wrote:

> On 12/08/2010 04:25 AM, Jeff Spaleta wrote:
>> On Wed, Dec 8, 2010 at 2:39 AM, BJ Dierkes <wdierkes@5dollarwhitebox.org> wrote:
>>> Hello all,
>>>
>>
>> Just to be clear... PyPI has an implied "one source" requirement
>> embedded in its repository structure and you have optimized your
>> upstream project release structure to meet PyPI's implied requirement.
>>
>> Question does PyPI handle dependency tracking?
>>
>> If I easy_install cement.devtools does cement get installed automagically?
>
> Yes, setup function from python setuptools has named argument
> 'install_requires' that causes dependencies to be pulled in during
> install time. Assuming cement.devtools has install_requires = ['cement']
> this will work as you'd expect
>

That is exactly right.

---
derks


--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Jeff Spaleta 12-08-2010 06:53 PM

Python Packages + Multiple Sources
 
On Wed, Dec 8, 2010 at 6:06 PM, BJ Dierkes <wdierkes@5dollarwhitebox.org> wrote:
> That is exactly right.

reading over the instructions on the pypi page for cement.devtools
explicitly tells people to easy_install cement prior to
easy_install'ing cement.devtools, so I wanted clarification as to
whether that was necessasry.


-jef
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Till Maas 12-08-2010 07:29 PM

Python Packages + Multiple Sources
 
On Tue, Dec 07, 2010 at 08:39:50PM -0600, BJ Dierkes wrote:

> All three pieces follow each release meaning, when 0.8.12 (current stable) was released... new tarbals were released for all three. The reason for separate tarbals is primarily for maintaining releases via PyPi [2]. I need all three pieces to be separate so that users can 'easy_install cement', without pulling in a dozen dependencies for cement.devtools or cement.test. I don't have the luxury of creating 'subpackages' in PyPi, so I have to break up the sources.

> What I would like to see is if this type of situation would lend itself to making an exception to the FPG regarding 'one source per package'. I assume the section 'Bundling of multiple projects' [4] is relevant, though it is pretty vague. I guess what I'm looking for is for someone with more time in the community to give some advice on this situation. Ideally, I would like to be able to maintain a single package set for say 'cement', but with Source0 (cement), Source1 (cement.devtools), Source2 (cement.test).

To quote the guidelines:
| Fedora packages should make every effort to avoid having multiple,
| separate, upstream projects bundled together in a single package.

Since all tarballs belong to the same upstream project, there is nothing
wrong with using all three in one SPEC.

> Or would it be recommended to have a separate tarbal like 'cement-all-0.8.12.tar.gz' which would include all parts of the project, and use that as Source0?

I do not see any additional value for Fedora when doing this.

Regards
Till

> [4] http://fedoraproject.org/wiki/PackagingGuidelines#Bundling_of_multiple_projects
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

BJ Dierkes 12-08-2010 09:36 PM

Python Packages + Multiple Sources
 
On Dec 8, 2010, at 1:53 PM, Jeff Spaleta wrote:

> On Wed, Dec 8, 2010 at 6:06 PM, BJ Dierkes <wdierkes@5dollarwhitebox.org> wrote:
>> That is exactly right.
>
> reading over the instructions on the pypi page for cement.devtools
> explicitly tells people to easy_install cement prior to
> easy_install'ing cement.devtools, so I wanted clarification as to
> whether that was necessasry.
>

I've updated this to be more accurate, thank you.

---
derks



--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Toshio Kuratomi 12-09-2010 06:13 PM

Python Packages + Multiple Sources
 
On Wed, Dec 08, 2010 at 09:29:10PM +0100, Till Maas wrote:
> On Tue, Dec 07, 2010 at 08:39:50PM -0600, BJ Dierkes wrote:
>
> > All three pieces follow each release meaning, when 0.8.12 (current stable) was released... new tarbals were released for all three. The reason for separate tarbals is primarily for maintaining releases via PyPi [2]. I need all three pieces to be separate so that users can 'easy_install cement', without pulling in a dozen dependencies for cement.devtools or cement.test. I don't have the luxury of creating 'subpackages' in PyPi, so I have to break up the sources.
>
> > What I would like to see is if this type of situation would lend itself to making an exception to the FPG regarding 'one source per package'. I assume the section 'Bundling of multiple projects' [4] is relevant, though it is pretty vague. I guess what I'm looking for is for someone with more time in the community to give some advice on this situation. Ideally, I would like to be able to maintain a single package set for say 'cement', but with Source0 (cement), Source1 (cement.devtools), Source2 (cement.test).
>
> To quote the guidelines:
> | Fedora packages should make every effort to avoid having multiple,
> | separate, upstream projects bundled together in a single package.
>
> Since all tarballs belong to the same upstream project, there is nothing
> wrong with using all three in one SPEC.
>
> > Or would it be recommended to have a separate tarbal like 'cement-all-0.8.12.tar.gz' which would include all parts of the project, and use that as Source0?
>
> I do not see any additional value for Fedora when doing this.

Separate packages do have benefits for users. Here's an example:

Let's say that I'm doing new development on cement-0.9 atm but someone
reports a security issue that needs attention on the current stable release
of cement.devtools-0.8.12. If I was upstream, I'd be very tempted to only
release a new cement.devtools-0.8.12.1 and not bother with a cement-0.8.12.1
version (with no changes other than the version bump) since 1) I want to get
the security update out ASAP and 2) Why inconvenience my users with
downloading cement again when only users of cement.devtools need to do
something.

For packagers, whether the upstream decided to release cement-0.8.12.1 or
not, you have the same choice as long as you use separate srpms. You can
say... hmmm... upstream's changelog for cement is "* Keep version number in
sync with cement.devtools release" I think I'll skip this update to cement
so that users don't have to download the new package for no benefit. if you
use on srpm, a change to any of the tarballs is going to mean you need to
update for all of them.

So if you're going to draft a change to the Fedora Guideline, I'd phrase it
as "If you are the upstream for the package and you are always going to
release updates of the tarballs simultaneously (even if there's only changes
to some of the tarballs) you may include multiple tarballs in the same
package." I don't know that I'm convinced that I'd vote for that but
I think that's the approach that would make the most sense.

-Toshio
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel


All times are GMT. The time now is 08:46 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.