FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Fedora Packaging

 
 
LinkBack Thread Tools
 
Old 02-16-2012, 03:14 AM
Toshio Kuratomi
 
Default Packaging guidelines - mandatory rebuilding gems

Sending to packaging@lists.fp.o where this belongs.

On Tue, Feb 14, 2012 at 05:27:06PM -0500, Mo Morsi wrote:
>
> Thoughts:
>
> * - I am also not a fan of the current approach of always repackaging the gem,
> seems like extra steps / added complexity which is unneeded, what is the
> justification for it? In the majority of cases, gem install works out of the
> box and the gem doesn't need any modifications.
>
There's a lot of justification in the ticket. There's various pieces.

* The historical ruby/gem guidelines are a huge hack that FPC approved because
it seemed that gems wouldn't work with the normal separation of %prep
%build %install. The recent draft showed that this is no longer the case.
* Proper separation allows rpmbuild -b[pci] --shortcircuit to work properly.
* The previous guidelines actually didn't allow patching. The original
proposed draft guidelines ended up being overly complex as they had
separate cases for gem, gem w/ C extension, patched gem, patched gem w/
C extension, patched gem w/ C extension that fails to build. This
strategy has a single way to handle all of this. The cost is three extra
lines which are easy to conceptualize and explain. (unpacking the code
from an archive, then assembling the code into a form that can be built
from).
* Proper separation of the steps allows people new to ruby packaging but old
hands at rpm packaging (a large portion of reviewers) to come to
understand the basics of packaging rubygems.

> * - what is the rationality behind having a gem package provide ruby
> (RUBYLIBRARY)? Seems like the non-gem and gem package distinction is more
> explicit when the gems provide rubygem(gemname) and non-gems provide ruby
> (libraryname). Since the dependency needs to be represented anyways in any
> other packages that depend on the gem, an explicit dependency on rubygem
> (gemname) might not be a bad idea.
>
In the old guidelines, rubygems were distinct from plain ruby packages.
This was because at the time they were written:
require 'rubylibrary'

imported a plain ruby library and:
gem_require 'rubylibrary'

The had wholly different syntax at the source code level. The piece of
vondruch's draft that stated that non-gem subpackages were no longer needed
for gem packages lead me to find that the rubygems package now sets up the
ruby library path so that "require 'rubylibrary'" works for both gems and
nongems. With that change, we don't need to package all ruby libraries in
a non-gem version and the need to have separate requires and provides
lessens. (The caveat here is that ruby code would need to have require
'rubygems' somewhere early in its code but if that's considered sufficiently
compatible for one of these cases, then it's sufficient for both cases)

When might you absolutely need to use Require: rubygem()? I know of the
specific case when you are requiring a specific version of code via::
gem 'rubylibrary'

I also imagine that there's ways that ruby code can make use of the metadata
that gems provide but I don't know how to detect those. In either of those
cases, the rubygem() require would seem to be most appropriate.

When might you want to use the Require ruby() version? When a package's
code uses plain "require 'rubylibrary'", it would work whether the rpm
package is shipping a rubygem or a plain ruby library. Therefore, if the
packager dosn't want to guess whether we're shipping the relevant library as
a rubygem or as a non-gem ruby library, they should be able to do a simple
"Requires: ruby(library)". If the packager knows that we package that
pacticular library as a rubygem, they can use the rubygem() version of the
requires instead but they are no longer required to in this case.

So I think answering your question, whether the dependency is satisfied by
a gem or non-gem library is no longer of interest in the simple case of
a script or piece of code using "require 'rubylibrary'". For places where
it does matter, the Guideline should require using Requires: rubygem() (if
you have a list of other things to look for, that could be added to flesh
out that section).

> * - Binary Extensions* - gem install then recompile doesn't make alot of sense
> when the other alternative, gem unpack, patch, rebuild, and them install would
> work in both cases. Why is this split out like this?
>
I'm not sure I understand this. Are you referring to the draft guideline
that vondruch originally proposed with the many separate pages or the
changes that I have been making to have only a single case? The single case
does the gem unpack, patch, rebuild (via gem install), and finally install
via mkdir/cp/mv.

I've forked this to this page:
https://fedoraproject.org/wiki/User:Toshio/RubyPackagingDraft

for now so that people can better see what that proposal looks like. I'll
have to keep the two pages in sync for a bit until someone does or does not
bring up a rubygem where the reduced-complexity guidelines are as direly
broken as people are claiming.

> * - the rspec package will soon finally be updated to rspec2 so the BR:
> rspec-core in the test guidelines can be changes to a BR: rspec

Nice. Can you specify what versions of Fedora that will be operable on?
It's nice to note things like that in the Guidelines.

>
>
> Answers to specific questions expressed in the draft:
>
> - Interpreter independence - seems reasonable, how we address supporting these
> libraries on multiple interpreters needs to be flushed out, abliet not
> necessarily for this release as time is short
>
> - Move text about interpreter independence to here - seems reasonable to me
>
So -- vondruch seems to have issues with this. Code doesn't seem to be
entirely interpreter independent? Or rather, only certain code is? The FPC
really needs to understand this portion of the issue better as it affects
things like where the directories belong in the FHS, whether it makes sense
to share rubygems across interpreters and whether it makes sense to share
vendorlib or vendorarch across interpreters. Example srpms are the best way
to make clear what's happening here :-).

Note that when you talk about this being possibly something for later
releases -- unfortunately, that seems to be an underlying assumption of the
draft guidelines that we were presented with so we need to be able to get
correct the parts that make sense.

> - Confirm change (remove ruby(rubygems) dep) - also seems reasonable to me
>
I've finalized this.

> - Give examples? and Do we want to tell what the arguments to gem install do? -
> both would be appreciated
>
heh :-) Unfortunately, I don't believe we've had a ruby person on the FPC
since lutter (who came up with the original guidelines). So if examples and
explanations of arguments are desired, I'll need to recieve them from people
who use ruby.

> - Replacement instructions - as discussed not a huge fan of this
>
Unless someone can present examples where this doesn't work that aren't
flaws with upstream code, this is likely to become the way to do things.

> - Library placement - the bit about 'foo' is somewhat confusing, perhaps
> replacing it with something like "[ext|lib]" or similar would work instead?
>
Yeah -- this was in the portion of the guidelines to be replaced. In my
draft, I've simply removed that section (as it's already in the previous
section where I renamed it $REQUIRE_PATHS with a comment about where
$REQUIRE_PATHS comes from)

-Toshio
--
packaging mailing list
packaging@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/packaging
 
Old 02-17-2012, 03:13 PM
Mo Morsi
 
Default Packaging guidelines - mandatory rebuilding gems

On 02/15/2012 11:14 PM, Toshio Kuratomi wrote:

Sending to packaging@lists.fp.o where this belongs.

On Tue, Feb 14, 2012 at 05:27:06PM -0500, Mo Morsi wrote:

Thoughts:

- I am also not a fan of the current approach of always repackaging the gem,
seems like extra steps / added complexity which is unneeded, what is the
justification for it? In the majority of cases, gem install works out of the
box and the gem doesn't need any modifications.


There's a lot of justification in the ticket. There's various pieces.

* The historical ruby/gem guidelines are a huge hack that FPC approved because
it seemed that gems wouldn't work with the normal separation of %prep
%build %install. The recent draft showed that this is no longer the case.
* Proper separation allows rpmbuild -b[pci] --shortcircuit to work properly.
* The previous guidelines actually didn't allow patching. The original
proposed draft guidelines ended up being overly complex as they had
separate cases for gem, gem w/ C extension, patched gem, patched gem w/
C extension, patched gem w/ C extension that fails to build. This
strategy has a single way to handle all of this. The cost is three extra
lines which are easy to conceptualize and explain. (unpacking the code
from an archive, then assembling the code into a form that can be built
from).
* Proper separation of the steps allows people new to ruby packaging but old
hands at rpm packaging (a large portion of reviewers) to come to
understand the basics of packaging rubygems.


OK this makes a lot more sense to me now. The counter-point I would
raise is that the majority of gems that I've seen do not require
patching. While I agree that this approach brings things more inline w/
the Fedora way of doing things, it does come at the expense of adding
additional steps which will not be necessary most of the time.


Perhaps a compromise would be to make the gem unpacking steps optional,
and able to be skipped if not necessary?


That being said, I just wanted to verify that
"../%{gem_name}-%{version}/" is the correct location of the gem when the
user builds it in the rpm spec.


Also it might be desirable to allow the user to specify the gemspec via
SOURCE1 (or a subsequent source) so as to be able to pull the original /
developer-maintainer gemspec from upstream. I've never used the 'gem
spec' command and can't attest to its usage.


Furthermore, this [1] may be of interest. It is a rake task which we
wrote and regularly maintain as part of the Aeolus project [2] for
building rpms from Rake (the ruby build system). It may be of use to the
Fedora community to build rpms from ruby sources (we use it to build our
rpms, we also have a yum repo Rake module [3] as well)




- what is the rationality behind having a gem package provide ruby
(RUBYLIBRARY)? Seems like the non-gem and gem package distinction is more
explicit when the gems provide rubygem(gemname) and non-gems provide ruby
(libraryname). Since the dependency needs to be represented anyways in any
other packages that depend on the gem, an explicit dependency on rubygem
(gemname) might not be a bad idea.


In the old guidelines, rubygems were distinct from plain ruby packages.
This was because at the time they were written:
require 'rubylibrary'

imported a plain ruby library and:
gem_require 'rubylibrary'

The had wholly different syntax at the source code level. The piece of
vondruch's draft that stated that non-gem subpackages were no longer needed
for gem packages lead me to find that the rubygems package now sets up the
ruby library path so that "require 'rubylibrary'" works for both gems and
nongems. With that change, we don't need to package all ruby libraries in
a non-gem version and the need to have separate requires and provides
lessens. (The caveat here is that ruby code would need to have require
'rubygems' somewhere early in its code but if that's considered sufficiently
compatible for one of these cases, then it's sufficient for both cases)

When might you absolutely need to use Require: rubygem()? I know of the
specific case when you are requiring a specific version of code via::
gem 'rubylibrary'

I also imagine that there's ways that ruby code can make use of the metadata
that gems provide but I don't know how to detect those. In either of those
cases, the rubygem() require would seem to be most appropriate.

When might you want to use the Require ruby() version? When a package's
code uses plain "require 'rubylibrary'", it would work whether the rpm
package is shipping a rubygem or a plain ruby library. Therefore, if the
packager dosn't want to guess whether we're shipping the relevant library as
a rubygem or as a non-gem ruby library, they should be able to do a simple
"Requires: ruby(library)". If the packager knows that we package that
pacticular library as a rubygem, they can use the rubygem() version of the
requires instead but they are no longer required to in this case.

So I think answering your question, whether the dependency is satisfied by
a gem or non-gem library is no longer of interest in the simple case of
a script or piece of code using "require 'rubylibrary'". For places where
it does matter, the Guideline should require using Requires: rubygem() (if
you have a list of other things to look for, that could be added to flesh
out that section).


OK again this makes more sense, thanks again for the explanation. If I
understand correctly (and perhaps this should be somewhat elaborated in
the guidelines), if a ruby package ships a top level module which can
simply be 'required' as is that package is to 'Provide:
ruby(modulename)'. If a ruby package ships w/ a module that is meant to
be pulled in via rubygems it is to 'Provider: rubygem(modulename)'


As far as packages that depend on it, they are to pull in the dependency
via whichever means (Requires ruby() or rubygem()) the ruby package
which they are shipping pulls it in.


Of course this brings up the issue of naming conflicts, for example if
two independent upstream projects separately provide the ruby and
rubygem packages (for example the ruby-sqlite3 and rubygem-sqlite3
packages correspond to separate projects), but I imaging this isn't a
problem just for ruby.







- Binary Extensions - gem install then recompile doesn't make alot of sense
when the other alternative, gem unpack, patch, rebuild, and them install would
work in both cases. Why is this split out like this?


I'm not sure I understand this. Are you referring to the draft guideline
that vondruch originally proposed with the many separate pages or the
changes that I have been making to have only a single case? The single case
does the gem unpack, patch, rebuild (via gem install), and finally install
via mkdir/cp/mv.

I've forked this to this page:
https://fedoraproject.org/wiki/User:Toshio/RubyPackagingDraft

for now so that people can better see what that proposal looks like. I'll
have to keep the two pages in sync for a bit until someone does or does not
bring up a rubygem where the reduced-complexity guidelines are as direly
broken as people are claiming.


Ah ok yes, was mistaken (thought you added that bit). Can ignore this.





- the rspec package will soon finally be updated to rspec2 so the BR:
rspec-core in the test guidelines can be changes to a BR: rspec

Nice. Can you specify what versions of Fedora that will be operable on?
It's nice to note things like that in the Guidelines.


I have the patch to update rspec to rspec2 ready to go, so unless there
are objections from ruby-sig (I've heard none up to this point) will
push this in for F17.


I just went through and verified the last package that needed to be
updated, aeolus-conductor, works as intended against rspec2 so we should
be good to go (based on Vit's rspec2 emails, shout out if I missed anything)






Answers to specific questions expressed in the draft:

- Interpreter independence - seems reasonable, how we address supporting these
libraries on multiple interpreters needs to be flushed out, abliet not
necessarily for this release as time is short

- Move text about interpreter independence to here - seems reasonable to me


So -- vondruch seems to have issues with this. Code doesn't seem to be
entirely interpreter independent? Or rather, only certain code is? The FPC
really needs to understand this portion of the issue better as it affects
things like where the directories belong in the FHS, whether it makes sense
to share rubygems across interpreters and whether it makes sense to share
vendorlib or vendorarch across interpreters. Example srpms are the best way
to make clear what's happening here :-).


Noarch / pure-ruby code should work across interpreters. eg MRI (c-ruby)
interpreter will parse the same pure-ruby-1.9.3 code as JRuby does.


It gets problematic when the Ruby C API is incorporated to build native
platform extensions for Ruby. AFAIK (admittedly I have yet to do this
myself) bindings need to be written for each interpreter the code needs
to run on, so we will need to compile the package again MRI, JRuby,
other interpreters specifically (if the upstream project supports those
interpreters).


I'm not sure if there is a standard way to build Ruby extensions against
multiple interpreters and/or decouple the interpreter implementation
from the extension build process, perhaps something worth looking into.





- Give examples? and Do we want to tell what the arguments to gem install do? -
both would be appreciated


heh :-) Unfortunately, I don't believe we've had a ruby person on the FPC
since lutter (who came up with the original guidelines). So if examples and
explanations of arguments are desired, I'll need to recieve them from people
who use ruby.


/me just joined the FPC mailing list :-)

Will see about drafting some examples if I can get some cycles free at
some point.



-Mo


[1]
https://github.com/aeolusproject/aeolus-configure/blob/master/rake/rpmtask.rb

[2] http://aeolusproject.org
[3]
https://github.com/aeolusproject/aeolus-configure/blob/master/rake/yumtask.rb


--
packaging mailing list
packaging@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/packaging
 

Thread Tools




All times are GMT. The time now is 07:51 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org