FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo Development

 
 
LinkBack Thread Tools
 
Old 04-20-2012, 12:39 PM
Kent Fredric
 
Default RFC: Add new remote-id types in metadata.dtd

On 20 April 2012 23:21, Corentin Chary <corentin.chary@gmail.com> wrote:
>
> Currently it uses SRC_URI and HOMEPAGE, but honestly it wouldn't be
> hard to use any other environment variable and to do some checks on a
> webservice.
> Anyway for tricky cases it can still be done by hand.
>
> --
> Corentin Chary
> http://xf.iksaif.net
>


If you really want to support Perl Modules, ( which theres not much
need for at present, looks like the team have gone through already for
the most part and added remote-id's where possible already ), anything
that inherits 'perl-module.eclass' has a bit of magic, in that neither
SRC_URI or HOMEPAGE is required in the ebuild, and it just gets the
package name from what gentoo is using. We've tried to be as close to
upstream as possible for the ease of maintenance.

However, there are still exception cases, for instance, BioPerl has to
define 'MY_PN' to tell the perl-module eclass to use a different token
( and when this is present, it should be sufficient to say that that
should be the remote-id instead of the package name:
see dev-perl/Moose # an example with neither src_uri or homepage
see sci-biology/bioperl # an example where the package name has
been forced overridden as its changed upstream

But resolving module names is much trickier, its easy-ish to map a
module name to a package using the service, but doing it the other way
round is not so straight forward, as one package can have many
modules, and its common in perl to state dependencies in terms of the
module to require, not the package its in, but there's also often a
defacto "main module".

But I'm myself still working out how to best do that with the service
, so auto-populating a "cpan-module" identifier can be left to later,
its just something I considered "useful" to have metadata wise because
that value is more useful to users.

--
Kent

perl -e* "print substr( "edrgmaM* SPA NOcomil.ic@tfrken", $_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

http://kent-fredric.fox.geek.nz
 
Old 04-20-2012, 01:34 PM
Corentin Chary
 
Default RFC: Add new remote-id types in metadata.dtd

On Sat, Apr 21, 2012 at 12:39:26AM +1200, Kent Fredric wrote:
> If you really want to support Perl Modules, ( which theres not much
> need for at present, looks like the team have gone through already for
> the most part and added remote-id's where possible already ), anything
> that inherits 'perl-module.eclass' has a bit of magic, in that neither
> SRC_URI or HOMEPAGE is required in the ebuild, and it just gets the
> package name from what gentoo is using. We've tried to be as close to
> upstream as possible for the ease of maintenance.

Actually the eclass seems to fill SRC_URI and HOMEPAGE, so when you source
the ebuild they are available.

> However, there are still exception cases, for instance, BioPerl has to
> define 'MY_PN' to tell the perl-module eclass to use a different token
> ( and when this is present, it should be sufficient to say that that
> should be the remote-id instead of the package name:
> see dev-perl/Moose # an example with neither src_uri or homepage
> see sci-biology/bioperl # an example where the package name has
> been forced overridden as its changed upstream
>
> But resolving module names is much trickier, its easy-ish to map a
> module name to a package using the service, but doing it the other way
> round is not so straight forward, as one package can have many
> modules, and its common in perl to state dependencies in terms of the
> module to require, not the package its in, but there's also often a
> defacto "main module".
>
> But I'm myself still working out how to best do that with the service
> , so auto-populating a "cpan-module" identifier can be left to later,
> its just something I considered "useful" to have metadata wise because
> that value is more useful to users.

Yeah, not very important, but seems to work with this patch:
https://github.com/iksaif/portage-janitor/commit/972aff94744741e34e99f917337430d245883c48

Example:
$ python remoteids.py --diff WWW-Bugzilla Moose bioperl
--- a/dev-perl/WWW-Bugzilla/metadata.xml
+++ b/dev-perl/WWW-Bugzilla/metadata.xml
@@ -3,6 +3,7 @@
<pkgmetadata>
<herd>perl</herd>
<upstream>
+ <remote-id type="cpan-module">WWW::Bugzilla</remote-id>
<remote-id type="cpan">WWW-Bugzilla</remote-id>
</upstream>
</pkgmetadata>
--- a/dev-perl/Moose/metadata.xml
+++ b/dev-perl/Moose/metadata.xml
@@ -3,6 +3,7 @@
<pkgmetadata>
<herd>perl</herd>
<upstream>
+ <remote-id type="cpan-module">Moose</remote-id>
<remote-id type="cpan">Moose</remote-id>
</upstream>
</pkgmetadata>
--- a/sci-biology/bioperl/metadata.xml
+++ b/sci-biology/bioperl/metadata.xml
@@ -8,6 +8,7 @@
<flag name="db">Install sci-biology/bioperl-run</flag>
</use>
<upstream>
+ <remote-id type="cpan-module">BioPerl</remote-id>
<remote-id type="cpan">BioPerl</remote-id>
</upstream>
</pkgmetadata>

--
Corentin Chary
http://xf.iksaif.net/
 
Old 04-20-2012, 07:35 PM
Kent Fredric
 
Default RFC: Add new remote-id types in metadata.dtd

On 21 April 2012 01:34, Corentin Chary <corentin.chary@gmail.com> wrote:
> Yeah, not very important, but seems to work with this patch:
> https://github.com/iksaif/portage-janitor/commit/972aff94744741e34e99f917337430d245883c48
>
> Example:
> $ python remoteids.py --diff WWW-Bugzilla Moose bioperl
> --- a/dev-perl/WWW-Bugzilla/metadata.xml
> +++ b/dev-perl/WWW-Bugzilla/metadata.xml
> @@ -3,6 +3,7 @@
> *<pkgmetadata>
...
> --
> Corentin Chary
> http://xf.iksaif.net/
>

What does it to for say, Scalar-List-Utils ? or libwww-perl ?


--
Kent

perl -e* "print substr( "edrgmaM* SPA NOcomil.ic@tfrken", $_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

http://kent-fredric.fox.geek.nz
 
Old 04-20-2012, 08:33 PM
Corentin Chary
 
Default RFC: Add new remote-id types in metadata.dtd

On Fri, Apr 20, 2012 at 9:35 PM, Kent Fredric <kentfredric@gmail.com> wrote:
> On 21 April 2012 01:34, Corentin Chary <corentin.chary@gmail.com> wrote:
>> Yeah, not very important, but seems to work with this patch:
>> https://github.com/iksaif/portage-janitor/commit/972aff94744741e34e99f917337430d245883c48
>>
>> Example:
>> $ python remoteids.py --diff WWW-Bugzilla Moose bioperl
>> --- a/dev-perl/WWW-Bugzilla/metadata.xml
>> +++ b/dev-perl/WWW-Bugzilla/metadata.xml
>> @@ -3,6 +3,7 @@
>> *<pkgmetadata>
> ...
>> --
>> Corentin Chary
>> http://xf.iksaif.net/
>>
>
> What does it to for say, Scalar-List-Utils ? *or libwww-perl ?

Nothing, it's one of the cases you'll have to handle by-hand.
Another option is to parse the HTML from
http://search.cpan.org/dist/Scalar-List-Utils/ but I'm not a fan of
HTML-parsing.

--
Corentin Chary
http://xf.iksaif.net
 
Old 04-21-2012, 03:33 AM
Kent Fredric
 
Default RFC: Add new remote-id types in metadata.dtd

On 21 April 2012 08:33, Corentin Chary <corentin.chary@gmail.com> wrote:
> On Fri, Apr 20, 2012 at 9:35 PM, Kent Fredric <kentfredric@gmail.com> wrote:
>> On 21 April 2012 01:34, Corentin Chary <corentin.chary@gmail.com> wrote:
>>> Yeah, not very important, but seems to work with this patch:
>>> https://github.com/iksaif/portage-janitor/commit/972aff94744741e34e99f917337430d245883c48
>>>
>>> Example:
>>> $ python remoteids.py --diff WWW-Bugzilla Moose bioperl
>>> --- a/dev-perl/WWW-Bugzilla/metadata.xml
>>> +++ b/dev-perl/WWW-Bugzilla/metadata.xml
>>> @@ -3,6 +3,7 @@
>>> *<pkgmetadata>
>> ...
>>> --
>>> Corentin Chary
>>> http://xf.iksaif.net/
>>>
>>
>> What does it to for say, Scalar-List-Utils ? *or libwww-perl ?
>
> Nothing, it's one of the cases you'll have to handle by-hand.
> Another option is to parse the HTML from
> http://search.cpan.org/dist/Scalar-List-Utils/ but I'm not a fan of
> HTML-parsing.
>
> --
> Corentin Chary
> http://xf.iksaif.net
>

http://api.metacpan.org/release/Scalar-List-Utils

Will be far better an option than parsing HTML =)

And even better:

curl -XPOST 'http://api.metacpan.org/module/_search' -d '
{
"fields": [
"module.name",
"release"
],
"query": {
"constant_score": {
"filter" : {
"and" : [
{ "term": { "distribution":"Scalar-List-Utils" } },
{ "term": { "status":"latest"} },
{ "term": { "mime":"text/x-script.perl-module"}},
{ "term": { "indexed":"true"}},
{ "term": { "module.authorized":"true"}}
]
}
}
}
}
'


--
Kent

perl -e* "print substr( "edrgmaM* SPA NOcomil.ic@tfrken", $_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

http://kent-fredric.fox.geek.nz
 
Old 04-21-2012, 09:18 AM
Corentin Chary
 
Default RFC: Add new remote-id types in metadata.dtd

On Sat, Apr 21, 2012 at 03:33:18PM +1200, Kent Fredric wrote:
> On 21 April 2012 08:33, Corentin Chary <corentin.chary@gmail.com> wrote:
> > On Fri, Apr 20, 2012 at 9:35 PM, Kent Fredric <kentfredric@gmail.com> wrote:
> >> On 21 April 2012 01:34, Corentin Chary <corentin.chary@gmail.com> wrote:
> >>> Yeah, not very important, but seems to work with this patch:
> >>> https://github.com/iksaif/portage-janitor/commit/972aff94744741e34e99f917337430d245883c48
>
> http://api.metacpan.org/release/Scalar-List-Utils
>
> Will be far better an option than parsing HTML =)
>
> And even better:
>
> curl -XPOST 'http://api.metacpan.org/module/_search' -d '
> {
> "fields": [
> "module.name",
> "release"
> ],
> "query": {
> "constant_score": {
> "filter" : {
> "and" : [
> { "term": { "distribution":"Scalar-List-Utils" } },
> { "term": { "status":"latest"} },
> { "term": { "mime":"text/x-script.perl-module"}},
> { "term": { "indexed":"true"}},
> { "term": { "module.authorized":"true"}}
> ]
> }
> }
> }
> }
> '
>

It is !

$ ./remoteids.py --diff WWW-Bugzilla Moose bioperl Scalar-List-Utils libwww-perl
--- a/dev-perl/WWW-Bugzilla/metadata.xml
+++ b/dev-perl/WWW-Bugzilla/metadata.xml
@@ -3,6 +3,8 @@
<pkgmetadata>
<herd>perl</herd>
<upstream>
+ <remote-id type="cpan-module">WWW::Bugzilla::Search</remote-id>
+ <remote-id type="cpan-module">WWW::Bugzilla</remote-id>
<remote-id type="cpan">WWW-Bugzilla</remote-id>
</upstream>
</pkgmetadata>
--- a/dev-perl/Moose/metadata.xml
+++ b/dev-perl/Moose/metadata.xml
@@ -3,6 +3,17 @@
<pkgmetadata>
<herd>perl</herd>
<upstream>
+ <remote-id type="cpan-module">Class::MOP</remote-id>
+ <remote-id type="cpan-module">Moose::Meta::Attribute::Native::Trait::Cou nter</remote-id>
+ <remote-id type="cpan-module">Moose::Meta::Attribute::Native::Trait::Str ing</remote-id>
+ <remote-id type="cpan-module">Moose::Meta::Method:elegation</remote-id>
+ <remote-id type="cpan-module">Moose::Meta::TypeConstraint::Role</remote-id>
+ <remote-id type="cpan-module">Moose::Meta::Attribute::Custom::Moose</remote-id>
+ <remote-id type="cpan-module">Moose::Meta::Attribute</remote-id>
+ <remote-id type="cpan-module">Moose::Meta::Method</remote-id>
+ <remote-id type="cpan-module">Moose::Error::Croak</remote-id>
+ <remote-id type="cpan-module">Moose::Util::MetaRole</remote-id>
+ <remote-id type="cpan-module">Moose::Role</remote-id>
<remote-id type="cpan">Moose</remote-id>
</upstream>
</pkgmetadata>
--- a/perl-core/Scalar-List-Utils/metadata.xml
+++ b/perl-core/Scalar-List-Utils/metadata.xml
@@ -3,6 +3,9 @@
<pkgmetadata>
<herd>perl</herd>
<upstream>
+ <remote-id type="cpan-module">List::Util</remote-id>
+ <remote-id type="cpan-module">Scalar::Util</remote-id>
+ <remote-id type="cpan-module">List::Util::XS</remote-id>
<remote-id type="cpan">Scalar-List-Utils</remote-id>
</upstream>
</pkgmetadata>
--- a/dev-perl/libwww-perl/metadata.xml
+++ b/dev-perl/libwww-perl/metadata.xml
@@ -3,6 +3,17 @@
<pkgmetadata>
<herd>perl</herd>
<upstream>
+ <remote-id type="cpan-module">LWP::Protocol::data</remote-id>
+ <remote-id type="cpan-module">LWP::Protocol::gopher</remote-id>
+ <remote-id type="cpan-module">LWP:ebug</remote-id>
+ <remote-id type="cpan-module">LWP::UserAgent</remote-id>
+ <remote-id type="cpan-module">LWP::Authen:igest</remote-id>
+ <remote-id type="cpan-module">LWP::Protocol::file</remote-id>
+ <remote-id type="cpan-module">LWP::Protocol::loopback</remote-id>
+ <remote-id type="cpan-module">LWP::Protocol::MyFTP</remote-id>
+ <remote-id type="cpan-module">LWP::Protocol::ftp</remote-id>
+ <remote-id type="cpan-module">LWP::Protocol::cpan</remote-id>
+ <remote-id type="cpan-module">LWP::MemberMixin</remote-id>
<remote-id type="cpan">libwww-perl</remote-id>
</upstream>
</pkgmetadata>


--
Corentin Chary
http://xf.iksaif.net/
 
Old 04-21-2012, 09:32 AM
Michał Górny
 
Default RFC: Add new remote-id types in metadata.dtd

On Thu, 19 Apr 2012 21:32:49 +0200
Corentin Chary <corentin.chary@gmail.com> wrote:

> On Thu, Apr 19, 2012 at 6:54 PM, Michał Górny <mgorny@gentoo.org>
> wrote:
> > On Thu, 19 Apr 2012 17:31:11 +0200
> > Corentin Chary <corentin.chary@gmail.com> wrote:
> >
> >> - * * *<!ATTLIST remote-id type
> >> (freshmeat|sourceforge|sourceforge-jp|cpan|vim|google-code|ctan|pypi|rubyforge|cran)
> >> #REQUIRED>
> >> + * * *<!ATTLIST remote-id type
> >> (freshmeat|sourceforge|sourceforge-jp|cpan|vim|google-code|ctan|pypi|rubyforge|cran|rubygems|github|gito rious|pecl|pear|bitbucket)
> >> #REQUIRED>
> >
> > Wouldn't it be better if we kept them sorted?
>
> There were not, so I just appended them, but I can provide another
> patch sorting them too if preferred.

I'll appreciate if you could do that, along with that cpan-module
addition, and if noone objects, I'll just commit that.

--
Best regards,
Michał Górny
 
Old 04-21-2012, 11:38 AM
Corentin Chary
 
Default RFC: Add new remote-id types in metadata.dtd

On Sat, Apr 21, 2012 at 11:32:26AM +0200, Michał Górny wrote:
> On Thu, 19 Apr 2012 21:32:49 +0200
> Corentin Chary <corentin.chary@gmail.com> wrote:
>
> > On Thu, Apr 19, 2012 at 6:54 PM, Michał Górny <mgorny@gentoo.org>
> > wrote:
> > > On Thu, 19 Apr 2012 17:31:11 +0200
> > > Corentin Chary <corentin.chary@gmail.com> wrote:
> > >
> > >> - * * *<!ATTLIST remote-id type
> > >> (freshmeat|sourceforge|sourceforge-jp|cpan|vim|google-code|ctan|pypi|rubyforge|cran)
> > >> #REQUIRED>
> > >> + * * *<!ATTLIST remote-id type
> > >> (freshmeat|sourceforge|sourceforge-jp|cpan|vim|google-code|ctan|pypi|rubyforge|cran|rubygems|github|gito rious|pecl|pear|bitbucket)
> > >> #REQUIRED>
> > >
> > > Wouldn't it be better if we kept them sorted?
> >
> > There were not, so I just appended them, but I can provide another
> > patch sorting them too if preferred.
>
> I'll appreciate if you could do that, along with that cpan-module
> addition, and if noone objects, I'll just commit that.
>
> --
> Best regards,
> Michał Górny


--- a/metadata/dtd/metadata.dtd 2010-03-02 18:52:11.000000000 +0100
+++ b/metadata/dtd/metadata.dtd 2012-04-21 13:09:52.557223402 +0200
@@ -61,7 +61,7 @@
<!ELEMENT bugs-to (#PCDATA)>
<!-- specify a type of package identification tracker -->
<!ELEMENT remote-id (#PCDATA)>
- <!ATTLIST remote-id type (freshmeat|sourceforge|sourceforge-jp|cpan|vim|google-code|ctan|pypi|rubyforge|cran) #REQUIRED>
+ <!ATTLIST remote-id type (bitbucket|cpan|cpan-module|cran|ctan|freshmeat|github|gitorious|google-code|launchpad|pear|pecl|pypi|rubyforge|rubygems|s ourceforge|sourceforge-jp|vim) #REQUIRED>

<!-- category/package information for cross-linking in descriptions
and useflag descriptions -->


--
Corentin Chary
http://xf.iksaif.net/
 
Old 05-12-2012, 07:43 PM
Torsten Veller
 
Default RFC: Add new remote-id types in metadata.dtd

* Corentin Chary <corentin.chary@gmail.com>:
> On Sat, Apr 21, 2012 at 03:33:18PM +1200, Kent Fredric wrote:
> > { "term": { "status":"latest"} },
> > { "term": { "module.authorized":"true"}}

What does this mean?
- "latest"? this term looks like maintenance work.
- what is "authorized"?

It doesn't even list "Moose" for Moose?
02packages.details.txt.gz lists 72 package names for Moose-2.0602.

> --- a/dev-perl/Moose/metadata.xml
> +++ b/dev-perl/Moose/metadata.xml
> @@ -3,6 +3,17 @@
> <pkgmetadata>
> <herd>perl</herd>
> <upstream>
> + <remote-id type="cpan-module">Class::MOP</remote-id>
> + <remote-id type="cpan-module">Moose::Meta::Attribute::Native::Trait::Cou nter</remote-id>
> + <remote-id type="cpan-module">Moose::Meta::Attribute::Native::Trait::Str ing</remote-id>
> + <remote-id type="cpan-module">Moose::Meta::Method:elegation</remote-id>
> + <remote-id type="cpan-module">Moose::Meta::TypeConstraint::Role</remote-id>
> + <remote-id type="cpan-module">Moose::Meta::Attribute::Custom::Moose</remote-id>
> + <remote-id type="cpan-module">Moose::Meta::Attribute</remote-id>
> + <remote-id type="cpan-module">Moose::Meta::Method</remote-id>
> + <remote-id type="cpan-module">Moose::Error::Croak</remote-id>
> + <remote-id type="cpan-module">Moose::Util::MetaRole</remote-id>
> + <remote-id type="cpan-module">Moose::Role</remote-id>
> <remote-id type="cpan">Moose</remote-id>
> </upstream>
> </pkgmetadata>
 
Old 05-17-2012, 12:02 AM
Kent Fredric
 
Default RFC: Add new remote-id types in metadata.dtd

On 13 May 2012 07:43, Torsten Veller <tove@gentoo.org> wrote:
> * Corentin Chary <corentin.chary@gmail.com>:
>> On Sat, Apr 21, 2012 at 03:33:18PM +1200, Kent Fredric wrote:
>> > * * * * * * * * * * * * * * * * * * { "term": { "status":"latest"} },
>> > * * * * * * * * * * * * * * * * * * { "term": { "module.authorized":"true"}}
>
> What does this mean?
> - "latest"? this term looks like maintenance work.
> - what is "authorized"?

"latest" means that it will fetch metadata for whatever is deemed the
most recent non-dev release, which is really the only sane option to
go for if you want a list of modules that currently pertain to the
distribution. You could request *all* releases and then find a union
of elements ... but that would be both erroneous and very time
consuming.

> It doesn't even list "Moose" for Moose?

Its probably falling outside the initial 10 results, I forgot it did that,

> 02packages.details.txt.gz lists 72 package names for Moose-2.0602.
>

Need to bolt on a { "size": 100 } to the query to expand how may
results it will return.

curl -XPOST 'http://api.metacpan.org/module/_search' -d '
{
"fields": [
"module.name",
"release"
],
"query": {
"constant_score": {
"filter" : {
"and" : [
{ "term": { "distribution":"Moose" } },
{ "term": { "status":"latest"} },
{ "term": {
"mime":"text/x-script.perl-module"}},
{ "term": { "indexed":"true"}},
{ "term": { "module.authorized":"true"}}
]

}
}
},
"size": 100
}

^ that | grep module.name | wc -l # 83

--
Kent

perl -e* "print substr( "edrgmaM* SPA NOcomil.ic@tfrken", $_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

http://kent-fredric.fox.geek.nz
 

Thread Tools




All times are GMT. The time now is 07:55 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org