Kent Fredric posted on Fri, 29 Jun 2012 14:07:58 +1200 as excerpted:

> For the most part it seems to get upstream / portage versioning right,
> but occasionally you get miss-matches for some reason.
>
> It would be nice to allow to provide some mapping mechanism that existed
> on the overlay itself to inform euscan how to map upstream versions to
> downstream ones, but implementing that would be far too complex I feel.
>
> Instead, it would be nice to have a mechanism in the interface to set a
> "Upstream version is" value for each package if euscan can't tell.
>
> Ie:
>
> http://euscan.iksaif.net/package/dev-perl/HTML-TreeBuilder-LibXML/
>
> Upstream is 0.71 , portage is ( normalised ) to 0.710.0 , and these are
> in fact the same version. So in 0.710.0 , it would be nice to be able to
> set the upstream version manually to 0.71 so that euscan no longer
> reported it as outdated.

What about a simple variable that can be set in the ebuild? Say for the
above example something like:

EUSCAN_VERSION=0.71

I don't know how difficult that would be for euscan to pickup on, but
since this would have no bearing on actual package behavior, only on
euscan, I'd guess it shouldn't require going thru the formal PMS process,
tho specifying it well enough to know whether it can be a function or
must be a straight variable assignment, etc, as well as whether quotes
are required or not, would be useful, and that's normally part of the PMS
process so at least getting input from them would be useful.

Alternatively, define some formula that can be placed in the package's
metadata.xml. That's perhaps a better functionality match (we're talking
about metadata, after all), but getting a formula that can deal with all
the corner-cases is likely to be more difficult (and take longer) than
simply specifying a variable to be defined for each ebuild that euscan
can't immediately get correct.

--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman

On Fri, 29 Jun 2012 14:07:58 +1200
Kent Fredric <kentfredric@gmail.com> wrote:

> On 27 June 2012 19:51, Federico "fox" Scrinzi <fox91@anche.no> wrote:
> > The main question is: what would you like to have on this dashboard?
> > Currently (in the development version) there's the possibility to
> > login and watch/unwatch packages/categories/herds/... and see the
> > watched stuff in the account dashboard. We're planning on
> > implementing a weekly(?) custom newsletter based on the packages
> > you're watching, which features would you like?
> >
> > The project repo for the GSoC is here:
> > https://github.com/volpino/euscan
> >
> > Thanks!
> >
>
> For the most part it seems to get upstream / portage versioning right,
> but occasionally you get miss-matches for some reason.
>
> It would be nice to allow to provide some mapping mechanism that
> existed on the overlay itself to inform euscan how to map upstream
> versions to downstream ones, but implementing that would be far too
> complex I feel.
>
> Instead, it would be nice to have a mechanism in the interface to set
> a "Upstream version is" value for each package if euscan can't tell.
>
> Ie:
>
> http://euscan.iksaif.net/package/dev-perl/HTML-TreeBuilder-LibXML/
>
> Upstream is 0.71 , portage is ( normalised ) to 0.710.0 , and these
> are in fact the same version. So in 0.710.0 , it would be nice to be
> able to set the upstream version manually to 0.71 so that euscan no
> longer reported it as outdated.

I think we could actually handle perl pretty easily. I believe euscan
will start using CPAN API to check the package versions, and we can
embed the normalization there.

--
Best regards,
Michał Górny

On Fri, Jun 29, 2012 at 4:07 AM, Kent Fredric <kentfredric@gmail.com> wrote:
> On 27 June 2012 19:51, Federico "fox" Scrinzi <fox91@anche.no> wrote:
>> The main question is: what would you like to have on this dashboard?
>> Currently (in the development version) there's the possibility to login
>> and watch/unwatch packages/categories/herds/... and see the watched
>> stuff in the account dashboard. We're planning on implementing a
>> weekly(?) custom newsletter based on the packages you're watching, which
>> features would you like?
>>
>> The project repo for the GSoC is here: https://github.com/volpino/euscan
>>
>> Thanks!
>>
>
> For the most part it seems to get upstream / portage versioning right,
> but occasionally you get miss-matches for some reason.
>
> It would be nice to allow to provide some mapping mechanism that
> existed on the overlay itself to inform euscan how to map upstream
> versions to downstream ones, but implementing that would be far too
> complex I feel.
>
> Instead, it would be nice to have a mechanism in the interface to set
> a "Upstream version is" value for each package if euscan can't tell.
>
> Ie:
>
> http://euscan.iksaif.net/package/dev-perl/HTML-TreeBuilder-LibXML/
>
> Upstream is 0.71 , portage is ( normalised ) to 0.710.0 , and these
> are in fact the same version. So in 0.710.0 , it would be nice to be
> able to set the upstream version manually to 0.71 so that euscan no
> longer reported it as outdated.
>
> http://euscan.iksaif.net/package/dev-perl/Authen-SASL-Cyrus-server/
>
> 0.13 == 0.13-serve
>
> http://euscan.iksaif.net/package/dev-perl/Module-Extract-Namespaces/
>
> 0.140.200_rc == 0.14_0.2
>
> http://euscan.iksaif.net/package/dev-perl/Math-BaseCnv/
> http://euscan.iksaif.net/package/dev-perl/XML-Tidy/
>
> 1.8 == 1.8.B59BrZ
> 1.8 == 1.8.B2AMvd
>
> ( Upstream for those 2 packages have a versioning scheme tantamount to
> intolerable cruelty.
> https://rt.cpan.org/Public/Bug/Display.html?id=60275 )
>
> http://euscan.iksaif.net/package/dev-perl/Perl-Critic-Moose/
> 0.999.2_rc == 0.999._002
>
> http://euscan.iksaif.net/package/dev-perl/aliased/
> 0.300.100_rc == 0.30_0.1
>
> http://euscan.iksaif.net/package/dev-perl/EV/
> 4.110.0 *== 4.11
>
>
> --
> Kent
>
> perl -e* "print substr( "edrgmaM* SPA NOcomil.ic@tfrken", $_ * 3,
> 3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"
>
> http://kent-fredric.fox.geek.nz
>

Something that could help with that: we plan to add something like
debian/watch in metadata.xml. This should allow to specify a regexp to
mangle versions (sed like syntax: /match/replace/). For some specific
packages, we also already have specific handlers (cpan, php, pypi,
etc...) which have specific version mangling functions.

On Fri, Jun 29, 2012 at 8:53 AM, Michał Górny <mgorny@gentoo.org> wrote:
> On Fri, 29 Jun 2012 14:07:58 +1200
> Kent Fredric <kentfredric@gmail.com> wrote:
>
>> On 27 June 2012 19:51, Federico "fox" Scrinzi <fox91@anche.no> wrote:
>> > The main question is: what would you like to have on this dashboard?
>> > Currently (in the development version) there's the possibility to
>> > login and watch/unwatch packages/categories/herds/... and see the
>> > watched stuff in the account dashboard. We're planning on
>> > implementing a weekly(?) custom newsletter based on the packages
>> > you're watching, which features would you like?
>> >
>> > The project repo for the GSoC is here:
>> > https://github.com/volpino/euscan
>> >
>> > Thanks!
>> >
>>
>> For the most part it seems to get upstream / portage versioning right,
>> but occasionally you get miss-matches for some reason.
>>
>> It would be nice to allow to provide some mapping mechanism that
>> existed on the overlay itself to inform euscan how to map upstream
>> versions to downstream ones, but implementing that would be far too
>> complex I feel.
>>
>> Instead, it would be nice to have a mechanism in the interface to set
>> a "Upstream version is" value for each package if euscan can't tell.
>>
>> Ie:
>>
>> http://euscan.iksaif.net/package/dev-perl/HTML-TreeBuilder-LibXML/
>>
>> Upstream is 0.71 , portage is ( normalised ) to 0.710.0 , and these
>> are in fact the same version. So in 0.710.0 , it would be nice to be
>> able to set the upstream version manually to 0.71 so that euscan no
>> longer reported it as outdated.
>
> I think we could actually handle perl pretty easily. I believe euscan
> will start using CPAN API to check the package versions, and we can
> embed the normalization there.

It's already the case:
https://github.com/iksaif/euscan/blob/master/pym/euscan/handlers/cpan.py
but my mangling functions are probably broken in some cases. If
somebody with a better knowledge of CPAN versionning scheme could fix
them it would be great !

Thanks,

On 29 June 2012 17:32, Duncan <1i5t5.duncan@cox.net> wrote:
>
> EUSCAN_VERSION=0.71
>
> I don't know how difficult that would be for euscan to pickup on, but
> since this would have no bearing on actual package behavior, only on
> euscan, I'd guess it shouldn't require going thru the formal PMS process,
> tho specifying it well enough to know whether it can be a function or
> must be a straight variable assignment, etc, as well as whether quotes
> are required or not, would be useful, and that's normally part of the PMS
> process so at least getting input from them would be useful.
>
> Alternatively, define some formula that can be placed in the package's
> metadata.xml. *That's perhaps a better functionality match (we're talking
> about metadata, after all), but getting a formula that can deal with all
> the corner-cases is likely to be more difficult (and take longer) than
> simply specifying a variable to be defined for each ebuild that euscan
> can't immediately get correct.

The problem is with this approach, some devs will want to set
EUSCAN_VERSION automagically using mangling $PV

As it is, we *already* have something equivalent to EUSCAN_VERSION for
things derived from perl-module.eclass, MODULE_VERSION=

Its not 1:1 identical to the intent of EUSCAN_VERSION, MODULE_VERSION
is only really required in the generation of SRC_URI , but because of
this, there is a loose MODULE_VERSION <-> distfile mapping, and a much
looser $PV <-> MODULE_VERSION association.

Sure, its fine for MODULE_VERSION to be made by mangling $PV, but the
problem is that the *reason* people mangle $PV to make MODULE_VERSION
is so they don't have to update MODULE_VERSION manually when bumping
the package.

Adding a non-bash non-$PV-manglable field EUSCAN_VERSION field will
possibly make manually incrementing this value a mandatory thing.

Not to mention, if it turns out to be "wrong" on the EUSCAN index,
some dev has to drop the change into the repository, do all the
manifest updating and commit signing and pushing it to CVS nonsense,
when really, its metadata only relevant to euscan, so its really in
the wrong place to start with.

Having it handled as an exception list on the euscan would be much
tidier, and the path from noticing the problem to solving the problem
becomes substantially shorter.



--
Kent

perl -e* "print substr( "edrgmaM* SPA NOcomil.ic@tfrken", $_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

http://kent-fredric.fox.geek.nz

On 30 June 2012 00:13, Corentin Chary <iksaif@gentoo.org> wrote:

> It's already the case:
> https://github.com/iksaif/euscan/blob/master/pym/euscan/handlers/cpan.py
> but my mangling functions are probably broken in some cases. If
> somebody with a better knowledge of CPAN versionning scheme could fix
> them it would be great !
>
> Thanks,

The thing is there isn't a true versioning scheme for CPAN, just a
defacto one agreed upon by the community. There are several
versioning schemes in employ, but the mechanics of each are rather
messy, and then you have some lovely fellow like pip come along and
put garbage in their version, and we have to handle it manually.

For the most part though, people use "sensible" versions of a very few
basic varieties, and we downstream normalise these smattering of
varieties into a single form to make everything nice and tidy. Its
still a work in progress migrating older ebuilds to our new
normalisation scheme, but its getting there.

And we do have a tool that will convert /most/ CPAN versions into
portage versions, which works as long as upstream are sane:

dev-perl/Gentoo-PerlMod-Version

Which relies on another perl module 'version.pm' (
virtual/perl-version ) which handles most the real dirty work of
normalising things, and we just do a bit of post-processing of that to
make it nicer for us ( mostly stripping leading 0's in digit groups,
and mapping the upstream 'developer release' hints ( ie: -TRIAL or
_01 components ) to the _rc suffix.

If you wanted to you could call that script, or delve into the guts of
it and version.pm and try understand how it works, but you could be
there a while.

But we're unfortunately going to *always* need a way to correct
versions, because upstream occasionally produce bogus nonsense
versions that don't even make sense. ( ie: making versions go
backwards and expect it to work, but it does, because the cpan indexer
takes whatever was most recently uploaded ... or something like that.
)


--
Kent

perl -e* "print substr( "edrgmaM* SPA NOcomil.ic@tfrken", $_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

http://kent-fredric.fox.geek.nz

Am Mittwoch, 27. Juni 2012, 09:51:06 schrieb Federico "fox" Scrinzi:
> The main question is: what would you like to have on this dashboard?

Here's another suggestion for euscan: right now we already have rss feeds, but
they are far too verbose for me and clutter up my news list. I dont really
need an item whenever something enters portage

How about a news feed that only contains "upstream" items?

Cheers,
Andreas

--
Andreas K. Huettel
Gentoo Linux developer
kde (team lead), sci, tex, arm, printing
dilfridge@gentoo.org
http://www.akhuettel.de/

On Monday 09 July 2012 23:41:06 you wrote:
> The main question is: what would you like to have on this dashboard?
> Currently (in the development version) there's the possibility to login
> and watch/unwatch packages/categories/herds/... and see the watched
> stuff in the account dashboard. We're planning on implementing a
> weekly(?) custom newsletter based on the packages you're watching, which
> features would you like?

I talked with Federico on irc.

I proposed to have an 'integration' between euscan and pybugz.

In this manner everyone can file a version bump request via euscan.
What do you think about?
--
Agostino Sarubbo / ago -at- gentoo.org
Gentoo/AMD64 Arch Security Liaison
GPG: 0x7CD2DC5D

On Tue, 10 Jul 2012 00:04:15 +0200
Agostino Sarubbo <ago@gentoo.org> wrote:

> I proposed to have an 'integration' between euscan and pybugz.

Sounds superficially like a giant security hole / spam gateway. What
kind of integration are we talking about?

> In this manner everyone can file a version bump request via euscan.
> What do you think about?

If the false positives go down significantly, we wouldn't need
interested users to "report" version bumps - you could automate that
all the way, and publish links to the bugs already reported.

But couldn't we more simply and securely implement publishing a
hyperlink from euscan's pages to the bugs.g.o form that fills out some
of the required information in advance? That way it's everybody's
personal bugzilla accounts that are responsible for the spam, and we
might more easily find duplicates.

Speaking of duplicates, the way subsequent version bump requests for
subsequent versions (in different SLOTs, mind you) are usually treated
now is that the later bugs are marked as duplicates of the former and
that the former bug's Summary is updated to show the currently best
version(s). If we're going to stick to that, we *definitely* need a way
to catch out duplicates.


jer

On 10 July 2012 13:28, Jeroen Roovers <jer@gentoo.org> wrote:
> On Tue, 10 Jul 2012 00:04:15 +0200
> Agostino Sarubbo <ago@gentoo.org> wrote:
>
>> I proposed to have an 'integration' between euscan and pybugz.
>
> Sounds superficially like a giant security hole / spam gateway. What
> kind of integration are we talking about?
>
>> In this manner everyone can file a version bump request via euscan.
>> What do you think about?
>
> If the false positives go down significantly, we wouldn't need
> interested users to "report" version bumps - you could automate that
> all the way, and publish links to the bugs already reported.
>
> But couldn't we more simply and securely implement publishing a
> hyperlink from euscan's pages to the bugs.g.o form that fills out some
> of the required information in advance? That way it's everybody's
> personal bugzilla accounts that are responsible for the spam, and we
> might more easily find duplicates.
>
> Speaking of duplicates, the way subsequent version bump requests for
> subsequent versions (in different SLOTs, mind you) are usually treated
> now is that the later bugs are marked as duplicates of the former and
> that the former bug's Summary is updated to show the currently best
> version(s). If we're going to stick to that, we *definitely* need a way
> to catch out duplicates.
>

This is verging on that oft requested thing that Gentoo could do with,
a sort of task queue for regular and repeated tasks that can be
described in one line and aren't worthy of a full bug.

ie: Version Bumps, stabilisation requests, generic repeated
transformations ( ie: make the metadata better, etc )

Things that if they really needed a full blown bug could be
cross-linked with bugzilla, but wouldn't need be the default option.

Though, thats really an entirely different discussion, I'm just
teasing the subject so ideas brew and the discussion eventually turns
up.

--
Kent

perl -e "print substr( "edrgmaM SPA NOcomil.ic@tfrken", $_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

http://kent-fredric.fox.geek.nz