FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo Development

 
 
LinkBack Thread Tools
 
Old 05-24-2012, 07:47 PM
Mike Frysinger
 
Default comprehensive eclass checking in repoman

i implemented eclass checking for some of the most common ones in the tree,
but Zac didn't particularly care for the maintaining of lists of functions
used by eclasses directly in repoman (due to the concern of them getting out
of sync).

so the proposal is to utilize the existing eclass documentation markers to
extract the complete list of functions provided by an eclass. the upside is
the metadata stays current, and we can scale better to all eclasses w/out
requiring manual intervention. the downside is that if people don't properly
document their eclasses, repoman might throw false positives (warnings, not
errors) about unused eclasses being inherited, and will miss throwing errors
when functions are used but the respective eclasses aren't inherited.

however, i think that's a good hammer to throw at eclass maintainers to keep
their documentation up-to-date and accurate. any other opinions/feedback ?
-mike
 
Old 05-24-2012, 08:12 PM
Kent Fredric
 
Default comprehensive eclass checking in repoman

On 25 May 2012 07:47, Mike Frysinger <vapier@gentoo.org> wrote:
> -mike

Were it me, I'd have a tool that scrapes the eclass files's
documentation and emits a .json file which repoman can then optionally
use for consistency checks.

Mostly because I don't like the idea of repoman re-probing all the
eclasses every time you check an ebuild, and would like a way of
consistency checking an ebuild and seeing what metadata it produced
without needing to see it through the eyes of "repoman full" on a
consuming ebuild.

My 2c
--
Kent

perl -e* "print substr( "edrgmaM* SPA NOcomil.ic@tfrken", $_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

http://kent-fredric.fox.geek.nz
 
Old 05-24-2012, 08:19 PM
Mike Frysinger
 
Default comprehensive eclass checking in repoman

On Thursday 24 May 2012 16:12:35 Kent Fredric wrote:
> Were it me, I'd have a tool that scrapes the eclass files's
> documentation and emits a .json file which repoman can then optionally
> use for consistency checks.

python provides its own pickling system. no need to add yet another layer.

> Mostly because I don't like the idea of repoman re-probing all the
> eclasses every time you check an ebuild

caching of data most likely will be done (the decision will be data driven),
but that's independent of the underlying behavioral question.
-mike
 
Old 05-24-2012, 08:28 PM
Zac Medico
 
Default comprehensive eclass checking in repoman

On 05/24/2012 01:19 PM, Mike Frysinger wrote:
> On Thursday 24 May 2012 16:12:35 Kent Fredric wrote:
>> Were it me, I'd have a tool that scrapes the eclass files's
>> documentation and emits a .json file which repoman can then optionally
>> use for consistency checks.
>
> python provides its own pickling system. no need to add yet another layer.

Python's json module is similar to the pickly module, so you might just
use that instead. For example, I've converted /var/cache/edb/mtimedb
from pickle to json:

http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=848da97a64b2d3d13c3fbc794c5 7dae714009854

>> Mostly because I don't like the idea of repoman re-probing all the
>> eclasses every time you check an ebuild
>
> caching of data most likely will be done (the decision will be data driven),
> but that's independent of the underlying behavioral question.

I expect that reading and validating the cache is probably not going to
be much faster than just parsing the eclasses over again.
--
Thanks,
Zac
 
Old 05-24-2012, 08:52 PM
Kent Fredric
 
Default comprehensive eclass checking in repoman

On 25 May 2012 08:28, Zac Medico <zmedico@gentoo.org> wrote:
>
> I expect that reading and validating the cache is probably not going to
> be much faster than just parsing the eclasses over again.
> --

Unless, you don't care if the cache is out-dated because the cache is
optional / the syntax checking is optional, and its only made
available when you generate it manually.

And considering how fast eclasses change, I doubt you'd need to
regenerate it often. Though how much time it takes to parse and stuff
really needs to be properly benched, its more that there is an
intermediate state that can be inspected by human eyes instead of a
lot of magic going on



--
Kent

perl -e* "print substr( "edrgmaM* SPA NOcomil.ic@tfrken", $_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"

http://kent-fredric.fox.geek.nz
 
Old 05-24-2012, 09:00 PM
Zac Medico
 
Default comprehensive eclass checking in repoman

On 05/24/2012 01:52 PM, Kent Fredric wrote:
> On 25 May 2012 08:28, Zac Medico <zmedico@gentoo.org> wrote:
>>
>> I expect that reading and validating the cache is probably not going to
>> be much faster than just parsing the eclasses over again.
>> --
>
> Unless, you don't care if the cache is out-dated because the cache is
> optional / the syntax checking is optional, and its only made
> available when you generate it manually.

Putting the responsibility of cache validation on the user is not really
an option here, because it's just going to lead to bug reports of the
form "repoman is using stale eclass info in its checks".
--
Thanks,
Zac
 
Old 05-24-2012, 10:16 PM
Steven J Long
 
Default comprehensive eclass checking in repoman

Mike Frysinger wrote:
> ..the proposal is to utilize the existing eclass documentation markers
> ..the metadata stays current, and we can scale better to all eclasses

> if people don't properly document their eclasses, repoman might throw
> false positives (warnings, not errors) about unused eclasses

> will miss throwing errors when functions are used but the respective
> eclasses aren't inherited.

> however, i think that's a good hammer to throw at eclass maintainers to
> keep their documentation up-to-date and accurate.
> any other opinions/feedback?

I think it's an excellent idea to give this kind of QA early, to avoid
issues like recent eutils inheritance changes in the future; it's not a
hammer so much as a helpful reminder, that improves things for everyone.

You could maybe tighten the false-negative side by scanning all functions
defined in an eclass, and warning if they're undocumented.

Steve.
--
#friendly-coders -- We're friendly, but we're not /that/ friendly ;-)
 
Old 05-25-2012, 12:14 AM
Mike Frysinger
 
Default comprehensive eclass checking in repoman

On Thursday 24 May 2012 18:16:23 Steven J Long wrote:
> You could maybe tighten the false-negative side by scanning all functions
> defined in an eclass, and warning if they're undocumented.

that happens now when you emerge eclass-manpages, but i suspect no one cares.
if you run the script by hand on your eclass, it'll tell you the same thing.
-mike
 
Old 05-25-2012, 03:47 AM
Ryan Hill
 
Default comprehensive eclass checking in repoman

On Thu, 24 May 2012 15:47:09 -0400
Mike Frysinger <vapier@gentoo.org> wrote:

> i implemented eclass checking for some of the most common ones in the tree,
> but Zac didn't particularly care for the maintaining of lists of functions
> used by eclasses directly in repoman (due to the concern of them getting out
> of sync).
>
> so the proposal is to utilize the existing eclass documentation markers to
> extract the complete list of functions provided by an eclass. the upside is
> the metadata stays current, and we can scale better to all eclasses w/out
> requiring manual intervention. the downside is that if people don't properly
> document their eclasses, repoman might throw false positives (warnings, not
> errors) about unused eclasses being inherited, and will miss throwing errors
> when functions are used but the respective eclasses aren't inherited.
>
> however, i think that's a good hammer to throw at eclass maintainers to keep
> their documentation up-to-date and accurate. any other opinions/feedback ?

Is there any sane way to handle sub-eclasses? eg. foo-base inherits
foo-functions.

I have some crazy ideas on how to do eclass versioning I may one day threaten
the world with if they ever let me out of my padded cell.


--
fonts, gcc-porting
toolchain, wxwidgets
@ gentoo.org
 
Old 05-25-2012, 03:50 AM
Ryan Hill
 
Default comprehensive eclass checking in repoman

On Thu, 24 May 2012 21:47:23 -0600
Ryan Hill <dirtyepic@gentoo.org> wrote:

> Is there any sane way to handle sub-eclasses? eg. foo-base inherits
> foo-functions.

Oh and even if there isn't, big +1 from me.


--
fonts, gcc-porting
toolchain, wxwidgets
@ gentoo.org
 

Thread Tools




All times are GMT. The time now is 12:23 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org