Linux Archive

Linux Archive (http://www.linux-archive.org/)
-   Gentoo Embedded (http://www.linux-archive.org/gentoo-embedded/)
-   -   Licence compliance - capturing all source files used to make a build? (http://www.linux-archive.org/gentoo-embedded/639100-licence-compliance-capturing-all-source-files-used-make-build.html)

Peter Stuge 02-29-2012 04:39 PM

Licence compliance - capturing all source files used to make a build?
 
Ed W wrote:
> Hi, how do others handle open source licence compliance when building
> some base system using gentoo?

Review the packages that get built, and adhere to their licenses. It
can be a fair bit of work.


> In particular I guess simply capturing the ebuilds is not sufficient
> and it's necessary to capture and distribute all the source and
> patch files used to create a build.

How do you build your system? If you use catalyst, the open source
gold standard citizen publishes spec files, snapshot, toolchain and
toolchain source.


> The emerge tool doesn't obviously give a way to capture this stuff.

First step is to analyze licenses. emerge does know the license for a
package, and it is available in /var/db/pkg/ after install.


> I looked in the eclasses, particularly the epatch file and I'm not
> clear that I can easily hook into that.

If you have patches which use a different license than the package
they modify then you have more work to do. Portage doesn't help here.
A good start would be to add record of all patches applied by emerge.
Indeed add it into the epatch command.


> Whilst the above is largely targeting GPL type licences, are there
> other things I should consider for other licences?

Yes. You obviously need to adhere to all licenses used by the
packages in your system. :)


> Other things I need to ensure I distribute for GPL?

Read the licenses, really.


> Any pointers to (simple) documentation on how one can be a
> compliant open source citizen..?

It's not simple. You have to learn the requirements of each license
and see if and how they allow themselves to be combined. There are
businesses doing exactly that. If you want to DIY I think you just
have to start by reading the licenses. You may or may not want an
IP lawyer sitting beside you while doing it.


//Peter

Peter Stuge 03-01-2012 02:36 AM

Licence compliance - capturing all source files used to make a build?
 
Ed W wrote:
>> It's not simple. You have to learn the requirements of each license
>> and see if and how they allow themselves to be combined. There are
>> businesses doing exactly that. If you want to DIY I think you just
>> have to start by reading the licenses. You may or may not want an
>> IP lawyer sitting beside you while doing it.
>
> This is the kind of unhelpful answer that I can find plenty of examples of
> through google...
>
> Consider that all software comes with some kind of licence. Generally if
> you ask a non opensource company about licensing costs then even the sales
> droid can help you out. I do find it quite baffling that on average if you
> question an opensource user then their answer on licensing is that one
> should redirect the question to one of the most expensive and opaque
> professions on earth...

That's not what I wrote, I wrote that *you* should read the licenses.

I wouldn't have mentioned an IP lawyer at all had it not been for the
fact that I know that you are in the US. :)

Lawyer or not depends on how much self education one is interested
in. I've given open source law advice to customers and have had some
contact guiding IP lawyers to better understanding. But it really is
a big field, and you weren't clear on if you already had a good
understanding of the requirements in the various licenses.


> If your mate gave you that answer in the pub when you asked what
> price for a beer you would immediately cotton on that they don't
> really know and are bluffing...

No bluff.


> The bit people seem to miss is that legal documents are for forcing
> arbitration in the event of dispute - in the meantime people are supposed
> to rub along in a cooperative manner. That many OSS advocates seem to feel
> that employing expensive lawyers is the only way to talk to them shows that
> they are probably missing the bigger picture...

Mh. A pretty picture, but the reason IP lawyers can live off open
source alone is that there are people who do not and do not want to
understand the licenses themselves. They may or may not desire to
behave well. If they do want to behave well, they may want IP lawyers
to not have to learn themselves, or because they don't trust
themselves or their understanding of the legal system. If they don't
want to behave well the IP lawyers will be working for the opposition
and have a job hunting them down. :)

I do not and have never advocated immediately talking to a lawyer
over first trying to understand licenses oneself.

That said, the legal systems of the world are such that it actually
doesn't matter what the own understanding is, the only thing that
matters is the opinions of the legal systems. And that's where the
lawyers have experience.

OTOH, I think that by now, there are enough documented cases that
allow also developers themselves to understand the issues if they
want to, and I very much encourage this.


> On a more constructive note: I think I do understand the key terms of the
> main software licences we use, from my understanding they are not all that
> onerous.

Sounds good. It is important to have gotten this right, since it is
what drives everything else.


> So can we perhaps move this topic onto tips, suggestions and
> practical matters about moving forward? I'm not sure that one of
> the most expensive type of lawyers is best employed talking
> scripting tips?

No, and I never said they were.


>> If you have patches which use a different license than the package
>> they modify then you have more work to do. Portage doesn't help here.
>> A good start would be to add record of all patches applied by emerge.
>> Indeed add it into the epatch command.
>
> OK, so this is what I asked the list. Please don't turn it back at me...
>
> Firstly can we not assume that the patches in gentoo *are* in compliance,
> otherwise gentoo's various packaged binaries would cause Gentoo to be out
> of compliance?

Hm, this last sentence is inconsistent, but I guess it should be
without the first "not" based on the rest you write.

If you dare assume that all patches use the same license as the
package they apply to, then I would say that it's actually easy
to do an inventory of the packages and licenses your system uses.

The package/license inventory is the first step.


> So, back to the problem: one of the bigger challenges seems to be how to
> actually capture the absolute list of patches applied? Any suggestions?

I think you will have to extend portage. You can have a look at PMS
(emerge app-doc/pms) to find what is currently possible. I think the
A environment variable (Table 12.1 page 46) is the closest to what
you want, but it does not seem to cover patches.


> I already suggested creating my own "patch" utility which saves it's
> input - seems ugly - other suggestions?

I gave it already in the last mail; you should modify the epatch
function to also do some bookkeeping.


> I'm not using catalyst.

Ok, then you have more manual work to do, because catalyst already
does some of the things required by e.g. GPL for you, or at least it
makes them easier to do.


> Any tips from others on capturing, presenting, managing and
> deploying GPL code?

This is a little like asking "Any tips from others on building a car
from scratch?" It's not a very practical question; it's much too
broad. I think it would be good to focus on something specific.


> Hoping for useful answers here rather than "talk to some really
> expensive professional who knows nothing about programming".

The programming involved is trivial IMO, although there are of course
pricey commercial products for software inventory and license
management out there. The difficult part is to understand the legal
requirements that create technical requirements for your distribution
of open source software.

That process obviously has nothing to do with programming, and if you
need legal advice it really is a good idea to talk to lawyers, not
programmers.

At the same time, I do advocate studying and discussing licenses.
They are not magical, I actually find them pretty straightforward.


> Gentoo seems very attractive for building embedded system - however, there
> seem to be some missing steps to help with deployment. I thought that was
> ontopic for this list? Any tips from others who are building things?

I use catalyst, and I control what gets deployed with custom ebuilds
and snapshots. The fewer packages in the final system the better;
less stuff to track.


//Peter

Peter Stuge 03-01-2012 01:53 PM

Licence compliance - capturing all source files used to make a build?
 
Hey,

Ed W wrote:
>> I wouldn't have mentioned an IP lawyer at all had it not been for the
>> fact that I know that you are in the US. :)
>
> I'm in the UK

Ha! Awesome. :) Sorry, must have mixed you up then!


>> I use catalyst, and I control what gets deployed with custom ebuilds
>> and snapshots. The fewer packages in the final system the better;
>> less stuff to track.
>
> Whilst I guess it should be possible to tear apart catalyst and find out
> how they do it, does anyone happen to know or have a heads up on the code
> for catalyst?

The catalyst code has no part in this, but it takes a portage snapshot
as one of it's inputs, and if you maintain a custom snapshot (with
only packages you need) then you know what gets used.


> It must be a solved problem so I should think others have solved
> this in various ways?

I'm not sure it is a solved problem. If you want a different solution
than basically maintaining your own portage snapshot then the easiest
way to track patches is (third time now) to add bookkeeping in the
epatch function.


//Peter

Peter Stuge 03-01-2012 03:27 PM

Licence compliance - capturing all source files used to make a build?
 
wireless wrote:
> It just seems like this question should be solved and already documented
> somewhere? With dozens (hundreds) of commercial linux distros, surely
> they list licenses for codes they include therein?

The license question is easy to answer using what goes into /var/db/pkg.

The next step is obviously to make sure that you are in compliance.
Part of that means being able to sometimes provide source code for
the binaries you have distributed. If those binaries include patches
added by portage build scripts then it's not sufficient to provide
the upstream tarball.


//Peter

Peter Stuge 03-01-2012 06:05 PM

Licence compliance - capturing all source files used to make a build?
 
Ed W wrote:
>>> Whilst I guess it should be possible to tear apart catalyst and find out
>>> how they do it, does anyone happen to know or have a heads up on the code
>>> for catalyst?
>>
>> The catalyst code has no part in this, but it takes a portage snapshot
>> as one of it's inputs, and if you maintain a custom snapshot (with
>> only packages you need) then you know what gets used.
>
> But not all the patches are in the portage tree? Trivial example might
> be the kernel where the ebuild is tiny and references an http location
> for the patches?

Then you would change the kernel ebuild in your snapshot, so that it
becomes self-contained.

For the specific example of the kernel you could of course just pick
vanilla-sources, but the issue is real.


> My understanding is that for a GPL licence one should provide a
> copy of these patches in the "code dump", not just an http link?
> Is that your understanding?

I think your understanding is incomplete, and I recommend that you
read through the license again.

There isn't just a single way to provide the source, but yes, if you
have downloaded and included a patch in your binary, then you have to
provide that patch yourself, because if you refer to someone else and
they stop providing the patch you would no longer be in compliance.


> So by implication it's not clear that catalyst does satisfy your GPL
> requirements for distribution?

I never say it did. I said that it helps with some things.


> I suspect something more is probably happening, eg some of the linked
> patches probably get included into the source download location and
> probably you can pick them up there - however, there are now a LOT of
> ways to fetching source and patches and it would be hard to be sure
> of 100% coverage?

Fourth time: Add bookkeeping into the epatch function.

Downloading is irrelevant, especially since sometimes many more
patches are downloaded than are actually applied.


> Has someone done some actual probing on this? Peter what does catalyst
> provide for say gcc/kernel sources in it's source output? All the patches?

It's the other way around:

You provide a snapshot to catalyst, and catalyst builds kernel from
that. You say what you want catalyst to build, and you create the
package.

You may end up doing more ebuild maintenance, but you likely want to
do just that anyway, in order to keep track of what actually goes
into your system.


//Peter

Peter Stuge 03-02-2012 01:35 PM

Licence compliance - capturing all source files used to make a build?
 
Mike Frysinger wrote:
> if you capture all of the $PORTDIR/$CATEGORY/$PN/ and $A, then
> there should be no need to manually hook into epatch

The point of hooking into epatch would be to only have exactly those
patches which get applied. Some ebuilds come with a huge set of
patches, but only few may be applied depending on USE and version.
It's nice to have just the right ones.


//Peter


All times are GMT. The time now is 05:11 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.