FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian Development

 
 
LinkBack Thread Tools
 
Old 12-13-2011, 07:59 PM
Lars Wirzenius
 
Default Binary blobs in source packages

On Tue, Dec 13, 2011 at 08:30:22PM +0000, Philipp Kern wrote:
> But then I don't see how you could avoid circular build-dependencies
> with compilers written in their own language. fpc/fp-compiler does the
> same.

You can avoid it by having a bootstrap compiler written in another
suitable language, which is capable (even if just) of building the
real compiler. So you write two compilers, A and B. A is written
in, say, AWK or Perl, and compiles the source code for B into
executable code. The executable B is then used to actually compile
software written in the language in question. Indeed, during the
package build, you would build the compiler first with A, and then
again with B, and the binary package would contain just the binary
built with B.

The benefit of the simple bootstrapping compiler A is that it can
often be written quite simply (and, indeed, might not be a real
compiler at all, just an interpreter), and does not need, for example,
to have any optimization phase at all. The only program it ever
needs to compile is B.

Even so, writing A is a fair bit of effort, and many language
implementers don't do that. Which makes things harder for distros,
but nobody cares about the poor distro developer.

--
Freedom-based blog/wiki/web hosting: http://www.branchable.com/
 
Old 12-13-2011, 08:21 PM
Stéphane Glondu
 
Default Binary blobs in source packages

Le 13/12/2011 21:30, Philipp Kern a écrit :
> But then I don't see how you could avoid circular build-dependencies
> with compilers written in their own language. fpc/fp-compiler does the
> same.

OCaml, F# (and Scala, it seems) do that by targetting a bytecode for
which there exists an independent interpreter (resp. OCaml's own
bytecode, .NET, Java). A (cross-platform) binary is then bundled with
the sources and is used for initial bootstrap.

In the OCaml case, a bytecode compiler (ocamlc) is bundled in the
upstream tarball, and the build system recompiles it until it reaches a
fixpoint. There are also native code backends for selected architectures
that can be compiled easily once everything has been compiled in bytecode.

--
Stéphane


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 4EE7C1EF.8060508@debian.org">http://lists.debian.org/4EE7C1EF.8060508@debian.org
 
Old 12-14-2011, 04:44 PM
Wookey
 
Default Binary blobs in source packages

+++ Steve Langasek [2011-12-13 10:45 -0800]:
> On Tue, Dec 13, 2011 at 07:29:23PM +0100, Mehdi Dogguy wrote:
> > > I think the traditional expectation here is that compilers will do
> > > their initial bootstrap using an out-of-archive binary, and that once
> > > in the archive, they'll be maintained using a self-build-depends
> > > instead.
>
> > You mean having a circular build-dependency?
>
> Yes.
>
> > That isn't great :/
>
> <shrug> It's how self-hosting compilers work. That's how the gcc package
> works, too.

This question is part of the general subject of 'making debian
automatically bootstrapable'.

Lots of languages have this properly of being written in themselves.
However they do generally have some way of getting themselves
bootstrapped.

Languages are generally a degenerate case of circular build-deps with
a very short loop (i.e just on themselves).

My prefered way to deal with the circular dependency is for at least
one package in a loop to have a 'bootstrap-build' supported in the
rules file which does not have a circular dependency (defined by
Build-Depends-Stage1 in control). So a tool which bootstraps debian
from scratch understands to build this 'not the real package'
bootstrap version first, then go on to build the full version once
dependencies are satisfied. The gcc packaging already implements
exactly this, although not using the exact machinery proposed in
http://wiki.debian.org/DebianBootstrap (That doc is out of date,
better info in my debconf talk
http://wookware.org/talks/bootstrappable-Debconf11.tar.gz )

For some packages this might mean that a $language-minimal package is
really needed to avoid the circular build-dep.

I anyone is aware of packages where it really isn't possible to do an
automatic bootstrap without a circular dependency (for the initial
bootstrap build), I would like to know about it.

Wookey
--
Principal hats: Linaro, Emdebian, Wookware, Balloonboard, ARM
http://wookware.org/


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20111214174425.GM28838@dream.aleph1.co.uk">http://lists.debian.org/20111214174425.GM28838@dream.aleph1.co.uk
 
Old 12-15-2011, 04:03 AM
Thomas Goirand
 
Default Binary blobs in source packages

On 12/15/2011 01:44 AM, Wookey wrote:
> defined by
> Build-Depends-Stage1 in control
Hi,

I watched the debconf11 video about bootstraping video (yes,
I in Banja Luka, but regrettably didn't attend this one).
My understanding of it was that last summer, there was no

Build-Depends-Stage1

defined in the policy. Has this changed? How does it work?
Where is this documented?

Thomas


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 4EE97FBB.8050909@debian.org">http://lists.debian.org/4EE97FBB.8050909@debian.org
 
Old 12-15-2011, 08:09 AM
Joachim Breitner
 
Default Binary blobs in source packages

Hi,

Am Mittwoch, den 14.12.2011, 17:44 +0000 schrieb Wookey:
> I anyone is aware of packages where it really isn't possible to do an
> automatic bootstrap without a circular dependency (for the initial
> bootstrap build), I would like to know about it.

again, GHC comes to mind. When I ported it to s390x, I had to use an old
version (6.4.2), find some 64bit patches that were used by someone to
port it to powerpc64 (https://slyfox.ath.cx/ghc/patches/6.4.2/) and
built it partly on s390, copied the generated C files, and then finished
the build on s390x. Then I used that to build 6.10.1, working around a
bug by manually building one file of the compiler without optimization.
And then I build 7.0.4 (the current version) using that file.

Certainly not something that is worth automating...

If you get the impression that upstream does not care too much about
exotic architectures – then you are right. Unfortunately, that is
nothing that we can change easily.

Greetings,
Joachim

--
Joachim "nomeata" Breitner
Debian Developer
nomeata@debian.org | ICQ# 74513189 | GPG-Keyid: 4743206C
JID: nomeata@joachim-breitner.de | http://people.debian.org/~nomeata
 
Old 12-16-2011, 09:08 AM
Goswin von Brederlow
 
Default Binary blobs in source packages

Thomas Koch <thomas@koch.ro> writes:

> Hi,
>
> I just made a fool of myself on the simple-build-tool list by claiming that
> Debian would build scala without scala. I only checked debian/rules and
> debian/control and since scala is in main, I assumed that I must be right.
>
> However scala comes with a bytecode-compiled scala compiler in lib/ which is
> required only during build but not installed in the binary packages.
>
> I already opened reportbug to fill a serious Debian Policy violation, but
> actually I couldn't find anything in the policy about it.
>
> So is it ok to ship binaries in the source package that are only required
> during build? Can I do the same with simple-build-tool, which requires itself
> to build?
>
> Regards,
>
> Thomas Koch, http://www.koch.ro

What you could do is use the existing previous version of scala to
build the new one and thereby avoid any precompiled binaries.

That has the obvious disadvantage that if the previous scala is broken
you need to manually build and upload a fixed one.


I don't think though that there is anything legally or morally wrong
with shipping and using a binary blob for bootstrapping where
neccessary. Important is that you include the sources for the blob and
that you can build the blob from scratch, which means you testing that
regulary before uploads.

A nice test is to build the blob and package repeadetly and check if the
result remains the same. Build compiler1 with upstream blob, build blob1
with compiler1, build compiler2 with blob1, build blob2 with compiler2.
Now blob1 and blob2 should be identical (other than timestamps and
such).

MfG
Goswin


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 87d3bo6d8f.fsf@frosties.localnet">http://lists.debian.org/87d3bo6d8f.fsf@frosties.localnet
 

Thread Tools




All times are GMT. The time now is 10:13 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright ©2007 - 2008, www.linux-archive.org