FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Fedora Development

 
 
LinkBack Thread Tools
 
Old 04-25-2008, 09:51 AM
Michael Schwendt
 
Default New vegastrike for F-10: 500MB !! ??

On Fri, 25 Apr 2008 11:09:57 +0200 (CEST), Nicolas Mailhot wrote:

>
> Le Ven 25 avril 2008 10:47, Jakub Jelinek a crit :
>
> > For Live CD it is obviously out of the question, it wouldn't even fit
> > there, but even in the Everything trees 500MB of data for a game is
> > IMHO too much.
>
> Can't the data set be broken out in several packages ?

And then? Have several packages which create a weird dep-chain?
Or make it a pain for the user to hunt for such packages?

> > Remember many people mirror the whole Everything tree to do
> > local installs etc.
>
> That just means we need more granular repositories, with one
> repository devoted to big lumps of optional resources like game data,
> artwork (images/audio/video), artistic fonts, document templates,
> cliparts, etc

Last time that was discussed briefly [a few years ago], no solution was
found. While you can separate [game] run-time from [game] data and put the
huge data pkgs into a separate repository, you don't want to enable that
repository by default, because that would cause yum to download even more
metadata. The run-time portion would need to depend on a disabled external
repository and warn about missing data.

Putting all games into one "Games" repo may work for some time until a
game pkg builds a library that is used by a pkg in another repo. Mapping
the comps groups into a repository hierarchy leads to much more complex
inter-repository dependencies and the necessity to maintain additional
lists for the "package name -> repository" mapping (or reuse the RPM spec
Group). The same applies to creating separate repositories for
"mandatory", "recommended" and "optional". They will keep growing anyway.

> IMHO it's good to have more FLOSS stuff packaged and we should not
> limit our ecosystem - that would put Fedora to a huge competitive
> disadvantage WRT entities with no such qualms.

The "Everything" repo is huge already. When will that be a reason to be
concerned?

--
Fedora release 8 (Werewolf) - Linux 2.6.23.15-137.fc8
loadavg: 1.11 1.21 1.33

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 04-25-2008, 11:41 AM
Andrew Farris
 
Default New vegastrike for F-10: 500MB !! ??

Thorsten Leemhuis wrote:

On 25.04.2008 11:09, Nicolas Mailhot wrote:

Le Ven 25 avril 2008 10:47, Jakub Jelinek a crit :


Remember many people mirror the whole Everything tree to do
local installs etc.

That just means we need more granular repositories, with one
repository devoted to big lumps of optional resources like game data,
artwork (images/audio/video), artistic fonts, document templates,
cliparts, etc


Just hardlinking the file between the different repos on the server
should IMHO solve most problems and avoids all the problems a separate
repo would bring.


I think the point was that people who have no need for it would be mirroring it
down to local repos. The disk storage space on servers is definitely a
consideration, but I don't think its the biggest issue with many large data
files like this; instead its the bandwidth they may incur for useless downloads
just because creating a local repo is easier than parsing out the files you want
and don't want.


--
Andrew Farris <lordmorgul@gmail.com> www.lordmorgul.net
gpg 0x8300BF29 fingerprint 071D FFE0 4CBC 13FC 7DEB 5BD5 5F89 8E1B 8300 BF29

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 04-25-2008, 01:15 PM
"Jon Ciesla"
 
Default New vegastrike for F-10: 500MB !! ??

> On 25.04.2008 11:09, Nicolas Mailhot wrote:
>> Le Ven 25 avril 2008 10:47, Jakub Jelinek a écrit :
>>
>>> Remember many people mirror the whole Everything tree to do
>>> local installs etc.
>> That just means we need more granular repositories, with one
>> repository devoted to big lumps of optional resources like game data,
>> artwork (images/audio/video), artistic fonts, document templates,
>> cliparts, etc
>
> Just hardlinking the file between the different repos on the server
> should IMHO solve most problems and avoids all the problems a separate
> repo would bring.

I agree. I think having separate program and data rpms is sufficient, as
long as the data rpm is noarch. 500MB is huge, but so what? That's
wonderful!

As far as local mirrors go, speaking as a local mirror maintainer, I have
no problem with a 500MB file rpm. I'd rather have that in my local mirror
than have all 3 machines with vegastrike on them pull them down
individually. Disk is cheap, bandwidth is expensive. At least where I
am.

And I agree that it needs to stay in rpm. I thought about what the
potential consequences of a non-rpm distribution of data files could be.
I nearly sent back my breakfast.

> CU
> thl
>
> --
> fedora-devel-list mailing list
> fedora-devel-list@redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-devel-list
>


--
novus ordo absurdum

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 04-25-2008, 02:44 PM
 
Default New vegastrike for F-10: 500MB !! ??

>> On Fri, 2008-04-25 at 08:59 +0200, Hans de Goede wrote:
>> >
>> > However the 0.5.0 datafiles are 500 Mb b2zipped! So is this a problem?
>>
> For Live CD it is obviously out of the question, it wouldn't even fit
> there, but even in the Everything trees 500MB of data for a game is IMHO
> too
> much. Remember many people mirror the whole Everything tree to do local
> installs etc.
>
> Jakub

Yeah, I ended up with a system that generated an exclude list for rsync
from a list of games that yum returned. That proved cumbersome (and there
were lots of other big packages I didn't want), so I moved to
InstantMirror, despite its problems. But this situation cries out for
proper InstantMirror implementation.

- Orion

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 04-25-2008, 03:20 PM
Jonathan Dieter
 
Default New vegastrike for F-10: 500MB !! ??

On Fri, 2008-04-25 at 08:59 +0200, Hans de Goede wrote:
> Hi All,
>
> I'm planning on updating vegastrike to the new 0.5.0 upstream release for F-10
> and maybe later F-9 / F-8 too, but vegastrike has been dead for a while and now
> makes some huge changes, so better to keep the old trusted version for F-8 /
> F-9 for a while.
>
> However the 0.5.0 datafiles are 500 Mb b2zipped! So is this a problem?

FWIW, I run a local mirror of Everything on a not-so-great link, and I
would still say to package this. I'd rather be forced to pull it down
than not have it at all.

On the deltarpm side of things, I do want to point out that compressed
files don't delta well *at all*. The problem is that a small change in
a file will lead to large changes in the compressed file. This is
biggest single reason I've seen that deltarpms won't be created for a
certain package in the presto test repository.

Again though, I say let's package it.

Jonathan
--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 04-25-2008, 03:56 PM
Kevin Kofler
 
Default New vegastrike for F-10: 500MB !! ??

Michael Schwendt <mschwendt <at> gmail.com> writes:
> Putting all games into one "Games" repo may work for some time until a
> game pkg builds a library that is used by a pkg in another repo.

That's not even just a hypothetical, we already had this situation in KDE 3
where kdeaddons(-extras) depended on kdegames because of the Atlantik Designer.
These days, only kdeaddons-atlantikdesigner is left of kdeaddons, so this could
presumably go to the games repo together with kdegames3 it depends on, but this
situation may occur again.

Kevin Kofler

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 04-26-2008, 12:41 AM
Matt Domsch
 
Default New vegastrike for F-10: 500MB !! ??

On Fri, Apr 25, 2008 at 08:59:33AM +0200, Hans de Goede wrote:
> Hi All,
>
> I'm planning on updating vegastrike to the new 0.5.0 upstream release for
> F-10 and maybe later F-9 / F-8 too, but vegastrike has been dead for a
> while and now makes some huge changes, so better to keep the old trusted
> version for F-8 / F-9 for a while.
>
> However the 0.5.0 datafiles are 500 Mb b2zipped! So is this a problem?

Note this is nearly twice the size of the largest files in the repo
today....

all > 60MB:

size date file

60138986 Apr 15 01:19 gnuradio-doc-3.1.1-4.fc9.x86_64.rpm
60151448 Apr 14 23:55 eclipse-pde-3.3.2-9.fc9.x86_64.rpm
60161418 Apr 15 08:20 scorched3d-41.3-2.fc9.x86_64.rpm
63817200 Apr 14 22:48 boswars-2.5-1.fc9.x86_64.rpm
64366160 Apr 14 23:58 egoboo-data-2.7.5-1.fc9.noarch.rpm
65302964 Apr 15 01:00 glest-data-3.1.2-1.fc9.noarch.rpm
67760204 Apr 15 02:56 libdap-doc-3.7.10-2.fc9.x86_64.rpm
69394021 Oct 26 21:55 beneath-a-steel-sky-cd-0.0372-2.fc8.noarch.rpm
76997044 Oct 26 21:46 astromenace-data-1.2-1.fc8.noarch.rpm
87876694 Apr 21 11:45 openoffice.org-core-2.4.0-12.6.fc9.x86_64.rpm
89890578 Apr 15 05:17 openclipart-0.18-1.fc9.noarch.rpm
101554175 Oct 27 04:16 tremulous-data-1.1.0-4.fc8.noarch.rpm
101779698 Apr 15 00:30 freedroidrpg-0.10.3-2.fc9.x86_64.rpm
103683033 Apr 15 00:24 fillets-ng-data-0.8.0-1.noarch.rpm
111786135 May 23 2007 uqm-content-voice-0.6.0-2.noarch.rpm
127392160 Apr 14 23:13 cloudy-docs-07.02.01-4.fc9.x86_64.rpm
130817667 Apr 14 16:23 vdrift-data-20071226-3.fc9.noarch.rpm
150797241 Apr 21 12:10 wesnoth-1.4.1-1.fc9.x86_64.rpm
157639375 Apr 14 16:01 vegastrike-data-0.4.3-5.fc9.noarch.rpm
193334077 Apr 15 09:06 texlive-texmf-doc-2007-20.fc9.noarch.rpm
212145315 Apr 15 00:22 fgfs-base-1.0.0-1.fc9.noarch.rpm
241912795 Apr 14 22:00 alienarena-data-20071011-2.fc9.noarch.rpm
267782755 Apr 15 02:20 kdelibs-apidocs-4.0.3-6.fc9.noarch.rpm
270532879 Apr 15 05:16 openarena-0.7.1-5.fc9.noarch.rpm
271998313 Oct 27 02:00 nexuiz-data-2.3-2.fc8.noarch.rpm

I'm not saying no, I'm just getting a little nervous. The F9 release will be about 120GB for the whole tree now.

--
Matt Domsch
Linux Technology Strategist, Dell Office of the CTO
linux.dell.com & www.dell.com/linux

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 04-26-2008, 12:57 AM
Matt Domsch
 
Default New vegastrike for F-10: 500MB !! ??

On Fri, Apr 25, 2008 at 11:27:01AM +0200, Rudolf Kastl wrote:
> On Fri, Apr 25, 2008 at 10:47 AM, Jakub Jelinek <jakub@redhat.com> wrote:
> > On Fri, Apr 25, 2008 at 05:31:26PM +1000, Rodd Clarkson wrote:
> > > On Fri, 2008-04-25 at 08:59 +0200, Hans de Goede wrote:
> > > > Hi All,
> > > >
> > > > I'm planning on updating vegastrike to the new 0.5.0 upstream release for F-10
> > > > and maybe later F-9 / F-8 too, but vegastrike has been dead for a while and now
> > > > makes some huge changes, so better to keep the old trusted version for F-8 /
> > > > F-9 for a while.
> > > >
> > > > However the 0.5.0 datafiles are 500 Mb b2zipped! So is this a problem?
> > >
> > > That depends. Are you wanting it installed on the Live CD? 8-]
> >
> > For Live CD it is obviously out of the question, it wouldn't even fit
> > there, but even in the Everything trees 500MB of data for a game is IMHO too
> > much. Remember many people mirror the whole Everything tree to do local
> > installs etc.
>
> but i really dont see the issue here, simply because you could just
> exclude that one package from mirroring. what if someone adds 100 x
> 5mb packages tomorrow... the everything mirror scripts will then have
> to mirror the same size.

yum users will get confused if mirrors start excluding by package name
(instead of by directory). MirrorManager expects that if you have a
directory, you have the _whole_ directory. yum metadata does to. yum
will request the file because the metadata says it's there; the mirror
will return a 404 which the user will see and then move to the next
entry in the mirrorlist; repeat... it'll work, but it'll be ugly.

--
Matt Domsch
Linux Technology Strategist, Dell Office of the CTO
linux.dell.com & www.dell.com/linux

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 04-26-2008, 03:16 AM
"Izhar Firdaus"
 
Default New vegastrike for F-10: 500MB !! ??

autodownloader?

--
Mohd Izhar Firdaus Bin Ismail
Amano Hikaru
天野晃 「あまの ひかる」
http://fedoraproject.org/wiki/MohdIzharFirdaus
http://blog.kagesenshi.org
92C2 B295 B40B B3DC 6866 5011 5BD2 584A 8A5D 7331

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 04-26-2008, 09:01 AM
"Pavel Shevchuk"
 
Default New vegastrike for F-10: 500MB !! ??

No, please no autodownloaders, they suck horribly and fail too often


On 4/26/08, Izhar Firdaus <kagesenshi.87@gmail.com> wrote:
> autodownloader?
>
>
> --
> Mohd Izhar Firdaus Bin Ismail
> Amano Hikaru
> 天野晃 「あまの ひかる」
> http://fedoraproject.org/wiki/MohdIzharFirdaus
> http://blog.kagesenshi.org
> 92C2 B295 B40B B3DC 6866 5011 5BD2 584A 8A5D 7331
>
>
> --
> fedora-devel-list mailing list
> fedora-devel-list@redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-devel-list
>


--
http://scwlab.com

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 

Thread Tools




All times are GMT. The time now is 11:57 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org