FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian Development

 
 
LinkBack Thread Tools
 
Old 05-17-2010, 05:05 AM
Petter Reinholdtsen
 
Default APT do not work with Squid as a proxy because of pipelining default

I am bothered by <URL: http://bugs.debian.org/565555 >, and the fact
that apt(-get,itude) do not work with Squid as a proxy. I would very
much like to have apt work out of the box with Squid in Squeeze. To
fix it one can either change Squid to work with pipelining the way APT
uses, which the Squid maintainer and developers according to the BTS
report is unlikely to implement any time soon, or change the default
setting in apt for Aquire::http::Pipeline-Depth to zero (0). I've
added a file like this in /etc/apt/apt.conf.d/ to solve it locally:

Aquire::http::Pipeline-Depth 0;

My question to all of you is simple. Should the APT default be
changed or Squid be changed? Should the bug report be reassigned to
apt or stay as a bug with Squid?

Happy hacking,
--
Petter Reinholdtsen


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20100517050500.GC3633@login2.uio.no">http://lists.debian.org/20100517050500.GC3633@login2.uio.no
 
Old 05-17-2010, 07:02 AM
Goswin von Brederlow
 
Default APT do not work with Squid as a proxy because of pipelining default

Petter Reinholdtsen <pere@hungry.com> writes:

> I am bothered by <URL: http://bugs.debian.org/565555 >, and the fact
> that apt(-get,itude) do not work with Squid as a proxy. I would very
> much like to have apt work out of the box with Squid in Squeeze. To
> fix it one can either change Squid to work with pipelining the way APT
> uses, which the Squid maintainer and developers according to the BTS
> report is unlikely to implement any time soon, or change the default
> setting in apt for Aquire::http::Pipeline-Depth to zero (0). I've
> added a file like this in /etc/apt/apt.conf.d/ to solve it locally:
>
> Aquire::http::Pipeline-Depth 0;
>
> My question to all of you is simple. Should the APT default be
> changed or Squid be changed? Should the bug report be reassigned to
> apt or stay as a bug with Squid?
>
> Happy hacking,

Given that squid already has a patch, although only for newer versions,
this really seems to be a squid bug. As such it should be fixed in
squid as not only apt might trigger the problem.

That said setting the Pipeline-Depth to 0 as default or when a proxy is
configured might be advisable. Adding a apt.conf.d sniplet to the stable
apt should be a trivial change. Much simpler than fixing squid itself.

And in testing/unstable one can fix it properly or update squid to 3.0.

My 2c,
Goswin


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 87fx1r57tp.fsf@frosties.localdomain">http://lists.debian.org/87fx1r57tp.fsf@frosties.localdomain
 
Old 05-17-2010, 07:54 PM
Florian Weimer
 
Default APT do not work with Squid as a proxy because of pipelining default

* Petter Reinholdtsen:

> I am bothered by <URL: http://bugs.debian.org/565555 >, and the fact
> that apt(-get,itude) do not work with Squid as a proxy. I would very
> much like to have apt work out of the box with Squid in Squeeze. To
> fix it one can either change Squid to work with pipelining the way APT
> uses, which the Squid maintainer and developers according to the BTS
> report is unlikely to implement any time soon, or change the default
> setting in apt for Aquire::http::Pipeline-Depth to zero (0). I've
> added a file like this in /etc/apt/apt.conf.d/ to solve it locally:
>
> Aquire::http::Pipeline-Depth 0;

Maybe it's safe to use pipelining when a proxy is not used? This is
how things have been implemented in browsers, IIRC.

On the other hand, you probably still need to somewhat complex retry
logic, but I guess you need that anyway (if the first download fails,
try without pipelining etc.).


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 87pr0umhhn.fsf@mid.deneb.enyo.de">http://lists.debian.org/87pr0umhhn.fsf@mid.deneb.enyo.de
 
Old 05-18-2010, 12:25 AM
Marvin Renich
 
Default APT do not work with Squid as a proxy because of pipelining default

* Robert Collins <robertc@robertcollins.net> [100517 17:42]:
> Due to the widespread usage of intercepting proxies, its very hard, if
> not impossible, to determine if a proxy is in use. Its unwise, at
> best, to assume that no proxy configured == no proxy processing your
> traffic .
>
> -Rob

IANADD, but if I had filed bug #565555, I would have selected severity
critical ("makes unrelated software on the system break"), and similarly
for any other transparent proxy in Debian that fails to work
transparently.

The proxy may not be on a Debian system, but wouldn't the following
logic in apt catch enough of the problem cases to be a useful
workaround:

If Acquire::http::Pipeline-Depth is not set and Acquire::http::Proxy
is set, use 0 for Pipeline-Depth; use current behavior
otherwise.

Documenting this problem somewhere that an admin would look when seeing
the offending "Hash sum mismatch" message would also help. Turning off
pipelining by default for everybody seems like the wrong solution to
this problem.

...Marvin


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20100518002527.GK1461@cleo.wdw">http://lists.debian.org/20100518002527.GK1461@cleo.wdw
 
Old 05-18-2010, 05:32 AM
Frank Lin PIAT
 
Default APT do not work with Squid as a proxy because of pipelining default

On Tue, 2010-05-18 at 14:02 +1200, Robert Collins wrote:
> Given that pipelining is broken by design, that the HTTP WG has
> increased the number of concurrent connections that are recommended,
> and removed the upper limit - no. I don't think that disabling
> pipelining hurts anyone - just use a couple more concurrent
> connections.

Lots of [new] users are using Debian in Non-Debian infrastructure, which
may use unpatched squid. They would get a bad initial perception of
Debian, if it wasn't working with standard setup.

My 2cents,

Franklin


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 1274160733.4972.94.camel@solid.paris.klabs.be">htt p://lists.debian.org/1274160733.4972.94.camel@solid.paris.klabs.be
 
Old 05-18-2010, 06:52 AM
Goswin von Brederlow
 
Default APT do not work with Squid as a proxy because of pipelining default

Marvin Renich <mrvn@renich.org> writes:

> * Robert Collins <robertc@robertcollins.net> [100517 17:42]:
>> Due to the widespread usage of intercepting proxies, its very hard, if
>> not impossible, to determine if a proxy is in use. Its unwise, at
>> best, to assume that no proxy configured == no proxy processing your
>> traffic .
>>
>> -Rob
>
> IANADD, but if I had filed bug #565555, I would have selected severity
> critical ("makes unrelated software on the system break"), and similarly
> for any other transparent proxy in Debian that fails to work
> transparently.
>
> The proxy may not be on a Debian system, but wouldn't the following
> logic in apt catch enough of the problem cases to be a useful
> workaround:
>
> If Acquire::http::Pipeline-Depth is not set and Acquire::http::Proxy
> is set, use 0 for Pipeline-Depth; use current behavior
> otherwise.
>
> Documenting this problem somewhere that an admin would look when seeing
> the offending "Hash sum mismatch" message would also help. Turning off
> pipelining by default for everybody seems like the wrong solution to
> this problem.
>
> ...Marvin

Maybe apt should check size and try to resume the download. I'm assuming
it gets the right header but then the data ends prematurely?

Could you try to capture a tcpdump of the actual traffic between apt and
the proxy?

MfG
Goswin


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 87eih9itvi.fsf@frosties.localdomain">http://lists.debian.org/87eih9itvi.fsf@frosties.localdomain
 
Old 05-18-2010, 12:02 PM
Marvin Renich
 
Default APT do not work with Squid as a proxy because of pipelining default

* Robert Collins <robertc@robertcollins.net> [100517 22:03]:
> Given that pipelining is broken by design, that the HTTP WG has
> increased the number of concurrent connections that are recommended,
> and removed the upper limit - no. I don't think that disabling
> pipelining hurts anyone - just use a couple more concurrent
> connections.
>
> -Rob

I was unaware that pipelining was considered "broken by design", so I
was trying to say that if there was an easy way for apt to choose
between pipelining and no pipelining (if it wasn't specifically set by
the admin) that would handle most of the cases, that was better than
disabling by default a feature that was beneficial to many.

If pipelining is considered broken, and concurrency is preferred, I'm
perfectly happy with that.

...Marvin


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20100518120244.GL1461@cleo.wdw">http://lists.debian.org/20100518120244.GL1461@cleo.wdw
 
Old 05-18-2010, 12:04 PM
Marvin Renich
 
Default APT do not work with Squid as a proxy because of pipelining default

* Goswin von Brederlow <goswin-v-b@web.de> [100518 02:53]:
> Marvin Renich <mrvn@renich.org> writes:
> > Documenting this problem somewhere that an admin would look when seeing
> > the offending "Hash sum mismatch" message would also help. Turning off
> > pipelining by default for everybody seems like the wrong solution to
> > this problem.
> >
> > ...Marvin
>
> Maybe apt should check size and try to resume the download. I'm assuming
> it gets the right header but then the data ends prematurely?
>
> Could you try to capture a tcpdump of the actual traffic between apt and
> the proxy?
>
> MfG
> Goswin

Fortunately, I am not behind a proxy, so I can't check this. :-)

...Marvin


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20100518120413.GM1461@cleo.wdw">http://lists.debian.org/20100518120413.GM1461@cleo.wdw
 
Old 05-18-2010, 12:09 PM
Mike Hommey
 
Default APT do not work with Squid as a proxy because of pipelining default

On Mon, May 17, 2010 at 09:54:28PM +0200, Florian Weimer wrote:
> * Petter Reinholdtsen:
>
> > I am bothered by <URL: http://bugs.debian.org/565555 >, and the fact
> > that apt(-get,itude) do not work with Squid as a proxy. I would very
> > much like to have apt work out of the box with Squid in Squeeze. To
> > fix it one can either change Squid to work with pipelining the way APT
> > uses, which the Squid maintainer and developers according to the BTS
> > report is unlikely to implement any time soon, or change the default
> > setting in apt for Aquire::http::Pipeline-Depth to zero (0). I've
> > added a file like this in /etc/apt/apt.conf.d/ to solve it locally:
> >
> > Aquire::http::Pipeline-Depth 0;
>
> Maybe it's safe to use pipelining when a proxy is not used? This is
> how things have been implemented in browsers, IIRC.

Mozilla browsers have had pipelining disabled for years, because
reality is that a whole lot of servers don't implement it properly if at
all.

Mike


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20100518120913.GA8700@glandium.org">http://lists.debian.org/20100518120913.GA8700@glandium.org
 
Old 05-18-2010, 12:11 PM
Luigi Gangitano
 
Default APT do not work with Squid as a proxy because of pipelining default

Il giorno 17/mag/2010, alle ore 09.02, Goswin von Brederlow ha scritto:
> Given that squid already has a patch, although only for newer versions,
> this really seems to be a squid bug. As such it should be fixed in
> squid as not only apt might trigger the problem.

Goswin, can you please point me to the patch you mention?

> That said setting the Pipeline-Depth to 0 as default or when a proxy is
> configured might be advisable. Adding a apt.conf.d sniplet to the stable
> apt should be a trivial change. Much simpler than fixing squid itself.
>
> And in testing/unstable one can fix it properly or update squid to 3.0.

I assume that squid3 is not affected by this bug, do you confirm this? If the patch you mentioned is related to squid3 a backport may or may not be feasible, but should try. :-)

Regards,

L

--
Luigi Gangitano -- <luigi@debian.org> -- <gangitano@lugroma3.org>
GPG: 1024D/924C0C26: 12F8 9C03 89D3 DB4A 9972 C24A F19B A618 924C 0C26


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 6BA9DD68-4527-4D44-A836-647352505831@debian.org">http://lists.debian.org/6BA9DD68-4527-4D44-A836-647352505831@debian.org
 

Thread Tools




All times are GMT. The time now is 09:40 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org