FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Fedora Development

 
 
LinkBack Thread Tools
 
Old 12-15-2010, 11:47 PM
"Nick Bebout"
 
Default Outage: Fedora related services down 2010-12-15 22:00 UTC

Due to problems with some NFS in our PHX2 facility we are experiencing
diminished capacity to several of our services. We are working with
our provider and engineers on how to deal with this issue soon.

To convert UTC to your local time, take a look at
http://fedoraproject.org/wiki/Infrastructure/UTCHowto
or run:

date -d '2010-12-15 22:00 UTC'

Reason for outage:
NFS operations with filer are peaking below expected rates causing
hangs on NFS clients.

Affected Services:

BFO - http://boot.fedoraproject.org/
Buildsystem - http://koji.fedoraproject.org/
CVS / Source Control
Main Website - http://fedoraproject.org/
Mirror List - https://mirrors.fedoraproject.org/
Mirror Manager - https://admin.fedoraproject.org/mirrormanager/
Package Database - https://admin.fedoraproject.org/pkgdb/


Unaffected Services:

Bodhi - https://admin.fedoraproject.org/updates/
DNS - ns1.fedoraproject.org, ns2.fedoraproject.org
Docs - http://docs.fedoraproject.org/
Email system
Fedora Account System - https://admin.fedoraproject.org/accounts/
Fedora Community - https://admin.fedoraproject.org/community/
Fedora Hosted - https://fedorahosted.org/
Fedora People - http://fedorapeople.org/
Fedora Talk - http://talk.fedoraproject.org/
Smolt - http://smolts.org/
Spins - http://spins.fedoraproject.org/
Start - http://start.fedoraproject.org/
Torrent - http://torrent.fedoraproject.org/
Translation Services - http://translate.fedoraproject.org/
Wiki - http://fedoraproject.org/wiki/


Ticket Link:

https://fedorahosted.org/fedora-infrastructure/ticket/2516

Contact Information:

Please join #fedora-admin in irc.freenode.net or respond to this email to
track the status of this outage.

--
Nick Bebout
Fedora Project Infrastructure Team
nb@fedoraproject.org


_______________________________________________
devel-announce mailing list
devel-announce@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel-announce
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 12-17-2010, 10:38 PM
Stephen John Smoogen
 
Default Outage: Fedora related services down 2010-12-15 22:00 UTC

This outage is considered finished and closed at the moment.

Due to some change in network, DNS traffic was not being properly
moving between Fedora network and Red Hat filer. It appears that in
doing so the filer's NFS permission caches were being flooded with bad
lookups thus increasing CPU and lowering throughput.


On Wed, Dec 15, 2010 at 15:42, Stephen John Smoogen <smooge@gmail.com> wrote:
> Due to problems with some NFS in our PHX2 facility we are experiencing
> diminished capacity to several of our services. We are working with
> our provider and engineers on how to deal with this issue soon.
>
> To convert UTC to your local time, take a look at
> http://fedoraproject.org/wiki/Infrastructure/UTCHowto
> or run:
>
> date -d '2010-12-15 22:00 UTC'
>
> Reason for outage:
> NFS operations with filer are peaking below expected rates causing
> hangs on NFS clients.
>
> Affected Services:
>
> BFO - http://boot.fedoraproject.org/
> Buildsystem - http://koji.fedoraproject.org/
> CVS / Source Control
> Main Website - http://fedoraproject.org/
> Mirror List - https://mirrors.fedoraproject.org/
> Mirror Manager - https://admin.fedoraproject.org/mirrormanager/
> Package Database - https://admin.fedoraproject.org/pkgdb/
>
>
> Unaffected Services:
>
> Bodhi - https://admin.fedoraproject.org/updates/
> DNS - ns1.fedoraproject.org, ns2.fedoraproject.org
> Docs - http://docs.fedoraproject.org/
> Email system
> Fedora Account System - https://admin.fedoraproject.org/accounts/
> Fedora Community - https://admin.fedoraproject.org/community/
> Fedora Hosted - https://fedorahosted.org/
> Fedora People - http://fedorapeople.org/
> Fedora Talk - http://talk.fedoraproject.org/
> Smolt - http://smolts.org/
> Spins - http://spins.fedoraproject.org/
> Start - http://start.fedoraproject.org/
> Torrent - http://torrent.fedoraproject.org/
> Translation Services - http://translate.fedoraproject.org/
> Wiki - http://fedoraproject.org/wiki/
>
>
> Ticket Link:
>
> https://fedorahosted.org/fedora-infrastructure/ticket/2516
>
> Contact Information:
>
> Please join #fedora-admin in irc.freenode.net or respond to this email to
> track the status of this outage.
>
>
> --
> Stephen J Smoogen.
> "The core skill of innovators is error recovery, not failure avoidance."
> Randy Nelson, President of Pixar University.
> "Let us be kind, one to another, for most of us are fighting a hard
> battle." -- Ian MacLaren
>



--
Stephen J Smoogen.
"The core skill of innovators is error recovery, not failure avoidance."
Randy Nelson, President of Pixar University.
"Let us be kind, one to another, for most of us are fighting a hard
battle." -- Ian MacLaren
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 12-17-2010, 10:38 PM
Stephen John Smoogen
 
Default Outage: Fedora related services down 2010-12-15 22:00 UTC

This outage is considered finished and closed at the moment.

Due to some change in network, DNS traffic was not being properly
moving between Fedora network and Red Hat filer. It appears that in
doing so the filer's NFS permission caches were being flooded with bad
lookups thus increasing CPU and lowering throughput.


On Wed, Dec 15, 2010 at 15:42, Stephen John Smoogen <smooge@gmail.com> wrote:
> Due to problems with some NFS in our PHX2 facility we are experiencing
> diminished capacity to several of our services. We are working with
> our provider and engineers on how to deal with this issue soon.
>
> To convert UTC to your local time, take a look at
> http://fedoraproject.org/wiki/Infrastructure/UTCHowto
> or run:
>
> date -d '2010-12-15 22:00 UTC'
>
> Reason for outage:
> NFS operations with filer are peaking below expected rates causing
> hangs on NFS clients.
>
> Affected Services:
>
> BFO - http://boot.fedoraproject.org/
> Buildsystem - http://koji.fedoraproject.org/
> CVS / Source Control
> Main Website - http://fedoraproject.org/
> Mirror List - https://mirrors.fedoraproject.org/
> Mirror Manager - https://admin.fedoraproject.org/mirrormanager/
> Package Database - https://admin.fedoraproject.org/pkgdb/
>
>
> Unaffected Services:
>
> Bodhi - https://admin.fedoraproject.org/updates/
> DNS - ns1.fedoraproject.org, ns2.fedoraproject.org
> Docs - http://docs.fedoraproject.org/
> Email system
> Fedora Account System - https://admin.fedoraproject.org/accounts/
> Fedora Community - https://admin.fedoraproject.org/community/
> Fedora Hosted - https://fedorahosted.org/
> Fedora People - http://fedorapeople.org/
> Fedora Talk - http://talk.fedoraproject.org/
> Smolt - http://smolts.org/
> Spins - http://spins.fedoraproject.org/
> Start - http://start.fedoraproject.org/
> Torrent - http://torrent.fedoraproject.org/
> Translation Services - http://translate.fedoraproject.org/
> Wiki - http://fedoraproject.org/wiki/
>
>
> Ticket Link:
>
> https://fedorahosted.org/fedora-infrastructure/ticket/2516
>
> Contact Information:
>
> Please join #fedora-admin in irc.freenode.net or respond to this email to
> track the status of this outage.
>
>
> --
> Stephen J Smoogen.
> "The core skill of innovators is error recovery, not failure avoidance."
> Randy Nelson, President of Pixar University.
> "Let us be kind, one to another, for most of us are fighting a hard
> battle." -- Ian MacLaren
>



--
Stephen J Smoogen.
"The core skill of innovators is error recovery, not failure avoidance."
Randy Nelson, President of Pixar University.
"Let us be kind, one to another, for most of us are fighting a hard
battle." -- Ian MacLaren
_______________________________________________
infrastructure mailing list
infrastructure@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/infrastructure
 

Thread Tools




All times are GMT. The time now is 05:30 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org