FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > CentOS > CentOS

 
 
LinkBack Thread Tools
 
Old 08-03-2012, 10:31 AM
Darod Zyree
 
Default Urgent help on replacing /var

2012/8/3 Emmanuel Noobadmin <centos.admin@gmail.com>:
> In a moment of epic stupidity, having ran out of space on the root
> partition of a server due to /var chewing up the space, I added a
> separate drive for the purpose of mounting it as /var
>
> To do so, I mounted the new drive as /var2, cp -R (in hindsight should
> had rsync to preserve attributes), deleted the original /var to free
> up space, edited fstab and rebooted... unsurprisingly to a fubar'd
> server.
>
> The thing is it still boots, I can get into single user mode but a
> full init get stuck at starting the syslogger. What is the best way to
> rescue the server now from my own stupidity and keeping all the
> existing configuration and data?
> _______________________________________________
> CentOS mailing list
> CentOS@centos.org
> http://lists.centos.org/mailman/listinfo/centos

Did you rewrite the selinux policy on /var or have you tried disabling
selinux if you haven't do so already?
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 08-03-2012, 10:52 AM
Emmanuel Noobadmin
 
Default Urgent help on replacing /var

On 8/3/12, Darod Zyree <darodzyree@gmail.com> wrote:
>
> Did you rewrite the selinux policy on /var or have you tried disabling
> selinux if you haven't do so already?

Thank you so much!

Turning off selinux allowed me get the system running.
However, after running fixfiles to restore the context for /var, I
still cannot boot to init 5, with the choke point now NFS statd.

I'll probably have to slowly hunt down the relevant selinux context
one by one when nobody's screaming about the server being down.
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 08-03-2012, 11:31 AM
Leonard den Ottolander
 
Default Urgent help on replacing /var

Hello Emmanuel,

On Fri, 2012-08-03 at 18:52 +0800, Emmanuel Noobadmin wrote:
> Turning off selinux allowed me get the system running.
> However, after running fixfiles to restore the context for /var, I
> still cannot boot to init 5, with the choke point now NFS statd.

If you copied a live /var over to the new partition you might have
issues from stale lock files that prevent services from
starting. /var/lock/subsys/nfsd comes to mind in this particular case.
If the service ain't running that file shouldn't be there.

I don't think stale pid files are as troublesome but it doesn't hurt to
have a look at /var/run as well.

Regards,
Leonard.

--
mount -t life -o ro /dev/dna /genetic/research


_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 08-03-2012, 11:41 AM
Karanbir Singh
 
Default Urgent help on replacing /var

On 08/03/2012 11:52 AM, Emmanuel Noobadmin wrote:
> I'll probably have to slowly hunt down the relevant selinux context
> one by one when nobody's screaming about the server being down.

Would restorecon not help get this bootrapped ? and then with selinux in
permissive mode, watch the audit log like a hawk.

- KB
--
Karanbir Singh
+44-207-0999389 | http://www.karan.org/ | twitter.com/kbsingh
ICQ: 2522219 | Yahoo IM: z00dax | Gtalk: z00dax
GnuPG Key : http://www.karan.org/publickey.asc
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 08-03-2012, 03:25 PM
Lamar Owen
 
Default Urgent help on replacing /var

On Friday, August 03, 2012 06:24:46 AM Emmanuel Noobadmin wrote:
> In a moment of epic stupidity, having ran out of space on the root
> partition of a server due to /var chewing up the space, I added a
> separate drive for the purpose of mounting it as /var
...

This sort of things pops up from time to time.... from a thread back in April.....

On Wednesday, April 11, 2012 05:38:13 PM Jason Pyeron wrote:
> > -----Original Message-----
> > From: centos-bounces@centos.org
> > [mailto:centos-bounces@centos.org] On Behalf Of Alexander Dalloz
> > Sent: Wednesday, April 11, 2012 17:12
> > To: CentOS mailing list
> > Subject: Re: [CentOS] How to fix a chown oops...
> >
> > Am 11.04.2012 23:02, schrieb Jason Pyeron:
> > > chown -R 7.0 /sbin/
> > > chown -R 98.98 //
> > >
> > > Is there a rpm way fix all the permissions of files managed by rpms?
> >
> > http://wiki.centos.org/TipsAndTricks/YumAndRPM#head-20a3ecce3d
> > 0762b9cdd3307ef2632e0c274a2bfd
>
> rpm -qa | while read line; do echo $line && rpm --setperms $line; done
>

...

By extension:


rpm -qa | while read line; do echo $line && rpm --setugids $line; done

should handle ownerships. Then, reenable selinux in permissive mode, and set it to relabel on the next boot.

_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 08-03-2012, 03:46 PM
John R Pierce
 
Default Urgent help on replacing /var

On 08/03/12 3:24 AM, Emmanuel Noobadmin wrote:
> In a moment of epic stupidity, having ran out of space on the root
> partition of a server due to /var chewing up the space, I added a
> separate drive for the purpose of mounting it as /var
>
> To do so, I mounted the new drive as /var2, cp -R (in hindsight should
> had rsync to preserve attributes), deleted the original /var to free
> up space, edited fstab and rebooted... unsurprisingly to a fubar'd
> server.
>
> The thing is it still boots, I can get into single user mode but a
> full init get stuck at starting the syslogger. What is the best way to
> rescue the server now from my own stupidity and keeping all the
> existing configuration and data?

if you had any database servers like postgresql or mysql, and their data
files were in the default locations under /var, your databases are
undoubtably corrupted, unless you stopped the DB server(s) before doing
this copy.

--
john r pierce N 37, W 122
santa cruz ca mid-left coast

_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 08-03-2012, 04:03 PM
Karanbir Singh
 
Default Urgent help on replacing /var

Hi,

On 08/03/2012 04:25 PM, Lamar Owen wrote:
> rpm -qa | while read line; do echo $line && rpm --setugids $line; done
> should handle ownerships. Then, reenable selinux in permissive mode, and set it to relabel on the next boot.

maybe add --setperms as well


--
Karanbir Singh
+44-207-0999389 | http://www.karan.org/ | twitter.com/kbsingh
ICQ: 2522219 | Yahoo IM: z00dax | Gtalk: z00dax
GnuPG Key : http://www.karan.org/publickey.asc
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 08-04-2012, 04:58 PM
Lamar Owen
 
Default Urgent help on replacing /var

On Friday, August 03, 2012 12:03:01 PM Karanbir Singh wrote:
> Hi,
>
> On 08/03/2012 04:25 PM, Lamar Owen wrote:
> > rpm -qa | while read line; do echo $line && rpm --setugids $line; done
> > should handle ownerships. Then, reenable selinux in permissive mode, and set it to relabel on the next boot.
>
> maybe add --setperms as well

Hmm, I thought by including that in my quoted text that it was implied that one should do both.... My bad. It was a long day....

I'm not sure if both can be used on a single command line, since both --setperms and --setugids are implemented via popt aliases; I reserve the right to be wrong, of course.

I also forgot to specify in my reply that these commands would only repair files owned by packages; any non-package-owned files in /var won't be helped by either --setperms or --setugids; but it will give the OP a good start.
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 08-05-2012, 04:00 AM
Emmanuel Noobadmin
 
Default Urgent help on replacing /var

On 8/3/12, Karanbir Singh <mail-lists@karan.org> wrote:
> On 08/03/2012 11:52 AM, Emmanuel Noobadmin wrote:
>> I'll probably have to slowly hunt down the relevant selinux context
>> one by one when nobody's screaming about the server being down.
>
> Would restorecon not help get this bootrapped ? and then with selinux in
> permissive mode, watch the audit log like a hawk.

fixfiles/restorecon managed to get init 5 past syslogger but it got
stuck still at NFS statd which locks up the entire server.

But with setenforce to permissive, the system appears to work fine and
yes I would be doing that watch the audit log thing during the next
scheduled down time.
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 08-05-2012, 04:03 AM
Emmanuel Noobadmin
 
Default Urgent help on replacing /var

On 8/3/12, Lamar Owen <lowen@pari.edu> wrote:
> On Friday, August 03, 2012 06:24:46 AM Emmanuel Noobadmin wrote:
>> In a moment of epic stupidity, having ran out of space on the root
>> partition of a server due to /var chewing up the space, I added a
>> separate drive for the purpose of mounting it as /var
> ...
>
> This sort of things pops up from time to time.... from a thread back in
> April.....

Lesson learnt, never try to fix things I'm not familiar with when
feeling pressured by relentless error messages, especially if nobody
else is complaining yet.

>> rpm -qa | while read line; do echo $line && rpm --setperms $line; done
>>

> By extension:
>
> rpm -qa | while read line; do echo $line && rpm --setugids $line; done
>
> should handle ownerships. Then, reenable selinux in permissive mode, and
> set it to relabel on the next boot.

Thanks for this tip, I'll try it and then see if there is anything
else in audit log that needs attention.
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 

Thread Tools




All times are GMT. The time now is 12:17 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org