On Mon, Apr 23, 2012 at 10:43:18AM -0400, Michael Orlitzky wrote:
> On 04/23/12 09:21, napalm@squareownz.org wrote:
> > I'm unsure if I should be posting this to the -hardened mailing list as
> > I'm using the hardened profile but all of a sudden I'm getting a rather
> > strange error when trying to start postgres.
> >
> > # /etc/init.d/postgresql-9.1 start
> > * Caching service dependencies ... [ ok ]
> > * The following file(s) are not readable by 'postgres':
> > * /etc/postgresql-9.1/postgresql.conf
> > * /etc/postgresql-9.1/pg_ident.conf
> > * /etc/postgresql-9.1/pg_hba.conf
> > * HINT: Try: 'chmod 644 /etc/postgresql-9.1/*.conf'
> > * ERROR: postgresql-9.1 failed to start
> >
> > That's what I'm getting when I attempt to start it and I don't seem to have modified anything.
> >
> > Looking into the init script I can see it's doing su postgres -c "test -r /etc/postgresql-9.1/pg_hba.conf" and the like but the output of:
> > su postgres -c "test -r /etc/postgresql-9.1/pg_hba.conf" || echo "fail"
> > is fail... so I'm quite at a loss as to what could be going on here. All of the files are owned by postgres, have the correct permissions (I ran chmod 644 as it hinted) and it should be able to traverse to the directory as everything has the execute bit from /etc onwards.
> >
>
> You can `su postgres` and then try to read the files yourself. That
> should reveal the problem.
>

Interestingly I can't `su postgres` even though I have set it a shell,
thanks for the tip though, I'm getting closer!

On 04/23/12 11:44, napalm@squareownz.org wrote:
>>
>> You can `su postgres` and then try to read the files yourself. That
>> should reveal the problem.
>>
>
> Interestingly I can't `su postgres` even though I have set it a shell,
> thanks for the tip though, I'm getting closer!

You can also try `su - postgres` which will attempt to switch to
postgres's home directory. Is that readable/executable?

On Mon, Apr 23, 2012 at 11:56:47AM -0400, Michael Orlitzky wrote:
> On 04/23/12 11:44, napalm@squareownz.org wrote:
> >>
> >> You can `su postgres` and then try to read the files yourself. That
> >> should reveal the problem.
> >>
> >
> > Interestingly I can't `su postgres` even though I have set it a shell,
> > thanks for the tip though, I'm getting closer!
>
> You can also try `su - postgres` which will attempt to switch to
> postgres's home directory. Is that readable/executable?
>

Postgres doesn't have a home directory and if I create one and chown it
postgresostgres I still can't do anything. I'm totally at a loss here.

Here's the strace output from `su - postgres`:
http://pastie.org/private/dilrgts7xqrafxu15widga

I'm so confused at the moment!
Thanks for the help so far though guys.

napalm@squareownz.org writes:

> Postgres doesn't have a home directory and if I create one and chown it
> postgresostgres I still can't do anything. I'm totally at a loss here.

Postgres should have a home directory - /var/lib/postgresql

If you run su - postgres, this is the directory you should be in.

On Tue, Apr 24, 2012 at 06:27:22AM +0100, Graham Murray wrote:
> napalm@squareownz.org writes:
>
> > Postgres doesn't have a home directory and if I create one and chown it
> > postgresostgres I still can't do anything. I'm totally at a loss here.
>
> Postgres should have a home directory - /var/lib/postgresql
>
> If you run su - postgres, this is the directory you should be in.
>
Ah okay, I've changed it back to this then. I do indeed enter there now.
I've emerged it without threads or pam as I suspected it may have been
either of them but that hasn't seemed to solve much of anything.

The current error I'm getting is:
# /etc/init.d/postgresql-9.1 start
* Starting PostgreSQL ...
* start-stop-daemon: did not create a valid pid in
* `/var/lib/postgresql/9.1/data/postmaster.pid'
* Check the PostgreSQL 9.1 log for a detailed explanation of the
* above error. [ !! ]

Which is what happens when start-stop-daemon fails to execute its
command. I'm not entirely sure what start-stop-daemon is or what
permissions it may need or be missing so I'm about to look into that.

I seem to be getting somewhere at least.

Thanks again for the help!

On 04/24/12 05:31, napalm@squareownz.org wrote:
> On Tue, Apr 24, 2012 at 06:27:22AM +0100, Graham Murray wrote:
>> napalm@squareownz.org writes:
>>
>>> Postgres doesn't have a home directory and if I create one and chown it
>>> postgresostgres I still can't do anything. I'm totally at a loss here.
>>
>> Postgres should have a home directory - /var/lib/postgresql
>>
>> If you run su - postgres, this is the directory you should be in.
>>
> Ah okay, I've changed it back to this then. I do indeed enter there now.
> I've emerged it without threads or pam as I suspected it may have been
> either of them but that hasn't seemed to solve much of anything.
>
> The current error I'm getting is:
> # /etc/init.d/postgresql-9.1 start
> * Starting PostgreSQL ...
> * start-stop-daemon: did not create a valid pid in
> * `/var/lib/postgresql/9.1/data/postmaster.pid'
> * Check the PostgreSQL 9.1 log for a detailed explanation of the
> * above error. [ !! ]
>
> Which is what happens when start-stop-daemon fails to execute its
> command. I'm not entirely sure what start-stop-daemon is or what
> permissions it may need or be missing so I'm about to look into that.
>

Is everything under (and including) /var/lib/postgresql owned by
postgresostgres?

~ # ls /var/lib/postgresql/9.1
total 4.0K
drwx------ 13 postgres postgres 4.0K 2012-04-23 18:58 data

~ # ls /var/lib/postgresql/9.1/data/
total 1.2M
-rw------- 1 postgres postgres 4 2012-02-14 00:14 PG_VERSION
drwx------ 7 postgres postgres 4.0K 2012-04-23 10:31 base
drwx------ 2 postgres postgres 4.0K 2012-04-23 18:59 global
drwx------ 2 postgres postgres 4.0K 2012-02-14 00:14 pg_clog
drwx------ 4 postgres postgres 4.0K 2012-02-14 00:14 pg_multixact
drwx------ 2 postgres postgres 4.0K 2012-04-23 18:58 pg_notify
drwx------ 2 postgres postgres 4.0K 2012-02-14 00:14 pg_serial
drwx------ 2 postgres postgres 4.0K 2012-04-24 09:57 pg_stat_tmp
drwx------ 2 postgres postgres 4.0K 2012-04-20 13:42 pg_subtrans
drwx------ 2 postgres postgres 4.0K 2012-02-14 00:14 pg_tblspc
drwx------ 2 postgres postgres 4.0K 2012-02-14 00:14 pg_twophase
drwx------ 3 postgres postgres 4.0K 2012-04-23 11:36 pg_xlog
-rw------- 1 postgres postgres 1.1M 2012-04-24 09:55 postmaster.log
-rw------- 1 postgres postgres 134 2012-04-23 18:58 postmaster.opts
-rw------- 1 postgres postgres 92 2012-04-23 18:58 postmaster.pid

On 23 April 2012, at 22:18, napalm@squareownz.org wrote:
> …
> and if I create one and chown it
> postgresostgres I still can't do anything. I'm totally at a loss here.

If you want to say something like "Postgres doesn't have a home directory", please post the output of `grep -i postgres /etc/passwd`. Maybe also the output of `echo ~postgres`.

Right now I assume you're looking in /home and not seeing /home/postgres. I assume you're running `mkdir /home/postgres`. This is not how home directories are created and allocated.

Stroller.

On Mon, April 23, 2012 3:21 pm, napalm@squareownz.org wrote:
> I'm unsure if I should be posting this to the -hardened mailing list as
> I'm using the hardened profile but all of a sudden I'm getting a rather
> strange error when trying to start postgres.
>
> # /etc/init.d/postgresql-9.1 start
> * Caching service dependencies ... [
> ok ]
> * The following file(s) are not readable by 'postgres':
> * /etc/postgresql-9.1/postgresql.conf
> * /etc/postgresql-9.1/pg_ident.conf
> * /etc/postgresql-9.1/pg_hba.conf
> * HINT: Try: 'chmod 644 /etc/postgresql-9.1/*.conf'
> * ERROR: postgresql-9.1 failed to start
>
> That's what I'm getting when I attempt to start it and I don't seem to
> have modified anything.
>
> Looking into the init script I can see it's doing su postgres -c "test -r
> /etc/postgresql-9.1/pg_hba.conf" and the like but the output of:
> su postgres -c "test -r /etc/postgresql-9.1/pg_hba.conf" || echo "fail"
> is fail... so I'm quite at a loss as to what could be going on here. All
> of the files are owned by postgres, have the correct permissions (I ran
> chmod 644 as it hinted) and it should be able to traverse to the directory
> as everything has the execute bit from /etc onwards.
>
> Any tips?

I don't have much experience with Hardenened, but are you certain that any
permissions (including ACLs) are set correctly for PostgreSQL to access
all its files?

Do you have "sec-policy/selinux-postgresql" installed? And did you
re-emerge this after the update?

--
Joost

On Thu, Apr 26, 2012 at 07:46:10AM +0200, J. Roeleveld wrote:
> On Mon, April 23, 2012 3:21 pm, napalm@squareownz.org wrote:
> > I'm unsure if I should be posting this to the -hardened mailing list as
> > I'm using the hardened profile but all of a sudden I'm getting a rather
> > strange error when trying to start postgres.
> >
> > # /etc/init.d/postgresql-9.1 start
> > * Caching service dependencies ... [
> > ok ]
> > * The following file(s) are not readable by 'postgres':
> > * /etc/postgresql-9.1/postgresql.conf
> > * /etc/postgresql-9.1/pg_ident.conf
> > * /etc/postgresql-9.1/pg_hba.conf
> > * HINT: Try: 'chmod 644 /etc/postgresql-9.1/*.conf'
> > * ERROR: postgresql-9.1 failed to start
> >
> > That's what I'm getting when I attempt to start it and I don't seem to
> > have modified anything.
> >
> > Looking into the init script I can see it's doing su postgres -c "test -r
> > /etc/postgresql-9.1/pg_hba.conf" and the like but the output of:
> > su postgres -c "test -r /etc/postgresql-9.1/pg_hba.conf" || echo "fail"
> > is fail... so I'm quite at a loss as to what could be going on here. All
> > of the files are owned by postgres, have the correct permissions (I ran
> > chmod 644 as it hinted) and it should be able to traverse to the directory
> > as everything has the execute bit from /etc onwards.
> >
> > Any tips?
>
> I don't have much experience with Hardenened, but are you certain that any
> permissions (including ACLs) are set correctly for PostgreSQL to access
> all its files?
>
> Do you have "sec-policy/selinux-postgresql" installed? And did you
> re-emerge this after the update?
>
> --
> Joost
>
I got things working in the end by deleting everything to do with
postgres, re-emerging and then restoring from a backup (it's fine
because the database is only updated a few times a day).

Still totally confused as to what the issue was. I hadn't been fiddling
with permissions or anything at all, didn't even go near the postgres
config files and there was no update to postgres so I'm just at a loss.

I don't have sec-policy/selinux-postgresql installed, more using PaX and
GRSecurity than selinux on my current installation, doubt that would
have helped.

I'm a bit annoyed that I couldn't solve the issue without doing the sort
of "turn it off and on" approach but it has done the trick so I guess
that's that.

I must have messed something up somewhere. Any guess as to if PAM or a
glibc update could have broken it? I wouldn't have thought glibc but I'm
a little clueless when it comes to PAM, then again I tried emerging
(without deleting everything) with USE="-pam" to no avail.

Anyway thanks for the help everyone, sorry I can't give a better
diagnosis. I did check strace logs and everything, couldn't locate the
error. Blargh!

Cheers,
David