Linux Archive

Linux Archive (http://www.linux-archive.org/)
-   Gentoo User (http://www.linux-archive.org/gentoo-user/)
-   -   All of a sudden, no apache2. No clue why. (http://www.linux-archive.org/gentoo-user/405912-all-sudden-no-apache2-no-clue-why.html)

"Kevin O'Gorman" 07-29-2010 09:24 PM

All of a sudden, no apache2. No clue why.
 
As of today, my apache2 web server seems to refuse to start.* I've tried a system reboot, to no avail --
connections are refused on port 80.

In /etc/init.d it looks like this: if I try to start it, it says it's already started.* netstat says there's no listener

on port 80.* If I try to restart it, it cannot start a listener.* I'm really bummed.

Any ideas how to get this going again?

Here's a short look at what I was doing in /etc/init.d

treat init.d # ./apache2 start
** WARNING:* apache2 has already been started.treat init.d # ./apache2 restart
** Stopping apache2 ...httpd (no pid file) not running******************************************* *********** [ ok ]
** Starting apache2 ...(98)Address already in use: make_sock: could not bind to address 64.166.164.49:80
no listening sockets available, shutting downUnable to open logs********************************************** ******************** [ ok ]
treat init.d # ./apache2 start** WARNING:* apache2 has already been started.
treat init.d # netstat -l --inetActive Internet connections (only servers)
Proto Recv-Q Send-Q Local Address********** Foreign Address******** State***** tcp******* 0***** 0 *:printer************** *:********************* LISTEN****
tcp******* 0***** 0 *:2222***************** *:********************* LISTEN**** tcp******* 0***** 0 t:domain*************** *:********************* LISTEN****
tcp******* 0***** 0 treat.kosmanor.c:domain *:********************* LISTEN**** tcp******* 0***** 0 localhost:domain******* *:********************* LISTEN****
tcp******* 0***** 0 *:ipp****************** *:********************* LISTEN**** tcp******* 0***** 0 *:smtp***************** *:********************* LISTEN****
tcp******* 0***** 0 localhost:rndc********* *:********************* LISTEN**** netstat: no support for `AF INET (sctp)' on this system.
treat init.d # netstat -l --inet -nActive Internet connections (only servers)
Proto Recv-Q Send-Q Local Address********** Foreign Address******** State***** tcp******* 0***** 0 0.0.0.0:515************ 0.0.0.0:*************** LISTEN****
tcp******* 0***** 0 0.0.0.0:2222*********** 0.0.0.0:*************** LISTEN**** tcp******* 0***** 0 192.168.1.149:53******* 0.0.0.0:*************** LISTEN****
tcp******* 0***** 0 64.166.164.49:53******* 0.0.0.0:*************** LISTEN****
tcp******* 0***** 0 127.0.0.1:53*********** 0.0.0.0:*************** LISTEN**** tcp******* 0***** 0 0.0.0.0:631************ 0.0.0.0:*************** LISTEN****
tcp******* 0***** 0 0.0.0.0:25************* 0.0.0.0:*************** LISTEN**** tcp******* 0***** 0 127.0.0.1:953********** 0.0.0.0:*************** LISTEN****
netstat: no support for `AF INET (sctp)' on this system.treat init.d #


Any ideas?* What else could I look at?

--
Kevin O'Gorman, PhD

Tomas Krasnican 07-29-2010 10:12 PM

All of a sudden, no apache2. No clue why.
 
Kevin O'Gorman wrote:
> As of today, my apache2 web server seems to refuse to start. I've tried
> a system reboot, to no avail --
> connections are refused on port 80.

I think that apache will try to create listener on address:port, which have already created (because it is possibly defined that).
After this fail apache will die and you can't see any active listener on this port in netstat -l, thats correct.

Try to check for twice definition of the same listener.

Or, example, if you have listener for 0.0.0.0:80 and you trying to create listener for 1.2.3.4.80.

...or dubble include of the same configuration file? I don't know, if is it possible...

Regards,
Tomas Krasnican

Kyle Bader 07-29-2010 10:14 PM

All of a sudden, no apache2. No clue why.
 
> ** Starting apache2 ...
> (98)Address already in use: make_sock: could not bind to address
> 64.166.164.49:80
> no listening sockets available, shutting down
> Unable to open
> logs********************************************** ******************** [ ok
> ]

Make sure an interface is listening on that address:

ip a |grep 64.166.164.49

Check for bound processes:

lsof -i @64.166.164.49

If that fails I'd strace the startup manually.

--

Kyle

William Kenworthy 07-29-2010 11:27 PM

All of a sudden, no apache2. No clue why.
 
There is an "Unable to open logs" in there ... are you doing some fancy
remote logging that cant start? - I cant see anything that says its
specifically port 80 thats causing the problem, just no listening
sockets available.

syslog is udp port 514, and then there is ssl on 443.

BillK



On Thu, 2010-07-29 at 14:24 -0700, Kevin O'Gorman wrote:
> As of today, my apache2 web server seems to refuse to start. I've
> tried a system reboot, to no avail --
> connections are refused on port 80.
>
> In /etc/init.d it looks like this: if I try to start it, it says it's
> already started. netstat says there's no listener
> on port 80. If I try to restart it, it cannot start a listener. I'm
> really bummed.
>
> Any ideas how to get this going again?
>
> Here's a short look at what I was doing in /etc/init.d
>
> treat init.d # ./apache2 start
> * WARNING: apache2 has already been started.
> treat init.d # ./apache2 restart
> * Stopping apache2 ...
> httpd (no pid file) not running
> [ ok ]
> * Starting apache2 ...
> (98)Address already in use: make_sock: could not bind to address
> 64.166.164.49:80
> no listening sockets available, shutting down
> Unable to open logs
> [ ok ]

"Kevin O'Gorman" 07-29-2010 11:47 PM

All of a sudden, no apache2. No clue why.
 
On Thu, Jul 29, 2010 at 3:12 PM, Tomas Krasnican <krasko@krasko.sk> wrote:

Kevin O'Gorman wrote:

> As of today, my apache2 web server seems to refuse to start. *I've tried

> a system reboot, to no avail --

> connections are refused on port 80.



I think that apache will try to create listener on address:port, which have already created (because it is possibly defined that).

After this fail apache will die and you can't see any active listener on this port in netstat -l, thats correct.



Try to check for twice definition of the same listener.



Or, example, if you have listener for 0.0.0.0:80 and you trying to create listener for 1.2.3.4.80.



...or dubble include of the same configuration file? I don't know, if is it possible...



Regards,

Tomas Krasnican

Thanks, but...

Grepping for Listen, all I see is*

hexDirs.conf:Listen 64.166.164.49:80

hexDirs.conf:Listen localhost:80

And grepping for Includes, I find nothing suspicious or cyclic.

--
Kevin O'Gorman, PhD

"Kevin O'Gorman" 07-29-2010 11:55 PM

All of a sudden, no apache2. No clue why.
 
On Thu, Jul 29, 2010 at 3:14 PM, Kyle Bader <kyle.bader@gmail.com> wrote:

> ** Starting apache2 ...

> (98)Address already in use: make_sock: could not bind to address

> 64.166.164.49:80

> no listening sockets available, shutting down

> Unable to open

> logs********************************************** ******************** [ ok

> ]



Make sure an interface is listening on that address:



ip a |grep 64.166.164.49


treat apache2 # ip a | grep 64.166.164.49
*** inet 64.166.164.49/29 brd 64.166.164.55 scope global eth0
treat apache2 #
That's no listener.



Check for bound processes:



lsof -i @64.166.164.49
*treat apache2 # lsof -i @64.166.164.49
COMMAND* PID* USER** FD** TYPE DEVICE SIZE/OFF NODE NAME

named** 4725 named** 21u* IPv4** 8406***** 0t0* TCP treat.kosmanor.com:domain (LISTEN)
named** 4725 named* 513u* IPv4** 8405***** 0t0* UDP treat.kosmanor.com:domain
ntpd*** 5117** ntp** 18u* IPv4** 9045***** 0t0* UDP treat.kosmanor.com:ntp

firefox 7832 kevin** 27u* IPv4* 29522***** 0t0* TCP treat.kosmanor.com:57043->nuq04s01-in-f83.1e100.net:https (ESTABLISHED)
firefox 7832 kevin** 56u* IPv4* 29352***** 0t0* TCP treat.kosmanor.com:57034->nuq04s01-in-f83.1e100.net:https (ESTABLISHED)

firefox 7832 kevin** 58u* IPv4* 29453***** 0t0* TCP treat.kosmanor.com:54324->nuq04s01-in-f18.1e100.net:https (ESTABLISHED)
firefox 7832 kevin** 63u* IPv4* 29536***** 0t0* TCP treat.kosmanor.com:59436->nuq04s01-in-f102.1e100.net:http (ESTABLISHED)

firefox 7832 kevin** 66u* IPv4* 29538***** 0t0* TCP treat.kosmanor.com:56773->74.125.164.30:http (ESTABLISHED)
firefox 7832 kevin** 72u* IPv4* 29475***** 0t0* TCP treat.kosmanor.com:37415->mg201a.mail.vip.mud.yahoo.com:http (CLOSE_WAIT)

treat apache2 #

And that's a DNS listener, an NTP listener, and firefox as a client, not a listener.* Though it makes me want to track down 1e100.net and find out who they are.


I'll see about strace.*
*
If that fails I'd strace the startup manually.



--



Kyle



--
Kevin O'Gorman, PhD

"Kevin O'Gorman" 07-30-2010 03:26 AM

All of a sudden, no apache2. No clue why.
 
On Thu, 2010-07-29 at 14:24 -0700, Kevin O'Gorman wrote:

> As of today, my apache2 web server seems to refuse to start. *I've

> tried a system reboot, to no avail --

> connections are refused on port 80.

>

> In /etc/init.d it looks like this: if I try to start it, it says it's

> already started. *netstat says there's no listener

> on port 80. *If I try to restart it, it cannot start a listener. *I'm

> really bummed.

>

> Any ideas how to get this going again?

>

> Here's a short look at what I was doing in /etc/init.d

>

> treat init.d # ./apache2 start

> ** WARNING: *apache2 has already been started.

> treat init.d # ./apache2 restart

> ** Stopping apache2 ...

> httpd (no pid file) not running

> [ ok ]

> ** Starting apache2 ...

> (98)Address already in use: make_sock: could not bind to address

> 64.166.164.49:80

> no listening sockets available, shutting down

> Unable to open logs

> [ ok ]







On Thu, Jul 29, 2010 at 4:27 PM, William Kenworthy <billk@iinet.net.au> wrote:

There is an "Unable to open logs" in there ... are you doing some fancy


remote logging that cant start? - I cant see anything that says its

specifically port 80 thats causing the problem, just no listening

sockets available.


syslog is udp port 514, and then there is ssl on 443.


BillK




I'm doing nothing fancy, but I did have a nearly full root directory.* I flushed out some
old portage stuff an I'm back to 19 GB free.* I still get the same result messages during
a reboot.

It is true I modified the local apache config /etc/apache2/kosmanor/hexDirs.conf slightly just before.* I added a /HexData alias, like

a few others that I have.* I've attached the config file in case you can find any unintended
change.


Looking at the logs, there does not seem to be logging going on since July 16, and things
were working then, I thought.* List of log directory follows.* I do seem to have two different styles of log rotation going on simultaneously, but that seems to have been the case for

quite a while now; I'll find that later.*

treat apache2 # ls -ltotal 4368
-rw-rw-rw- 1 root root***** 0 Jul 16 03:10 access_log-rw-rw-rw- 1 root root 191438 Jun 15* 2009 access_log.1.gz
-rw-rw-rw- 1 root root 678879 Jun 25 03:10 access_log-20100625.gz-rw-rw-rw- 1 root root 693424 Jul* 2 03:10 access_log-20100702.gz
-rw-rw-rw- 1 root root 701307 Jul* 9 03:10 access_log-20100709.gz-rw-rw-rw- 1 root root 840730 Jul 16 03:10 access_log-20100716.gz
-rw-rw-rw- 1 root root 234663 Jun* 8* 2009 access_log.2.gz-rw-rw-rw- 1 root root 270349 Jun* 1* 2009 access_log.3.gz
-rw-rw-rw- 1 root root 277761 May 25* 2009 access_log.4.gz-rw-rw-rw- 1 root root***** 0 Jul 16 03:10 error_log
-rw-rw-rw- 1 root root 117611 Jun 15* 2009 error_log.1.gz.out-rw-rw-rw- 1 root root* 61608 Jun 25 03:10 error_log-20100625.gz
-rw-rw-rw- 1 root root* 69397 Jul* 2 03:10 error_log-20100702.gz-rw-rw-rw- 1 root root 118085 Jul* 9 03:10 error_log-20100709.gz
-rw-rw-rw- 1 root root 114433 Jul 16 03:10 error_log-20100716.gz-rw-rw-rw- 1 root root** 5706 Jun* 8* 2009 error_log.2.gz
-rw-rw-rw- 1 root root** 5628 Jun* 1* 2009 error_log.3.gz-rw-rw-rw- 1 root root** 6344 May 25* 2009 error_log.4.gz
-rw-rw-rw- 1 root root***** 0 Feb* 3* 2008 ssl_access_log-rw-rw-rw- 1 root root***** 0 Feb* 3* 2008 ssl_error_log
-rw-rw-rw- 1 root root***** 0 Feb* 3* 2008 ssl_request_logtreat apache2 # pwd
/var/log/apache2treat apache2 #


The ownership may seem odd, but agrees with backups.* That July 16 log ends with
something familiar:

[Fri Jul 16 03:10:07 2010] [notice] SIGUSR1 received.* Doing graceful restart
(98)Address already in use: make_sock: could not bind to address 0.0.0.0:80no listening sockets available, shutting down
Unable to open logstreat apache2 #

So I guess my page has been down for a while.* A nuisance at most because it's just my
personal stuff, which a handful of people care about.* But I want it back up, not to mention the logging.



--
Kevin O'Gorman, PhD


#
# Listen: Allows you to bind Apache to specific IP addresses and/or
# ports, instead of the default. See also the <VirtualHost>
# directive.
#
# Change this to Listen on specific IP addresses as shown below to
# prevent Apache from glomming onto all bound IP addresses (0.0.0.0)
#
#Listen 12.34.56.78:80
# Listen 80
# no need to listen promiscuously (I think)
Listen 64.166.164.49:80
Listen localhost:80

#
# If you wish httpd to run as a different user or group, you must run
# httpd as root initially and it will switch.
#
# User/Group: The name (or #number) of the user/group to run httpd as.
# . On SCO (ODT 3) use "User nouser" and "Group nogroup".
# . On HPUX you may not be able to use shared memory as nobody, and the
# suggested workaround is to create a user www and use that user.
# NOTE that some kernels refuse to setgid(Group) or semctl(IPC_SET)
# when the value of (unsigned)Group is above 60000;
# don't use Group #-1 on these systems!
#
User apache
Group apache

#
# ServerAdmin: Your address, where problems with the server should be
# e-mailed. This address appears on some server-generated pages, such
# as error documents. e.g. admin@your-domain.com
#
ServerAdmin webmaster@kosmanor.com

#
# ServerName gives the name and port that the server uses to identify itself.
# This can often be determined automatically, but we recommend you specify
# it explicitly to prevent problems during startup.
#
# If this is not set to valid DNS name for your host, server-generated
# redirections will not work. See also the UseCanonicalName directive.
#
# If your host doesn't have a registered DNS name, enter its IP address here.
# You will have to access it by its address anyway, and this will make
# redirections work in a sensible way.
#
ServerName www.kosmanor.com

#
# UseCanonicalName: Determines how Apache constructs self-referencing
# URLs and the SERVER_NAME and SERVER_PORT variables.
# When set "Off", Apache will use the Hostname and Port supplied
# by the client. When set "On", Apache will use the value of the
# ServerName directive.
#
UseCanonicalName Off


#
# Each directory to which Apache has access can be configured with respect
# to which services and features are allowed and/or disabled in that
# directory (and its subdirectories).
#
# First, we configure the "default" to be a very restrictive set of
# features.
#
<Directory />
Options FollowSymLinks
AllowOverride None
</Directory>

#
# Note that from this point forward you must specifically allow
# particular features to be enabled - so if something's not working as
# you might expect, make sure that you have specifically enabled it
# below.
#

#
# UserDir: The name of the directory that is appended onto a user's home
# directory if a ~user request is received.
# enable by adding -D USERDIR to /etc/conf.d/apache2
#
<IfModule mod_userdir.c>
UserDir public_html

#
# Control access to UserDir directories. The following is an example
# for a site where these directories are restricted to read-only.
#
<Directory /home/*/public_html>
AllowOverride FileInfo AuthConfig Limit Indexes
Options MultiViews Indexes SymLinksIfOwnerMatch IncludesNoExec
<Limit GET POST OPTIONS PROPFIND>
Order allow,deny
Allow from all
</Limit>
<LimitExcept GET POST OPTIONS PROPFIND>
Order deny,allow
Deny from all
</LimitExcept>
</Directory>


# Enable this additional section if you would like to make use of a
# suexec-enabled cgi-bin directory on a per-user basis.
#
#<Directory /home/*/public_html/cgi-bin>
# Options ExecCGI
# SetHandler cgi-script
#</Directory>

</IfModule>


#
# DirectoryIndex: sets the file that Apache will serve if a directory
# is requested.
#
# The index.html.var file (a type-map) is used to deliver content-
# negotiated documents. The MultiViews Option can be used for the
# same purpose, but it is much slower.
#
DirectoryIndex index.html index.html.var

#
# AccessFileName: The name of the file to look for in each directory
# for additional configuration directives. See also the AllowOverride
# directive.
#
AccessFileName .htaccess

#
# The following lines prevent .htaccess and .htpasswd files from being
# viewed by Web clients.
#
<FilesMatch "^.ht">
Order allow,deny
Deny from all
</FilesMatch>


# override
ServerSignature Off

#
# Aliases: Add here as many aliases as you need (with no limit). The format is
# Alias fakename realname
#
# Note that if you include a trailing / on fakename then the server will
# require it to be present in the URL. So "/icons" isn't aliased in this
# example, only "/icons/". If the fakename is slash-terminated, then the
# realname must also be slash terminated, and if the fakename omits the
# trailing slash, the realname must also omit it.
#
# We include the /icons/ alias for FancyIndexed directory listings. If you
# do not use FancyIndexing, you may comment this out.
#

<Location />
Order allow,deny
Allow from all
</Location>

Alias /hex /hex/web
<Directory /hex/web>
Options FollowSymLinks
# charset most important for non 8859-1 sets.
AddCharset iso-8859-1 .da .en_US .C
AddCharset iso-8859-2 .pl
Order allow,deny
Allow from all
</Directory>

Alias /Oggie /Dogs/Oggie
<Directory /Oggie>
Order allow,deny
Allow from all
</Directory>

Alias /Gleni /Dogs/Gleni
<Directory /Gleni>
Order allow,deny
Allow from all
</Directory>

Alias /HexData /golem/golem
<Directory /HexData>
Order allow,deny
Allow from all
</Directory>


Alias /Cully /Dogs/Cully
<Directory /Cully>
Order allow,deny
Allow from all
</Directory>

Alias /theory /hex/theory
<Directory /hex/theory>
Options FollowSymLinks
AddCharset iso-8859-1 .da .en_US .C
AddCharset iso-8859-2 .pl
AuthUserFile /etc/apache2/kosmanor/passwords
AuthGroupFile /dev/null
AuthName "Hex Theory"
AuthType Basic
Require valid-user
Order allow,deny
Allow from all
</Directory>

ScriptAlias /hex-bin /hex/bin
<Directory /hex/bin>
Options FollowSymLinks
# AddHandler mod_python .py
# PythonHandler hexscript
# PythonDebug On
Order allow,deny
Allow from all
</Directory>


ScriptAlias /my-bin /hex/hexTest
<Directory /hex/hexTest>
AuthUserFile /etc/apache2/kosmanor/passwords
AuthGroupFile /dev/null
AuthName "OHex Advanced"
AuthType Basic
Require valid-user
Options FollowSymLinks
</Directory>


#
# ScriptAlias: This controls which directories contain server scripts.
# ScriptAliases are essentially the same as Aliases, except that
# documents in the realname directory are treated as applications and
# run by the server when requested rather than as documents sent to the client.
# The same rules about trailing "/" apply to ScriptAlias directives as to
# Alias.

#
# Redirect allows you to tell clients about documents which used to exist in
# your server's namespace, but do not anymore. This allows you to tell the
# clients where to look for the relocated document.
# Example:
# Redirect permanent /foo http://www.example.com/bar

Kyle Bader 07-30-2010 04:12 PM

All of a sudden, no apache2. No clue why.
 
>> > ** Starting apache2 ...
>> > (98)Address already in use: make_sock: could not bind to address
>> > 64.166.164.49:80
>> > no listening sockets available, shutting down
>> > Unable to open
>> > logs********************************************** ******************** [

Strace will probably reveal which log file can't be opened, something
like this will probably do the trick:

strace /path/to/apache2 -D <module list> -d /path/to/apache2dir

> And that's a DNS listener, an NTP listener, and firefox as a client, not a
> listener.* Though it makes me want to track down 1e100.net and find out who
> they are.

Google:

Domain Name: 1E100.NET
Registrar: MARKMONITOR INC.
Whois Server: whois.markmonitor.com
Referral URL: http://www.markmonitor.com
Name Server: NS1.GOOGLE.COM
Name Server: NS2.GOOGLE.COM
Name Server: NS3.GOOGLE.COM
Name Server: NS4.GOOGLE.COM

Registrant:
DNS Admin
Google Inc.
1600 Amphitheatre Parkway
Mountain View CA 94043
US
dns-admin@google.com +1.6502530000 Fax: +1.6506188571

--

Kyle

"Kevin O'Gorman" 07-30-2010 11:35 PM

All of a sudden, no apache2. No clue why.
 
On Fri, Jul 30, 2010 at 9:12 AM, Kyle Bader <kyle.bader@gmail.com> wrote:

>> > ** Starting apache2 ...

>> > (98)Address already in use: make_sock: could not bind to address

>> > 64.166.164.49:80

>> > no listening sockets available, shutting down

>> > Unable to open

>> > logs********************************************** ******************** [



Strace will probably reveal which log file can't be opened, something

like this will probably do the trick:



strace /path/to/apache2 -D <module list> -d /path/to/apache2dir



> And that's a DNS listener, an NTP listener, and firefox as a client, not a

> listener.* Though it makes me want to track down 1e100.net and find out who

> they are.



Google:



I should have known that.* 1e100 is a mathematical/programming synonym for google.
Amusing.

I do most of this in gmail, so those connections look normal.

I don't know what the usual module list is, so I guess I have to go trolling throught the

init.d scripts to figure it out, unless somebody knows a better way.
--
Kevin O'Gorman, PhD


All times are GMT. The time now is 03:12 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.