My VPS just went down for ten minutes. Everything was unresponsive - no HTTP, email or SSH. I logged in to Virtuozzo and only iptables was up. I did a full restart and everything is okay now, but nothing seems obvious in the logs. I'm able to work out what is wrong if I have info, but I can't work out how to get the info.
/var/log/messages shows the following at the time of the outage (with the line before included as well):
[code]
Mar 22 16:01:15 vps sshd(pam_unix)[20330]: session closed for user ***
Mar 27 11:57:54 vps shutdown: shutting down for system halt
Mar 27 11:57:56 vps init: Switching to runlevel: 0
Mar 27 12:57:57 vps saslauthd[11277]: server_exit : master exited: 11277
Mar 27 11:57:57 vps saslauthd: saslauthd shutdown succeeded
Mar 27 11:57:58 vps sshd: sshd shutdown succeeded
Mar 27 11:57:58 vps postfix: Shutting down postfix:
Mar 27 11:57:59 vps postfix: succeeded
Mar 27 11:57:59 vps postfix: [
Mar 27 11:57:59 vps postfix:
Mar 27 11:57:59 vps rc: Stopping postfix: succeeded
Mar 27 11:58:00 vps postfix: postfix stop failed
Mar 27 11:58:00 vps dovecot: dovecot shutdown succeeded
Mar 27 11:58:01 vps psmon: psmon shutdown succeeded
Mar 27 11:58:01 vps spawn-php: Stopping spawn-php:
Mar 27 11:58:02 vps spawn-php: succeeded
Mar 27 11:58:02 vps spawn-php:
Mar 27 11:58:02 vps rc: Stopping spawn-php: succeeded
Mar 27 11:58:03 vps postfix: succeeded
Mar 27 11:58:03 vps svnserve: svnserve shutdown succeeded
Mar 27 11:58:07 vps mysqld: Stopping MySQL: succeeded
Mar 27 12:58:07 vps xinetd[9724]: Exiting...
Mar 27 11:58:08 vps xinetd: xinetd shutdown succeeded
Mar 27 11:58:08 vps crond: crond shutdown succeeded
Mar 27 12:58:08 vps kernel: Kernel logging (proc) stopped.
Mar 27 12:58:08 vps kernel: Kernel log daemon terminating.
Mar 27 11:58:10 vps syslog: klogd shutdown succeeded
Mar 27 12:58:10 vps exiting on signal 15
Mar 27 12:10:17 vps syslogd 1.4.1: restart.
Mar 27 12:10:17 vps syslog: syslogd startup succeeded
Mar 27 12:10:17 vps kernel: klogd 1.4.1, log source = /proc/kmsg started.
Mar 27 12:10:17 vps syslog: klogd startup succeeded
...
[code]
(I'm assuming the rogue 12:58 time is a timezone issue). dmesg is empty, bootlog just shows the same info but without the timestamps, and I can't think whatelse might have useful information. I'm not even sure whether 11:58 was everything shutting down by itself or whether that was the restart and it took a while to process (although 12 minutes seems far too long).
Any ideas?
Thanks.
Help needed debugging outage
Started by
IBBoard
, Mar 27 2010 12:31 PM
4 replies to this topic
#1
Posted 27 March 2010 - 12:31 PM
The more information you provide, the better answer the community can give.
*** Sign up at ASO with a 15% discount (coupon: saveme15%) or $5 discount (coupon: saveme$5) ***
(Valid on shared hosting and VPS)
*** Sign up at ASO with a 15% discount (coupon: saveme15%) or $5 discount (coupon: saveme$5) ***
(Valid on shared hosting and VPS)
#2 [ASO] Frank
Posted 27 March 2010 - 01:03 PM
We just performed a kernel update on the VPS nodes, which is likely why you had an outage. Sorry if it was at a bad time.
#3
Posted 27 March 2010 - 01:35 PM
Hi Frank, I was just coming on here to ask much the same question, as mine went down. Is there any chance of VPS users getting a little bit of notice of this in future - even 5mins - just in case someone phones and asks us what's up with their/our website? An e-mail would be ideal, but even a post here would be helpful …
fuzzylime: we know design
Save money when you sign up! Go here then use code giveme15 to get 15% off or giveme5 to get $5 off your order.
Save money when you sign up! Go here then use code giveme15 to get 15% off or giveme5 to get $5 off your order.
#4 [ASO] Frank
Posted 27 March 2010 - 05:12 PM
Actually, we hope to never have to reboot them for kernel updates again. This update was for installing ksplice on all the servers. (http://www.ksplice.com/)
I wasn't the one actually doing the updates, so I cant comment on the notification. We certainly apologize if it caused any problems.
I wasn't the one actually doing the updates, so I cant comment on the notification. We certainly apologize if it caused any problems.
#5
Posted 28 March 2010 - 01:41 PM
It wasn't terribly inconvenient, although it did seem like things went down and didn't come back up again. There wasn't anything in the logs that looked obvious, but the site went down and Virtuozzo seemed to let me restart services without obviously saying "server is being rebooted" or anything.
As David says, a bit of warning would be good
Installing KSplice sounds like it'll have been worth it, though.
Now to work out whether to ask about a move to a CentOS5 server over Easter or wait until the new server details are announced...
As David says, a bit of warning would be good
Now to work out whether to ask about a move to a CentOS5 server over Easter or wait until the new server details are announced...
The more information you provide, the better answer the community can give.
*** Sign up at ASO with a 15% discount (coupon: saveme15%) or $5 discount (coupon: saveme$5) ***
(Valid on shared hosting and VPS)
*** Sign up at ASO with a 15% discount (coupon: saveme15%) or $5 discount (coupon: saveme$5) ***
(Valid on shared hosting and VPS)
0 user(s) are reading this topic
0 members, 0 guests, 0 anonymous users
Sign In
Create Account









