Server crashed

For some reason my server took a dump, I tried to shut it down to reboot it but, it went into a continues cycle, and wouldn't shut down, so I turned it off. When I restarted it it said

An error occurred during file system check.
Give root password for maintenance
or Control D to reboot.

If I give my password it takes me to a command shell. But I don't know what to do from there. If I do a Control D it just reboots, and it all starts over.

Is there a command I can use to repair the File system from the command shell, Or can I try to run an repair off a Installation DVD, or even try an upgrade from 5.7 to 6.2?

Server is down, so I need help

Best Regards Michael

Status: 
Closed (fixed)

Comments

It sounds like there's a filesystem problem on your server, so you'd definitely want to run a filesystem check.

The prompt your seeing there is the single usermode prompt. After you enter your password, you can run:

fsck /dev/DEVICENAME

Where "DEVICENAME" is the name of the device containing your partitions.

So if you have a filesystem on /dev/sda1, then you'd type:

fsck /dev/sda1

Most people have multiple partitions containing filesystems, you'd want to run a fsck check on all of them.

To find out which devices are used in what part of the filesystem tree, these commands can be helpful:

mount
cat /etc/fstab
df -h

Thanks for the replys

OK well I had a RAID System set up. and had 2 250gb drives combined as 1 drive. Well one drive died, so there was no saving the system. Guess I will have to start all over.

I was in the process of trying to build a redundant server, but never got it built, money is tight. I'm not sure if I want to start this all over again, as it took me for ever to get everything up and running, and would never have been able to do it, if it wasn't for the Tech support here.

Setting up the DNS Server, and the Name Server, and also the Mail server. drove me nuts. So I'm not sure if I can do it again.

Michael

Mmh, okay, several points in your post strike me as odd, and I'd like to mention these:

You say you had a RAID system, one drive died, so the system could not be saved. Well, that would only be true if you used RAID-0 (striping). For a server though, it is highly recommended to use RAID-1 (mirroring). With RAID-1, when one drive dies, you simply replace it, perform a resync, and continue working as if nothing happened.

Of course also a RAID-1 does not provide 100% data safety. Both drives can die at the same time, or a software error can destroy the file system. Hence, it is mandatory to have a backup of the server, in case something goes very wrong.

So to sum up, you've been operating this server, which took you forever to set up, on a RAID-0 without backups? Mmh, okay. Seems you like the risky way of doing things. I personally would not have done that and certainly cannot recommend it. ;)

What exactly was it that took you ages to set up? Now that you know how to set it up, is it problematic to reproduce those steps? I find that setting up a fresh Virtualmin from scratch - following a checklist I made for myself - is a matter of maybe an hour.

Also, a hint, "DNS Server" and "Name Server" are the same thing. :)

You say you had a RAID system, one drive died, so the system could not be saved. Well, that would only be true if you used RAID-0 (striping).

OK you figured it out, I'm a Noob when it comes to Linux. Yes I did have it set to RAID-0 I had 2 250gig drives, and I wanted to use the 2 drives as 1 @ 500gig. I learned my lessen on that point when my Main computer running Windows 7 with the same configuration, and one drive dies. That's why I was trying to build a redundant server, so I could switch over to the new server, and remove the RAID configuration, so I wouldn't have this problem. I just didn't get it in time.

So to sum up, you've been operating this server, which took you forever to set up, on a RAID-0 without backups? Mmh, okay. Seems you like the risky way of doing things.

I have all the Sites backed up, and all the databases backed up, I just didn't have the server backed up. At the time it didn't seem risky, but you have to realize I have always run my sites on a Windows server. I was always told if you want your server to run a lot better, set up a Linux Server. so that's what I did, but then this being the first time I have ever set down in front of a Linux system much less Build a Server. I thought I did pretty good. Yes I made a lot of mistake, but that's how you learn. And I have learned a lot, as I have my server almost all the way back up, and it only took me a few hours. Not as fast as you, but then I have only been at it for a little over a year.

What exactly was it that took you ages to set up?

Getting the Bind DNS working, and the Mail Server took me a long time to get going, I just didn't understand the whole concept. Thanks to the Tech Support they helped me a lot in getting it working

Also, a hint, "DNS Server" and "Name Server" are the same thing. :)

OK Copy that. Thank you for telling me, I learned something new.

Now if were done with the ass chewing, can we get back to the problem?

Like I said I believe I have everything working again on a single 1.5TB drive. The problem is I can't get the Bind DNS Server to start, and I'm not sure what to look for. So any help would be great.

Thanks Michael

The problem is I can't get the Bind DNS Server to start, and I'm not sure what to look for. So any help would be great.

Do you receive an error message of some kind when launching BIND?

If not, take a peek in /var/log/messages, and see what errors and warnings show up in there.

Well, none of those appear to be fatal errors.

Is BIND definitely not running?

For example, what output do you get when running this command:

netstat -an | grep :53

My apologies, I shall stop any "ass chewing" right away.

Though you didn't mention before that you have your box back up on a new drive, and you also didn't say anything about current BIND problems, otherwise I'd surely have given advice.

Good luck anyway!

I get this:

[root@rrcs-74-87-143-202 ~]# netstat -an | grep :53 tcp 0 0 74.87.143.205:53 0.0.0.0:* LISTEN tcp 0 0 74.87.143.204:53 0.0.0.0:* LISTEN tcp 0 0 74.87.143.203:53 0.0.0.0:* LISTEN tcp 0 0 74.87.143.202:53 0.0.0.0:* LISTEN tcp 0 0 127.0.0.1:53 0.0.0.0:* LISTEN tcp 0 0 ::1:53 :::* LISTEN tcp 0 0 :::53917 :::* LISTEN udp 0 0 74.87.143.205:53 0.0.0.0:* udp 0 0 74.87.143.204:53 0.0.0.0:* udp 0 0 74.87.143.203:53 0.0.0.0:* udp 0 0 74.87.143.202:53 0.0.0.0:* udp 0 0 127.0.0.1:53 0.0.0.0:* udp 0 0 74.87.143.205:53 0.0.0.0:* udp 0 0 74.87.143.204:53 0.0.0.0:* udp 0 0 74.87.143.203:53 0.0.0.0:* udp 0 0 74.87.143.202:53 0.0.0.0:* udp 0 0 127.0.0.1:53 0.0.0.0:* udp 0 0 74.87.143.205:53 0.0.0.0:* udp 0 0 74.87.143.204:53 0.0.0.0:* udp 0 0 74.87.143.203:53 0.0.0.0:* udp 0 0 74.87.143.202:53 0.0.0.0:* udp 0 0 127.0.0.1:53 0.0.0.0:* udp 0 0 74.87.143.205:53 0.0.0.0:* udp 0 0 74.87.143.204:53 0.0.0.0:* udp 0 0 74.87.143.203:53 0.0.0.0:* udp 0 0 74.87.143.202:53 0.0.0.0:* udp 0 0 127.0.0.1:53 0.0.0.0:* udp 0 0 127.0.0.1:53 0.0.0.0:* udp 0 0 0.0.0.0:5353 0.0.0.0:* udp 0 0 ::1:53 :::*

Locutus I'm sorry if wee got off on the wrong foot, I realize you were only trying to help, and I appreciate your help. I was only trying to see if I could save my server. Like I said I am still new at this, because once the server is up and running, don't mess with it, so a lot of the stuff I learned I forget easily.

Anyway I apologize if I came off to harsh l meant no harm.

Michael

It looks like BIND is running correctly on your server.

What issue is it that you're having exactly?

I see 2 problems, in the attached picture

1 it lists the host name as " localhost.localdomain" it should be asrservice.com I believe

and in the status it show "BIND DNS Server" is not up. as it sits right now I can't FTP in to the main site, and the web page won't come up for

I don't have my site there yet, because I don't have ftp access. But it should come up with the default Apache web page.

Michael

Since it appears that Apache is configured to host lectraink.com, it would indeed show the error you're seeing if there's no web files uploaded.

As far as your hostname goes -- we typically recommend using an FQDN there -- something in the format "host.domain.tld".

If you run the command "hostname", what output do you receive?

Lastly -- what error do you receive when trying to FTP into your site?

with the command "hostname" I get this.

rrcs-74-87-143-202.west.biz.rr.com

When I try to ftp in, I get the message "you do not have access to this server"

OK I copied some file to the www.lectraink.com site, and it does log in. I used "Edit Website" in Virtualmin so that part works. But I still can't ftp in. I do have access to the database. Also I'm not picking up my e-mail from the mail server..

@mike: Alrighty, apology accepted. :)

About the FTP issues: First thing to do when running into trouble of any kind with Linux is always "check logs", then "check more logs", and then "google the error message". ;)

In your case, you might want to check if you have logs in "/var/log/proftpd" that show the reason for login error.

Same goes for email. First see what error message the client gives you, then check "/var/log/mail.log" (I hope those are named the same on your CentOS as on my Ubuntu).

About losing the server: My suggestion against problems like this for the future would be - aside from the RAID thing earlier - to use the Filesystem Backup feature of Webmin and make TGZ archives of the root filesystem from time to time. By doing so, should you lose the harddrive (can happen even with RAID), you can at least restore that and don't have to configure everything from scratch.

Thank you Locutus, I got the ftp working, and I can log in to the websites I have installed. Turns out it was a Firewall problem, I should have know, thank you for giving me that advice. Now I have to figure out the email thing.

Michael

Can you describe the email problem you're having?

Also, if you go into System Settings -> Re-Check Config, does it notice any problems?

However, you may want to set your hostname to an FQDN, in the format host.domain.tld.

You can do that in Webmin -> Networking -> Network Configuration -> Hostname and DNS.

-Eric

Hi Eric, well I tried changing the Hostname, but I'm really not sure what to change it to, I tried"west.biz.rr.com" but now I get an error

"Failed to save DNS configuration : Error while checking current Postfix configuration. Please manually fix Postfix configuration."

So now what I looked at the "Postfix configuration, I haven't a clue what to change. Don't know if I should post the content here or now.

Thanks for the idea anyway.

Michael

You want to hove a hostname that includes a domain you own.

So if you own the domain "example.com", you could use that for setting up your hostname -- but you also need to add a name in front of it in order to make it the "FQDN" format of "host.domain.tld".

Once you have a name to use, set that up, and then see if those errors you were getting go away.

If they don't, run a "postconf -n", and paste the output into here.

Hi all, just an update. It seemed the more I messes with it the worse it got. When I decided to do a new install I thought that the newest version would be best, so I went with CentOS 6.2 I watched the install, and noticed that it looked like all the files were x86_64. So I thought this would work better. But I was wrong.

Last night I dumped the 6.2 version, and went back to 5.7, that was what I was running. So I installed it and it went great, except I noticed there were a lot of i386 files mixed in with the x86_64 files Not sure why that is.

Anyway with in 2 hours I had Linux installed and set up, and Virtualmin installed and configured. and now DNS is running fine, I get my mail again, and ftp is working file, and all my sites are back up.

I do have a few question, and one problem I not sure how to fix, but I will start a new thread for it. You can make this one fixed.

Thanks for everyone's help

Regards Michael