Problems building a 64-bit CentOS 6 image

Submitted by cruiskeen on Sun, 08/25/2013 - 18:40

After having spent more or less the whole day trying to do this, I guess I need some help. I am having a terrible time trying to build a 64-bit CentOS 6 image for Cloudmin, since there are no 64-bit images available from stock in Cloudmin.

It appears that it's only possible to install CentOS 6.4 on an empty system by building it using the IDE disk interface. If I use Virtio and attempt to boot the system from a DVD image, the CentOS system insists on booting from the empty hard disk, not from the DVD. I've attempted to go in and edit the config file for the server by inserting if=ide at the end of the cdrom image (in the .serv file in /etc/webmin/servers on the Cloudmin server) but the system still insists on booting from the hard drive.

I CAN build a system if I don't turn on the virtio driver, but that then cascades into a series of trial and error to later get the image to boot using virtio.

Ther'es also a small problem inthat CentOS installs itself mounting root on the UUID name for the disk, rather than on the normal device name (say, sda1) and then you need to go in and change that on the server after it is up and running.

I've read through a boatload of different issues on this, but it appears that the current Cloudmin release still will not let you boot from CDROM on an empty system if you use the Virtio driver. If I could get past that I could probably actually build myself a 64-bit image.

Status: 
Closed (fixed)

Comments

Submitted by cruiskeen on Sun, 08/25/2013 - 18:43

THe current .serv file on the cloudmin server looks like this:

kvm_bootdevice=d bw_limit= kvm_disksize=4096 directory= kvm_vga=1 kvm_swapfile= kvm_filesystem=/dev/storage_vol/centos64build_cloudmin_cruiskeenconsulting_com_img desc=initlal centos 64bit built from empty image and dvd id=1377472805198820 mac=02:54:00:69:E5:16 kvm_firewall= kvm_on=137037545658510 kvm_gw=69.4.98.193 kvm_cidr=27 kvm_filesystem_vg=storage_vol kvm_smp_cores=1 kvm_model=virtio port=10000 kvm_cdimage=/kvm/centos64build.cloudmin.cruiskeenconsulting.com.cd imagetemp=/var/cloudmin-images type=centos xen_wholedisk=0 mac_addresses=02:54:00:69:E5:16 ssh_user=root kvm_hostifaces=br0 ip=69.4.98.210 status=noping kvm_ip=69.4.98.210 missing=0 kvm_disktotal=8648345600 kvm_start=1 kvm_virtio=1 kvm_smp=2 group=root kvm_memory=4096 kvm_args= no_webmin_login=1 kvm_ioclass=3 module=server-manager kvm_net=tap,vlan=0,script=/kvm/centos64build.cloudmin.cruiskeenconsulting.com-eth0.sh kvm_filesystem_lv=centos64build_cloudmin_cruiskeenconsulting_com_img image=CentOS-6.4-install-dvd-iso host=centos64build.cloudmin.cruiskeenconsulting.com manager=kvm ssh_protocol=2 created=1377472805 ip_addresses=69.4.98.210 short_host=centos64buil...nsulting.com kvm_drives=file=/dev/storage_vol/centos64build_cloudmin_cruiskeenconsulting_com_img,media=disk,index=0,if=virtio,boot=on file=/kvm/centos64build.cloudmin.cruiskeenconsulting.com.cd,media=cdrom,index=2,if=ide ssh_nologin=1 imagelocal=1 kvm_vnc_passwd=agukcvp kvm_monitor_port=40006 kvm_dns=centos64build.cloudmin.cruiskeenconsulting.com. kvm_monitor=/kvm/centos64build.cloudmin.cruiskeenconsulting.com.monitor kvm_console=/kvm/centos64build.cloudmin.cruiskeenconsulting.com.console kvm_vnc_display=4 kvm=2110 statuslast= status_msg=Ping failed status_hostkey=0 statuslasttime=1377473716 statustime=1377473716 file=/etc/webmin/servers/1377472805198820.serv kvm_ip6= statushistory=down

Submitted by cruiskeen on Sun, 08/25/2013 - 18:44

hmm -- that got sorta mucked up

kvm_bootdevice=d
bw_limit=
kvm_disksize=4096
directory=
kvm_vga=1
kvm_swapfile=
kvm_filesystem=/dev/storage_vol/centos64build_cloudmin_cruiskeenconsulting_com_img
desc=initlal centos 64bit built from empty image and dvd
id=1377472805198820
mac=02:54:00:69:E5:16
kvm_firewall=
kvm_on=137037545658510
kvm_gw=69.4.98.193
kvm_cidr=27
kvm_filesystem_vg=storage_vol
kvm_smp_cores=1
kvm_model=virtio
port=10000
kvm_cdimage=/kvm/centos64build.cloudmin.cruiskeenconsulting.com.cd
imagetemp=/var/cloudmin-images
type=centos
xen_wholedisk=0
mac_addresses=02:54:00:69:E5:16
ssh_user=root
kvm_hostifaces=br0
ip=69.4.98.210
status=noping
kvm_ip=69.4.98.210
missing=0
kvm_disktotal=8648345600
kvm_start=1
kvm_virtio=1
kvm_smp=2
group=root
kvm_memory=4096
kvm_args=
no_webmin_login=1
kvm_ioclass=3
module=server-manager
kvm_net=tap,vlan=0,script=/kvm/centos64build.cloudmin.cruiskeenconsulting.com-eth0.sh
kvm_filesystem_lv=centos64build_cloudmin_cruiskeenconsulting_com_img
image=CentOS-6.4-install-dvd-iso
host=centos64build.cloudmin.cruiskeenconsulting.com
manager=kvm
ssh_protocol=2
created=1377472805
ip_addresses=69.4.98.210
short_host=centos64buil...nsulting.com
kvm_drives=file=/dev/storage_vol/centos64build_cloudmin_cruiskeenconsulting_com_img,media=disk,index=0,if=virtio,boot=on file=/kvm/centos64build.cloudmin.cruiskeenconsulting.com.cd,media=cdrom,index=2,if=ide
ssh_nologin=1
imagelocal=1
kvm_vnc_passwd=agukcvp
kvm_monitor_port=40006
kvm_dns=centos64build.cloudmin.cruiskeenconsulting.com.
kvm_monitor=/kvm/centos64build.cloudmin.cruiskeenconsulting.com.monitor
kvm_console=/kvm/centos64build.cloudmin.cruiskeenconsulting.com.console
kvm_vnc_display=4
kvm=2110
statuslast=
status_msg=Ping failed
status_hostkey=0
statuslasttime=1377473716
statustime=1377473716
file=/etc/webmin/servers/1377472805198820.serv
kvm_ip6=
statushistory=down

Submitted by JamieCameron on Sun, 08/25/2013 - 19:19

Did you trying configuring the VM to boot from CD at System State -> Change Boot Method?

If that doesn't help, one thing you should be able to do is create a VM using IDE disks, create an image of it, then create a new VM using VirtIO mode from that image. Cloudmin should fix up /etc/fstab and the GRUB config to use VirtIO devices during the VM creation process. Just make sure that you put the whole VM filesystem on a single partition and don't use LVM or RAID.

Submitted by cruiskeen on Sun, 08/25/2013 - 20:38

Yeah, it's definitely set to boot from the CD - but it doesn't do so if you set it to use virtio.

Also - if I do what you suggest (I tried it nine ways from sunday) the system won't boot - I believe because the kernel doesn't actually have any drivers for virtio loaded. Cloudmin does go in and change the device names but when you then boot the kernel can't find the device ssince it apparently only knows about /dev/sd? not the virtio devices.

THere's actually a large nunber of issues sprinkled around about people having problems with this, and it's not very clear to me that anyone has actually gotten it working. I've pretty much tried all the suggested workarounds but can't find a way to end up with a bootable 64-bit image that will use virtio for disk.

Submitted by cruiskeen on Sun, 08/25/2013 - 20:55

The heart of the issue here seems to be the same problem reported in http://virtualmin.com/node/22339 - if I could get the system to boot from the cd image I'm sure this would all work

I've also noted the issue at http://virtualmin.com/node/17535 - which seems to be the other half of the issue as to why you can't boot after changing to virtio.

I can try rebuilding the the initrd -- but just having any sort of 64-bit kvm CentOS image to build from would be a great service.

Submitted by JamieCameron on Sun, 08/25/2013 - 21:17

Ok, let me see if I can reproduce this - I've been successful following this procedure on x86 to create the standard Cloudmin images, but maybe 64-bit VMs don't work in quite the same way.

Submitted by JamieCameron on Mon, 08/26/2013 - 00:10

Ok, I have created a new 64-bit CentOS 6.4 image for KVM that is available for download by Cloudmin users. Give it a try, and let me know if it works for you..

Submitted by cruiskeen on Mon, 08/26/2013 - 07:52

I'm in the process of downloading the image to our servers. I'll let you know. Thanks for the quick response, as always.

Submitted by cruiskeen on Mon, 08/26/2013 - 09:57

I'm about 90% of the way through building a customized hosting image and so far this all looks great.

Note -- just because I end up working on Sundays and file a report didnt' necessarily mean that I expected YOU to work on Sunday and fix it :-).

Just wanted you to know it's appreciated, but it really could have waited.

One of these days I will try to step through why this has been so problematic though - at some point I'd like to be able to build these on our own systems.

Submitted by JamieCameron on Mon, 08/26/2013 - 19:02

So I did find a bug in Cloudmin when creating this image - after doing an install from CD into a VM with IDE disks and then creating an image of it, a new VM created in VirtIO mode from that image doesn't boot properly.

The fix for me was to edit /boot/grub/menu.lst in the VM and fix the root= parameter to the kernel. Once this was done, I was able to start the VM in VirtIO mode. The API commands mount-system and umount-system are useful for this kind of filesystem surgery.

The next Cloudmin release will fix this bug.

Submitted by cruiskeen on Mon, 08/26/2013 - 19:23

Just so I understand -- you created the image with the IDE drivers and then had to do this to be able to use Virtio?

Because the main stumbling block as far as I can see for me has been the inability to just boot on an empty system using Virtio in the first place --- It would be great to be able to have it use Virtio for the disk and IDE for the CDROM - but I could never get that to work.

Submitted by JamieCameron on Mon, 08/26/2013 - 20:48

Yes, I would have preferred to install directly onto VirtIO disks. But it seems that KVM and/or the KVM bios doesn't support mixed-mode systems in which the CD is IDE, but the disks are VirtIO.

Submitted by JamieCameron on Tue, 08/27/2013 - 18:34

FYI, three 64-bit images for KVM are now available for all users - CentOS 6.4, Debian Squeeze and Ubuntu 12.04.

Submitted by cruiskeen on Tue, 09/03/2013 - 08:17

Okay, I'll be happy to consider this one closed -