update failed : Failed to read from fastrpc.cgi

Submitted by Franco Nogarin on Tue, 01/29/2013 - 12:21 Pro Licensee

Hi,

My updates always say they fail when they succeed. I think it might be a bug or perhpas I have something setup wrong, here is an example after choosing to update two packages on my cloudmin master:


Updating 2 packages on smvmhost ..
Updating to dhcp3-server version 4.1.ESV-R4-0ubuntu5.6, webmin-server-manager version 6.8 ..............................
.. update failed : Failed to read from fastrpc.cgi :
.. all updates complete.
Refreshing information about available updates ..
.. refreshed successfully.

here is an example where I am updating a VM with 2 packages, same thing.


Updating 2 packages on sparcs-postgresql ..
Updating to isc-dhcp-client version 4.1.ESV-R4-0ubuntu5.6, isc-dhcp-common version 4.1.ESV-R4-0ubuntu5.6 ...........
.. update failed : Failed to read from fastrpc.cgi :
.. all updates complete.
Refreshing information about available updates ..

Status: 
Closed (fixed)

Comments

Submitted by JamieCameron on Tue, 01/29/2013 - 15:07

Make sure that ports 10000 to 10010 are open on the remote system's firewall - RPC errors like this can happen if those ports are blocked.

Submitted by Franco Nogarin on Tue, 01/29/2013 - 16:12 Pro Licensee

hmm, cloudmin host is accepting everything eg no rules, and all default action set to accept.

there is no IPTables firewall on the VMs so there should be no port blocking whatsoever.
when I check with

sudo ufw status

results on both systems (And in fact all systems) is:

sparcsadmin@sparcs-postgresql:~$ sudo ufw status
Status: inactive


sparcsadmin@smvmhost:~$ sudo ufw status
Status: inactive

checking the ports on the VM with netstat:

sparcsadmin@sparcs-postgresql:~$ sudo netstat --listen
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 *:webmin *:* LISTEN
tcp 0 0 *:ssh *:* LISTEN
tcp 0 0 localhost:postgresql *:* LISTEN
tcp6 0 0 [::]:ssh [::]:* LISTEN
udp 0 0 *:bootpc *:*
udp 0 0 *:10000 *:*
Active UNIX domain sockets (only servers)
Proto RefCnt Flags Type State I-Node Path
unix 2 [ ACC ] SEQPACKET LISTENING 6964 /run/udev/control
unix 2 [ ACC ] STREAM LISTENING 6585 @/com/ubuntu/upstart
unix 2 [ ACC ] STREAM LISTENING 7337 /var/run/dbus/system_bus_socket
unix 2 [ ACC ] STREAM LISTENING 8416 /var/run/postgresql/.s.PGSQL.5432
sparcsadmin@sparcs-postgresql:~$
sparcsadmin@sparcs-postgresql:~$ sudo netstat -plnt
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:10000 0.0.0.0:* LISTEN 875/perl
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 681/sshd
tcp 0 0 127.0.0.1:5432 0.0.0.0:* LISTEN 814/postgres
tcp6 0 0 :::22 :::* LISTEN 681/sshd
sparcsadmin@sparcs-postgresql:~$

now on the master:

root@smvmhost:/home/sparcsadmin# netstat --listen
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 localhost:40009 *:* LISTEN
tcp 0 0 *:webmin *:* LISTEN
tcp 0 0 *:5909 *:* LISTEN
tcp 0 0 *:5910 *:* LISTEN
tcp 0 0 *:ssh *:* LISTEN
tcp 0 0 *:5911 *:* LISTEN
tcp 0 0 localhost:6010 *:* LISTEN
tcp 0 0 localhost:6011 *:* LISTEN
tcp 0 0 localhost:40000 *:* LISTEN
tcp 0 0 localhost:40002 *:* LISTEN

sparcsadmin@smvmhost:~$ sudo netstat -plnt
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 127.0.0.1:40009 0.0.0.0:* LISTEN 9122/kvm
tcp 0 0 0.0.0.0:10000 0.0.0.0:* LISTEN 24666/perl
tcp 0 0 0.0.0.0:5909 0.0.0.0:* LISTEN 9114/kvm
tcp 0 0 0.0.0.0:5910 0.0.0.0:* LISTEN 7964/kvm
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1163/sshd
tcp 0 0 0.0.0.0:5911 0.0.0.0:* LISTEN 9122/kvm
tcp 0 0 127.0.0.1:6010 0.0.0.0:* LISTEN 21845/0
tcp 0 0 127.0.0.1:6011 0.0.0.0:* LISTEN 10772/1
tcp 0 0 127.0.0.1:40000 0.0.0.0:* LISTEN 7964/kvm
tcp 0 0 127.0.0.1:40002 0.0.0.0:* LISTEN 9114/kvm
tcp6 0 0 :::22 :::* LISTEN 1163/sshd
tcp6 0 0 ::1:6010 :::* LISTEN 21845/0
tcp6 0 0 ::1:6011 :::* LISTEN 10772/1
sparcsadmin@smvmhost:~$

does anything look amiss?
Franco

tpnsolutions's picture
Submitted by tpnsolutions on Tue, 01/29/2013 - 18:19

Hi,

If you're having with port "10000" try as a test changing the port number to something like "8080" from Webmin > Webmin > Webmin Configuration > Ports and Addresses.

Make sure to change both Listen on IPs and ports and Listen for broadcasts on UDP port values.

Save the settings, and see if the issue persists.

-Peter

Submitted by JamieCameron on Tue, 01/29/2013 - 20:02

That looks OK..

Does anything get logged to /var/webmin/miniserv.error on the remote system when this package upgrade operation fails?

Submitted by Franco Nogarin on Wed, 01/30/2013 - 12:50 Pro Licensee

@tpnsolutions I have tested port 10000 that is not the issue. I can connect to webmin and cloudmin directly on all machine using port 10000, but thank you.

@JamieCameron

When I update a VM from cloudmin GUI I get the error in the GUI but no error appears in the /var/webmin/miniserv.error log in the VM.

there are some errors in the hosts log but they are confusing.

I no longer have any updtes to perform on my Cloudmin Pro system, but I do on my Cloudmin GPL system, so I will clear the logs on the VM and on the host and perform some updates and update this issue with details.

here is an example from my confusing host log it would be nice if each entry was date-time stamped, it would make troubleshooting much easier...

[14/Jan/2013:18:48:18 -0700] [216.126.102.192] Bad Request
[14/Jan/2013:18:48:18 -0700] [216.126.102.192] Bad Request
[14/Jan/2013:18:48:18 -0700] [216.126.102.192] Bad Request
[14/Jan/2013:18:48:18 -0700] [216.126.102.192] Bad Request
Error: Error reading response length from fastrpc.cgi :
Error: No package names entered
Error: Missing or invalid hostname
Error: Missing or invalid new hostname
Error: Failed to lock file /etc/webmin/servers/135291918588150.serv after 5 minutes. Last error was :
Error: Console messages could not be displayed : No console messages were found
Error: Console messages could not be displayed : No console messages were found
Error: Console messages could not be displayed : No console messages were found
Error: Console messages could not be displayed : No console messages were found
Error: Console messages could not be displayed : No console messages were found
Error: Console messages could not be displayed : No console messages were found
Error: Console messages could not be displayed : No console messages were found
Error: Console messages could not be displayed : No console messages were found
Error: Console messages could not be displayed : No console messages were found
Error: Console messages could not be displayed : No console messages were found
Error: Console messages could not be displayed : No console messages were found
Error: Console messages could not be displayed : No console messages were found
Error: Failed to connect to 216.108.146.49:10000 : No route to host
Error: Failed to connect to 216.108.146.49:10000 : No route to host
Error: Failed to connect to 216.108.146.49:10000 : No route to host
Error: Failed to connect to 216.108.146.49:10000 : No route to host
Error: Failed to connect to 216.108.146.49:10000 : No route to host
Error: Failed to connect to 216.108.146.49:10000 : No route to host
Error: Console messages could not be displayed : No console messages were found
Error: No systems selected to operate on
Error: No systems selected to operate on
Error: Console messages could not be displayed : No console messages were found
Error: Console messages could not be displayed : No console messages were found
Error: The following systems cannot be shut down : geoserver
Error: Missing or invalid hostname
Error: Missing or invalid hostname

Submitted by Franco Nogarin on Wed, 01/30/2013 - 12:59 Pro Licensee

ok, here is a single update on a vm(Note this is acompletely differnt webmin system and network, same problem, so not specific to my install):

Updating 1 packages on hal ..
Updating to ffmpeg version 0.8.5-0ubuntu0.12.04.1 ............
.. update failed : Failed to read from fastrpc.cgi :
.. all updates complete.
Refreshing information about available updates ..
.. refreshed successfully.

the log webmin.error on VM after is still blank but found alot of this in the miniserv.log

192.168.80.3 - root [23/Jan/2013:21:25:25 -0700] "POST /fastrpc.cgi HTTP/1.0" 200 47
192.168.80.3 - root [30/Jan/2013:10:31:45 -0700] "POST /fastrpc.cgi HTTP/1.0" 200 47
192.168.80.3 - root [30/Jan/2013:10:31:48 -0700] "POST /fastrpc.cgi HTTP/1.0" 200 47
192.168.80.3 - root [30/Jan/2013:11:35:28 -0700] "POST /fastrpc.cgi HTTP/1.0" 200 47
192.168.80.3 - root [30/Jan/2013:11:35:52 -0700] "POST /fastrpc.cgi HTTP/1.0" 200 47

the host had nothing in that log either.

Now I will try an update directly on the host.

Updating 1 packages on wwvi-mind ..
Updating to webmin-server-manager version 6.8.kvm ..............................
.. update failed : Failed to read from fastrpc.cgi :
.. all updates complete.
Refreshing information about available updates ..
.. refreshed successfully.

now here is the errors in the miniserv.error log, which was blank before the update

[30/Jan/2013:11:41:29 -0700] miniserv.pl started
[30/Jan/2013:11:41:29 -0700] Using MD5 module Digest::MD5
[30/Jan/2013:11:41:29 -0700] PAM authentication enabled
Failed to initialize SSL connection
Failed to initialize SSL connection

no evidence of anything there :(
Franco

Submitted by JamieCameron on Wed, 01/30/2013 - 23:43

If you perform a package update directly on the remote system (with a command like apt-get install ffmpeg) does it report any errors?

Submitted by Franco Nogarin on Thu, 01/31/2013 - 09:56 Pro Licensee

no, it is only related to the fastrpc.cgi in webmin GUI, all systems do manual updates fine in the shell. And in case I did not make it clear, even though the GUI says the update fails, it does not. all updates on both cloudmin pro and cloudmin GPL on both networks on all hosts and all vms fail with that message but they actually dont fail, every one is actually properly updated.

Submitted by JamieCameron on Thu, 01/31/2013 - 18:23

Is there any chance I could login to your Cloudmin system and see what is going wrong here? If that is possible, email me directly at jcameron@virtualmin.com

Submitted by Franco Nogarin on Fri, 02/01/2013 - 18:22 Pro Licensee

I will email you this weekend, thanks. Franco

Submitted by JamieCameron on Sat, 02/02/2013 - 20:31

I logged into your system, but wasn't able to re-produce this problem. Does it happen for all package updates?

I did a test install of one package (ffmpeg) on sparcs-postgresql using the command :

vm2 update-systems --host sparcs-postgresql --package ffmpeg --install

and it worked fine..

Submitted by Franco Nogarin on Mon, 02/04/2013 - 10:03 Pro Licensee

yes all updates, not installing packages. to reproduce the problem:
1) login to cloudmin web gui.
2) click list managed systems.
3) look at sparcs-postgresql (This is a safe VM for you to use)
4) click on webmin - 2 updates
5) select only " webmin 1.610 1.620 "
6) click update selected packages

In the GUI you WILL see:


Updating 1 packages on sparcs-postgresql ..
Updating to webmin version 1.620 .................................................................................. .....................................
.. update failed : Failed to read from fastrpc.cgi :
.. all updates complete.
Refreshing information about available updates ..
.. refreshed successfully.

Submitted by Franco Nogarin on Mon, 02/04/2013 - 10:29 Pro Licensee

FYI, this does not happen when multiple systems are selected eg:

Updating 14 packages on 6 systems .................................................................................. ................................................................................ ................................................................................ ................................................................................ ................................................................................ ................................................................................ ................................................................................ ................................................................................ ................................................................................ ................................................................................ ................................................................................ ................................................................................ ................................................................................ ................................................................................ ..........................
Updated packages on smvmhost ..
Updating to linux-headers-server version 3.2.0.37.45, linux-image-server version 3.2.0.37.45, linux-server version 3.2.0.37.45 ..
.. installed linux-image-3.2.0-37-generic linux-headers-3.2.0-37 linux-headers-3.2.0-37-generic linux-image-server linux-headers-server linux-server
Updated packages on geoserver2 ..
Updating to base-files version 6.5ubuntu6.5, oracle-java7-installer version 7u13-0~webupd8~0, oracle-jdk7-installer version 7u13-0~webupd8~0, webmin version 1.620 ..
.. installed base-files oracle-java7-installer webmin oracle-jdk7-installer
Updated packages on smvmbackup64 ..
Updating to base-files version 6.5ubuntu6.5, webmin version 1.620 ..
.. installed base-files webmin
Updated packages on smvmhost64 ..
Updating to webmin version 1.620 ..
.. installed webmin
Updated packages on sparcs-fileserver ..
Updating to base-files version 6.5ubuntu6.5, webmin version 1.620 ..
.. installed base-files webmin
Updated packages on sparcs-old ..
Updating to base-files version 6.5ubuntu6.5, webmin version 1.620 ..
.. installed base-files webmin
.. update complete.

Submitted by JamieCameron on Mon, 02/04/2013 - 22:54

Is it only updates to that webmin package that is triggering this?

Submitted by Franco Nogarin on Tue, 02/05/2013 - 18:02 Pro Licensee

Hi Jamie, No, not just the webmin package.

to re-summarize:

  • It happens only in the GUI
  • it happens on both cloumin GPL and pro
  • it only happens when you update packages, not install them
  • it only happens when you update on one target, not in multiple.
  • the above process in comment #12 will reproduce it , you can select any package, but in MY vm, please just update webmin as it is a live production system. I left the webmin package for you to update.
  • I am pretty sure by now the problem is in the mass_update_form.cgi

Franco

Submitted by JamieCameron on Wed, 02/06/2013 - 01:35

Ok, there is a different code path when one system is being updated vs. multiple that would explain this.

Is there another package I can try an update of, other than just webmin ?

Submitted by Franco Nogarin on Wed, 02/06/2013 - 13:12 Pro Licensee

sure, go to the miscteam system and update any of the packages on the host "hal". there are 12 updates available on that VM and in the future I will save some updates in the sparcs system in case you need to test. Franco

Submitted by JamieCameron on Wed, 02/06/2013 - 22:10

Ok, I am re-trying this now ..

Submitted by JamieCameron on Wed, 02/06/2013 - 23:44

Ok, I have found the cause, and have applied a patch for it on your system. Future Webmin / Cloudmin releases will also include the fix.

Submitted by Franco Nogarin on Fri, 02/08/2013 - 09:20 Pro Licensee

Super thank you Jamie! So you patched miscteam, did you also patch the sparcs system or will that one wait until the next release? Franco

Submitted by JamieCameron on Fri, 02/08/2013 - 16:49

No, I only patched one system. To get the fix on the other machine, just copy across the file /usr/share/webmin/server-manager/mass_update.cgi

Submitted by Franco Nogarin on Fri, 02/08/2013 - 17:11 Pro Licensee

Thanks Mate!

Franco

Submitted by Issues on Fri, 02/22/2013 - 17:16

Automatically closed -- issue fixed for 2 weeks with no activity.