collectinfo.pl fails w/ "Floating point exception"

Submitted by datenimperator on Fri, 07/20/2012 - 01:52

I've installed a number of OpenVZ systems with a minimal Debian 6 and Virtualmin GPL on top. Every now and then, the virtual-server/collectinfo.pl cronjob dies with a floating point exception. It happens approx. 5-10 times a day, the job runs every 5 minutes. AFAICT it happens on all instances install that way.

Running the script on the command line doesn't show anything

I've no idea if webmin stores an error log, if any. The miniserv.conf setting to show stack traces (in the browser) doesn't help. Any idea what could be wrong?

  • Operating system Debian Linux 6.0
  • Webmin version 1.590
  • Virtualmin version 3.93.gpl GPL
  • Theme version 8.5
  • Time on server 20/Jul/2012 08:44 , Up 8 days, 21 hours, 39 minutes
  • Kernel and CPU Linux 2.6.32-12-pve on x86_64
  • CPU load averages 0.02 (1 min) 0.03 (5 mins) 0.00 (15 mins)
Status: 
Active

Comments

Submitted by datenimperator on Fri, 07/20/2012 - 01:55

An example error mail (somewhat anonymized):

Return-Path: <root@v3>
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
v1
X-Spam-Level:
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,TVD_SPACE_RATIO,
T_RP_MATCHES_RCVD autolearn=ham version=3.3.1
X-Original-To: domains@
Delivered-To: username@v1
Received: from v3 (v3[...])
by v1 (Postfix) with ESMTP id A844019E61F
for <domains@>; Fri, 20 Jul 2012 08:50:03 +0200 (CEST)
Received: by v3 (Postfix)
id 53ACC24031E; Fri, 20 Jul 2012 08:50:03 +0200 (CEST)
Delivered-To: root@v3
Received: by v3 (Postfix, from userid 0)
id 4F188240361; Fri, 20 Jul 2012 08:50:03 +0200 (CEST)
From: root@v3 (Cron Daemon)
To: root@v3
Subject: Cron <root@v3> /etc/webmin/virtual-server/collectinfo.pl
Content-Type: text/plain; charset=UTF-8
X-Cron-Env: <SHELL=/bin/sh>
X-Cron-Env: <HOME=/root>
X-Cron-Env: <PATH=/usr/bin:/bin>
X-Cron-Env: <LOGNAME=root>
Message-Id: <20120720065003.4F188240361@>
Date: Fri, 20 Jul 2012 08:50:03 +0200 (CEST)

Floating point exception

Submitted by andreychek on Fri, 07/20/2012 - 09:39

Howdy -- Jamie can comment more specifically on the issue you're seeing, but I'll offer two things in the meantime.

One, if you don't want collectinfo to run so frequently (on a VPS, and especially one with Virtualmin GPL, it may not be as useful), you could go into System Settings -> Virtualmin Config -> Status Collection, and you could try setting it to "60" rather than the default of "5".

That just means some stats will be updated every 60 minutes rather than 5 minutes.

Second, we've seen some unusual memory related things on OpenVZ, and I'd like to rule out that what you're seeing isn't related to resources... could you paste in a copy of your /proc/user_beancounters file?

Thanks!

Submitted by datenimperator on Fri, 07/20/2012 - 09:58

I've changed the frequency to every 60 minutes. What does that job do anyway? There's a very similar Webmin job, is it related?

Please find a dump of the file attached.

Submitted by andreychek on Fri, 07/20/2012 - 20:35

That job just updates statistics that are available on the System Information and stats screens (the latter of which is only available on Pro installs). It also checks for package updates.

We've seen some unusual issues on OpenVZ relating to resources, but that doesn't seem to be the case here -- there aren't any resource issues showing up in the user_beancounters file you attached. So that's good.

Do you by chance see any errors in /var/webmin/miniserv.error?

Submitted by datenimperator on Sat, 07/21/2012 - 02:32

miniserv.error simply contains the line "Floating point exception" without further explanation, along with other (possibly unrelated) entries. However:

I configured miniserv to show stack traces in the browser, hoping that would help. Since I did that I don't see any "Floating point exception" lines anymore although there have been some. Instead I see multiple lines reading "cat: /etc/lsb-release: No such file or directory"

In fact, package lsb-core isn't installed, nor is there a file lsb-release.

To give it a try I installed lsb-core on one of the machines, which installed a number of packages along it:

  alien at autopoint bc build-essential bzip2 cups-bsd cups-client cups-common
  debhelper dpkg-dev ed fakeroot g++ g++-4.4 gettext gettext-base git
  html2text intltool-debian less lib32z1 libalgorithm-diff-perl
  libalgorithm-diff-xs-perl libalgorithm-merge-perl libavahi-client3
  libavahi-common-data libavahi-common3 libc6-i386 libcroco3 libcups2
  libcupsimage2 libcurl3-gnutls libdpkg-perl libelf1 libglib2.0-0
  libglib2.0-data libidn11 liblua5.1-0 libmail-sendmail-perl libnspr4-0d
  libnss3-1d librpm1 librpmbuild1 librpmio1 libstdc++6-4.4-dev libtiff4
  libunistring0 lsb-core m4 ncurses-term patch pax po-debconf rpm rpm-common
  rpm2cpio shared-mime-info time

Wonder if it will help...

Submitted by datenimperator on Sat, 07/21/2012 - 07:04

Hm, installing lsb-core didn't change a thing. I'm still getting those exceptions from that machine. Also, that lines about the missing lsb-release file are written to the log when you open the "Operating System and Environment" from the Webmin configuration. Possibly unrelated...

Submitted by JamieCameron on Mon, 07/23/2012 - 17:30

Does anything get logged to /var/log/messages or /var/log/dmesg when this floating point exception error happens?

Submitted by datenimperator on Tue, 07/24/2012 - 02:15

No. /var/log/dmesg is empty and /var/log/messages contains only 3 older, unrelated messages about rsyslogd restarting because of updated settings.

Submitted by JamieCameron on Tue, 07/24/2012 - 13:04

I did some research, and it looks like this error occurs somewhere inside Perl, rather than in the Virtualmin code ... so there isn't much we can do about this :-(

All I can suggest is suppressing the error emails by running :

crontab -e

and changing the line that runs /etc/webmin/virtual-server/collectinfo.pl to add >/dev/null 2>&1 to the end of the command.

Submitted by datenimperator on Tue, 07/24/2012 - 23:47

Funny though: It's the very same Perl on all of my machines:

root@v1:~# perl -V
Summary of my perl5 (revision 5 version 10 subversion 1) configuration:
  
  Platform:
    osname=linux, osvers=2.6.32-5-amd64, archname=x86_64-linux-gnu-thread-multi
    uname='linux barber 2.6.32-5-amd64 #1 smp thu nov 3 03:41:26 utc 2011 x86_64 gnulinux '
    config_args='-Dusethreads -Duselargefiles -Dccflags=-DDEBIAN -Dcccdlflags=-fPIC -Darchname=x86_64-linux-gnu -Dprefix=/usr -Dprivlib=/usr/share/perl/5.10 -Darchlib=/usr/lib/perl/5.10 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.10.1 -Dsitearch=/usr/local/lib/perl/5.10.1 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dsiteman1dir=/usr/local/man/man1 -Dsiteman3dir=/usr/local/man/man3 -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Ud_ualarm -Uusesfio -Uusenm -DDEBUGGING=-g -Doptimize=-O2 -Duseshrplib -Dlibperl=libperl.so.5.10.1 -Dd_dosuid -des'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
    optimize='-O2 -g',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include'
    ccversion='', gccversion='4.4.5', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -fstack-protector -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib /lib64 /usr/lib64
    libs=-lgdbm -lgdbm_compat -ldb -ldl -lm -lpthread -lc -lcrypt
    perllibs=-ldl -lm -lpthread -lc -lcrypt
    libc=/lib/libc-2.11.3.so, so=so, useshrplib=true, libperl=libperl.so.5.10.1
    gnulibc_version='2.11.3'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -g -L/usr/local/lib -fstack-protector'


Characteristics of this binary (from libperl):
  Compile-time options: MULTIPLICITY PERL_DONT_CREATE_GVSV
                        PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP USE_64_BIT_ALL
                        USE_64_BIT_INT USE_ITHREADS USE_LARGE_FILES
                        USE_PERLIO USE_REENTRANT_API
  Locally applied patches:
DEBPKG:debian/arm_thread_stress_timeout - http://bugs.debian.org/501970 Raise the timeout of ext/threads/shared/t/stress.t to accommodate slower build hosts
DEBPKG:debian/cpan_config_path - Set location of CPAN::Config to /etc/perl as /usr may not be writable.
DEBPKG:debian/cpan_definstalldirs - Provide a sensible INSTALLDIRS default for modules installed from CPAN.
DEBPKG:debian/db_file_ver - http://bugs.debian.org/340047 Remove overly restrictive DB_File version check.
DEBPKG:debian/doc_info - Replace generic man(1) instructions with Debian-specific information.
DEBPKG:debian/enc2xs_inc - http://bugs.debian.org/290336 Tweak enc2xs to follow symlinks and ignore missing @INC directories.
DEBPKG:debian/errno_ver - http://bugs.debian.org/343351 Remove Errno version check due to upgrade problems with long-running processes.
DEBPKG:debian/extutils_hacks - Various debian-specific ExtUtils changes
DEBPKG:debian/fakeroot - Postpone LD_LIBRARY_PATH evaluation to the binary targets.
DEBPKG:debian/instmodsh_doc - Debian policy doesn't install .packlist files for core or vendor.
DEBPKG:debian/ld_run_path - Remove standard libs from LD_RUN_PATH as per Debian policy.
DEBPKG:debian/libnet_config_path - Set location of libnet.cfg to /etc/perl/Net as /usr may not be writable.
DEBPKG:debian/m68k_thread_stress - http://bugs.debian.org/495826 Disable some threads tests on m68k for now due to missing TLS.
DEBPKG:debian/mod_paths - Tweak @INC ordering for Debian
DEBPKG:debian/module_build_man_extensions - http://bugs.debian.org/479460 Adjust Module::Build manual page extensions for the Debian Perl policy
DEBPKG:debian/perl_synopsis - http://bugs.debian.org/278323 Rearrange perl.pod
DEBPKG:debian/prune_libs - http://bugs.debian.org/128355 Prune the list of libraries wanted to what we actually need.
DEBPKG:debian/use_gdbm - Explicitly link against -lgdbm_compat in ODBM_File/NDBM_File.
DEBPKG:fixes/assorted_docs - http://bugs.debian.org/443733 [384f06a] Math::BigInt::CalcEmu documentation grammar fix
DEBPKG:fixes/net_smtp_docs - http://bugs.debian.org/100195 [rt.cpan.org #36038] Document the Net::SMTP 'Port' option
DEBPKG:fixes/processPL - http://bugs.debian.org/357264 [rt.cpan.org #17224] Always use PERLRUNINST when building perl modules.
DEBPKG:debian/perlivp - http://bugs.debian.org/510895 Make perlivp skip include directories in /usr/local
DEBPKG:fixes/pod2man-index-backslash - http://bugs.debian.org/521256 Escape backslashes in .IX entries
DEBPKG:debian/disable-zlib-bundling - Disable zlib bundling in Compress::Raw::Zlib
DEBPKG:fixes/kfreebsd_cppsymbols - http://bugs.debian.org/533098 [3b910a0] Add gcc predefined macros to $Config{cppsymbols} on GNU/kFreeBSD.
DEBPKG:debian/cpanplus_definstalldirs - http://bugs.debian.org/533707 Configure CPANPLUS to use the site directories by default.
DEBPKG:debian/cpanplus_config_path - Save local versions of CPANPLUS::Config::System into /etc/perl.
DEBPKG:fixes/kfreebsd-filecopy-pipes - http://bugs.debian.org/537555 [16f708c] Fix File::Copy::copy with pipes on GNU/kFreeBSD
DEBPKG:fixes/anon-tmpfile-dir - http://bugs.debian.org/528544 [perl #66452] Honor TMPDIR when open()ing an anonymous temporary file
DEBPKG:fixes/abstract-sockets - http://bugs.debian.org/329291 [89904c0] Add support for Abstract namespace sockets.
DEBPKG:fixes/hurd_cppsymbols - http://bugs.debian.org/544307 [eeb92b7] Add gcc predefined macros to $Config{cppsymbols} on GNU/Hurd.
DEBPKG:fixes/autodie-flock - http://bugs.debian.org/543731 Allow for flock returning EAGAIN instead of EWOULDBLOCK on linux/parisc
DEBPKG:fixes/archive-tar-instance-error - http://bugs.debian.org/539355 [rt.cpan.org #48879] Separate Archive::Tar instance error strings from each other
DEBPKG:fixes/positive-gpos - http://bugs.debian.org/545234 [perl #69056] [c584a96] Fix \G crash on first match
DEBPKG:debian/devel-ppport-ia64-optim - http://bugs.debian.org/548943 Work around an ICE on ia64
DEBPKG:fixes/trie-logic-match - http://bugs.debian.org/552291 [perl #69973] [0abd0d7] Fix a DoS in Unicode processing [CVE-2009-3626]
DEBPKG:fixes/hppa-thread-eagain - http://bugs.debian.org/554218 make the threads-shared test suite more robust, fixing failures on hppa
DEBPKG:fixes/crash-on-undefined-destroy - http://bugs.debian.org/564074 [perl #71952] [1f15e67] Fix a NULL pointer dereference when looking for a DESTROY method
DEBPKG:fixes/tainted-errno - http://bugs.debian.org/574129 [perl #61976] [be1cf43] fix an errno stringification bug in taint mode
DEBPKG:fixes/safe-upgrade - http://bugs.debian.org/582978 Upgrade Safe.pm to 2.25, fixing CVE-2010-1974
DEBPKG:fixes/tell-crash - http://bugs.debian.org/578577 [f4817f3] Fix a tell() crash on bad arguments.
DEBPKG:fixes/format-write-crash - http://bugs.debian.org/579537 [perl #22977] [421f30e] Fix a crash in format/write
DEBPKG:fixes/arm-alignment - http://bugs.debian.org/289884 [f1c7503] Prevent gcc from optimizing the alignment test away on armel
DEBPKG:fixes/fcgi-test - Fix a failure in CGI/t/fast.t when FCGI is installed
DEBPKG:fixes/hurd-ccflags - http://bugs.debian.org/587901 Make hints/gnu.sh append to $ccflags rather than overriding them
DEBPKG:debian/squelch-locale-warnings - http://bugs.debian.org/508764 Squelch locale warnings in Debian package maintainer scripts
DEBPKG:fixes/lc-numeric-docs - http://bugs.debian.org/379329 [perl #78452] [903eb63] LC_NUMERIC documentation fixes
DEBPKG:fixes/lc-numeric-sprintf - http://bugs.debian.org/601549 [perl #78632] [b3fd614] Fix sprintf not to ignore LC_NUMERIC with constants
DEBPKG:fixes/concat-stack-corruption - http://bugs.debian.org/596105 [perl #78674] [e3393f5] Fix stack pointer corruption in pp_concat() with 'use encoding'
DEBPKG:fixes/cgi-multiline-header - http://bugs.debian.org/606995 [CVE-2010-2761 CVE-2010-4410 CVE-2010-4411] CGI.pm MIME boundary and multiline header vulnerabilities
DEBPKG:fixes/casing-taint-cve-2011-1487 - http://bugs.debian.org/622817 [perl #87336] fix unwanted taint laundering in lc(), uc() et al.
DEBPKG:fixes/safe-reval-rdo-cve-2010-1447 - [PATCH] Wrap by default coderefs returned by rdo and reval
DEBPKG:fixes/encode-heap-overflow - [PATCH] Fix decode_xs n-byte heap-overflow security bug in
DEBPKG:fixes/digest_eval_hole - Close the eval "require $module" security hole in
DEBPKG:fixes/unregister_signal_handler - [PATCH] main: Unregister signal handler before destroying my_perl
DEBPKG:patchlevel - http://bugs.debian.org/567489 List packaged patches for 5.10.1-17squeeze3 in patchlevel.h
  Built under linux
  Compiled at Dec 21 2011 09:15:46
  @INC:
    /etc/perl
    /usr/local/lib/perl/5.10.1
    /usr/local/share/perl/5.10.1
    /usr/lib/perl5
    /usr/share/perl5
    /usr/lib/perl/5.10
    /usr/share/perl/5.10
    /usr/local/lib/site_perl
    .

Virtualmin Pro on instance v1, Virtualmin GPL on instance v2. It's only v2 showing this error.

Submitted by JamieCameron on Wed, 07/25/2012 - 00:20

Perhaps it is an issue with lower-level system libraries, or the OpenVZ kernel?

In regular use Perl should never crash like that, regardless of what a script does..

Submitted by datenimperator on Wed, 07/25/2012 - 00:28

Yet they are both OpenVZ instances on the same host, Virtualmin Pro on one, GPL on the other. GPL fails, Pro never.

Submitted by JamieCameron on Wed, 07/25/2012 - 12:28

Do you have Virtualmin installed on any non-OpenVZ systems running the same Linux distribution, and if so have you seen the error there?