ClamAV fails to scan emails: probable issue with streaming mode

This is somewhat of an odd issue, and I'm not sure "whose fault" it is. After digging through Procmail configurations and trying to understand your Perl code for about two hours now :), it seems that ClamAV has some problems, and due to the way VMin is using it, it fails to scan emails properly, hence I'm inclined to label this as a bug - even though VMin is probably not the culprit, and I'm a bit stumped that I did not notice this earlier.

Anyway, I have a freshly installed Ubuntu 10.04 and am testing scanning emails for spam and viruses. Spam scanning works all okay, so Procmail is set up properly and all. Virus scanning is properly set up as well, all the settings are okay, the Procmail config file looks good.

Test emails containing EICAR though get through all untouched. After checking various options multiple times and suspecting Procmail of not calling the virus scanner, my investigation results boil down to an independant strange behavior of clamdscan: it fails to recognize viruses in streaming mode.

I created a file /tmp/test containing the EICAR signature. When I do clamdscan /tmp/test, it correctly reports the file as infected.

But when I do cat /tmp/test | clamdscan -, which is - as far as I understand your code - analogous to the way that clam-wrapper.pl calls the scanner, it does not recognize the file as infected.

clamdscan -V says:
ClamAV 0.96.3/12210/Fri Nov 5 16:06:13 2010

Any insight on this with the info so far?

Status: 
Active

Comments

Another piece of information that might help:

On another machine of mine, the streaming mode clamdscan works all okay.

The difference between the two is: The one with the working scanner is a 64-bit Ubuntu, the one with the failure is 32-bit.

I presume that when you using clamdscan you have clamd running?

If so, what gets logged to its log file /var/log/clamav/clamav.log when you try a scan?

Also, has clamd been restarted recently? It may be that it is somehow using an old virus DB..

Thanks for reply, Jamie! :)

Yepp, clamd is running okay. The call to clamdscan does not report any errors as well, the connection seems to be okay. It's just that clamd does not recognize the virus.

Concerning logs, on the machine where it does work, this line gets logged:

Fri Nov 5 22:08:37 2010 -> stream: Eicar-Test-Signature(69630e4574ec6798239b091cda43dca0:69) FOUND

On the "fail" machine... Nothing in the log. Odd as it may sound.

I restarted the service and also the whole box, yeah. Lemme see about virus DB... Here, this is what the "fail" machine logs upon service startup:

Fri Nov  5 20:55:44 2010 -> +++ Started at Fri Nov  5 20:55:44 2010
Fri Nov  5 20:55:44 2010 -> clamd daemon 0.96.3 (OS: linux-gnu, ARCH: i386, CPU: i486)
Fri Nov  5 20:55:44 2010 -> Log file size limit disabled.
Fri Nov  5 20:55:44 2010 -> Reading databases from /var/lib/clamav
Fri Nov  5 20:55:44 2010 -> Not loading PUA signatures.
Fri Nov  5 20:55:46 2010 -> Loaded 848314 signatures.

Looks fine to me... Also, thing is that the EICAR does get detected when I use the "filename" syntax (otherwise I'd totally agree that the ClamAV is damaged somehow), just not when I use the "stream" syntax. And that only on 32-bit. Really odd.

Do you maybe know about a "debug mode" for ClamAV, getting it log a little more info about what it does/receives?

Try out the LogVerbose line in /etc/clamav/clamd.conf

Seems like a ClamAV bug if scanning doesn't work in streaming mode.

Yeah, I agree... I turned on LogVerbose, still no useful text in syslog or clamav.log.

Okay, I'll have dinner now and later I'm gonna install a totally fresh Ubuntu/VMin on another 32 bit VM, to make sure it's not some configuration I already made. If the issue persists then, I suppose we can inform the ClamAV people about it. :)

Meanwhile, if it turns out necessary, a viable workaround in VMin might be saving the email to a file and using the file mode syntax of clamdscan.

I'd prefer to avoid saving the mail to a file, as that runs into issues with disk quotas and temp file storage. Currently using clamdscan - works nicely, because procmail can feed the email directly into it..

Yeah, I agree there...

I just did the test as announced. I installed a vanilla Ubuntu 10.04 32-bit. Selected only SSHD and Samba in the installer. Updated all packages to latest versions using Aptitude. Rebooted, downloaded VMin install script. Installed it, performed the first-start wizard. Have it load libraries, spamd, clamd and MySQL into memory.

Then, without any further configuration, tested the same as before, with the same result:

root@mira:~# clamdscan /tmp/test
/tmp/test: Eicar-Test-Signature(69630e4574ec6798239b091cda43dca0:69) FOUND
 
----------- SCAN SUMMARY -----------
Infected files: 1
Time: 0.016 sec (0 m 0 s)
root@mira:~#
root@mira:~# cat /tmp/test | clamdscan -
stream: OK
 
----------- SCAN SUMMARY -----------
Infected files: 0
Time: 0.000 sec (0 m 0 s)

Seems a serious bug in ClamAV.

You mean my "tmp/test" test subject?

Certainly... I simply pasted the EICAR virus scanner test signature in there. You can retrieve the fresh-from-the-source version here: http://www.eicar.org/download/eicar.com.txt

An interesting fact about this "test virus" is that the text consists of nothing but valid and printable ASCII characters, but still constitutes a valid 16-bit DOS .com application. :) On DOS emulators, or Windows OSes before Vista (where Microsoft dropped com support), you can execute that thing and it will print the message "EICAR-STANDARD-ANTIVIRUS-TEST-FILE" (which is also readable in the text line), to the console window.

The program is completely harmless, but all virus scanners out there will flag it as infected, for testing purposes if the scanner works.

So I tested this fine on my Fedora 9 box with ClamAV 0.96, and a virus file with the contents from that eicar.com.txt URL was correctly identified by clamdscan in streaming and file modes..

This is really an odd one. It must have something to do with the communication between clamdscan and clamd under Ubuntu 32-bit.

One test (I performed some more here) yielded an error message:

root@mira:~# cat /tmp/test | clamdscan -
stream: OK
ERROR: Communication error
 
----------- SCAN SUMMARY -----------
Infected files: 0
Time: 0.054 sec (0 m 0 s)

Also, when I use clamscan instead of clamdscan, the EICAR is detected correctly also in streaming mode.

I think you will need to ask either the ClamAV developers or Ubuntu about this one .. there isn't really much we can do.

I think I found something.

In /etc/clamav/clamd.conf there is an option named StreamMaxLength. According to the man page, its default is 10 megabytes. That's also what the file on my 64-bit Ubuntu shows.

On the 32-bit test version though, that parameter is set to zero!

After setting it to "10M", clamdscan works just as expected. Now, how the heck did this option get set to "0" on the 32-bit Ubuntu...

Does the VMin installer have any influence on that? Or does it just use default parameters for ClamAV installation?

Okay, this is odd once more. I just purged and re-installed the ClamAV packages on both test systems (32 and 64 bit), and the default setting for StreamMaxSize indeed appears to be "0". Consequently, after the re-install, clamdscan in streaming mode does not work on both systems.

I'm 99.9% sure though that the setting was at "10M" for the 64 bit system before. So, something must have configured it so at some point.

Okay, I think this can be closed for now. I'll investigate further to find out which process it was that set the correct entry in the clamd.conf. :)

Virtualmin certainly doesn't set that option .. in fact I never even knew about it!