Showing posts with label ClamAV. Show all posts
Showing posts with label ClamAV. Show all posts

Monday, July 2, 2012

Is that an infection or a false positive?

Have you been in a situation where there's a file being flagged by A/V and you don't really agree? (...is that rhetorical?)  I was in a situation where I was noticing files being flagged as a generic variant of ZeuS and while at first you can't necessarily disregard the alert -no matter your feelings on the A/V- you can do a little digging and try to determine what's actually going on.  This  is not something you are or should do for every infection you come across, but rather a more practical use is to understand why certain files may be mis-classified when they are in fact benign.


The particular A/V vendor that was reporting the alerts was classifying them as 'W32/Zbot.gen.*' ... the 'gen.b' was most noticeable.  I grabbed one of the files in question and started to poke around.  Some of the usual first steps led no where - internal hash lookups, external hash look-ups (cymru, VT etc.), pescanner had a generic YARA hit for banker based on a string which looked all too common, dynamic analysis didn't show anything... so I started to extend some of my initial steps:

I extracted the PE sections with 7zip so I could do sectional MD5 hashing and see if I could get any leads by comparing those to other known bad sectional hashes :

$ 7z x <file>.exe -osections

The above syntax will extract the contents of each of the PE sections, in a tree structure (note that these will most likely be hidden since they start with "." so make sure you list all) :




Now that the PE sections are dumped I opted to use ClamAv for creating the sectional based MD5's.  ClamAV gives you this ability by using the following syntax:

$ sigtool --mdb <file>

The format that gets created for these signatures is:
PESectionSize:MD5:MalwareName

 If you do it right, you should see similar output to this:

57344:fceb22b4c5be5a981e6b7bd1e47dca63:.data
45:8e5a1b84a87bcd2eed7b3ab698a72123:files.txt
77824:34aeb441429d8f6184d0f6ba5d34cddd:.rdata
479232:68c0e32b605b7ccead4ad9520d5a5acc:.text

Welp... no luck on that front either -  but since I didn't have all my samples to cross-check them against, it was more of a long shot anyway.  Now what sparked curiosity is that ClamAV was also raising alerts on this particular file with the name "Trojan.Murofet".  That name is interchanged with Zbot depending on which vendor you're using so it was still leaning towards the same kind of classification for this file.  Hey, if two A/V's are flagging it for pretty much the same thing isn't that more credibility?

I've been incorporating ClamAV and it's misc. tools more into my process because it's free, maintained, cross-platform, I'm able to create my own signatures and I can even view/edit theirs.  Think about how great the latter of that statement is... If I don't like how something's being detected, I can change it myself.  If I want to catch something from my personal collection I can create my own signature and here's the greatest part - I can see what was being used in order to classify a detection.  Most of the bigger A/V companies hold that little gem to themselves and thus make this type of analysis difficult.

Using ClamAV's sigtool I decompressed its main signature datababase :

$ sigtool -u /path/to/main.cvd


The second part of the above image shows me searching for the detection it was classifying it as (Murofet).  You'll notice that there's more than one entry in this case and that they're both a bit different.  The first hit, Trojan.Murofet, is a sectional hash signature taking on the following format:

69632:7e82be33bfa6b241bf081909d40e265c:Trojan.Murofet

Explained:
PESectionSize:MD5:MalwareName

The second hit, W32.Murofet, is a regular signature taking on the following format:

W32.Murofet:1:EP+0:e850000000e9????????73686c776170692e646c6c0061647661706933322e646c6c0075726c6d6f6e2e646c6c00536f6674776172655c4d6963726f736f667400746d7000687474703a2f2f002f666f72756d2f005589e583ec0453e8ea0100008945fc

Explained:
MalwareName:TargetType:Offset:HexSignature[:MinFL:[MaxFL]]


The second hit is of more interest in this case if we take a look at what it's really saying...

For any 32/64 bit EXE, if at the entrypoint you see "èPéshlwapi.dlladvapi32.dllurlmon.dllSoftware\Microsofttmphttp:///forum/U‰åƒì Sèê ‰Eü" then flag it as W32.Murofet.

The HEX string showed in the hit can be decoded from HEX to ASCII which will reveal the string displayed above between the quotes.  The use of double question marks inbetween is a wildcard stating "match any byte"


Since I now know what the signature within ClamAV was triggering on I wanted to take a look at the EXE's entry point and see if those strings were in fact there.  Even though this could have still been done within REMNux, I flipped over to a Windows analysis box and opened the file in CFF Explorer to get a different view of things.  From within the 'Sectional Headers' I could see the entrypoint (bottom right):




Geared with that value I opened a hex editor, HxD in this example, and pointed it to go to that offset :



and wouldn't ya know it ... I was presented with "shlwapi.dll , advapi32.dll,  urlmon.dll, Software\Microsoft, tmp, http://, forum" .  So does the presence of these strings make the file malicious or do they simply help in trying to determine its characteristics/capabilities from a static analysis perspective?  If you've ever analyzed a ZeuS sample you'd notice that what was uncovered here doesn't quite line up with the normal data encountered, however, what about ZeuS-Licat?  Trend Micro has a great write up [PDF] here.  What it appears is that there was a version of ZeuS somewhere else which dropped Licat and what we saw at the files entry point was newly added malicious code - now  appended to this once legitimate file (file infector characteristic).

Even if you can't dig into the signature responsible on the other A/V you shouldn't call it quits.  If you can find another tool (such as ClamAV) which is classifying it in a similar way then there's a good chance that it's following some of the uncovered signatures logic within ClamAV and you have an idea of how/if it was being mis-classified. Even if you look at a file and a large majority of it looks legitimate -and- even may still run as it once did (in this example the malicious code would execute at the entry point and then jump back to the files original entry point so it could run as it normally did) try and look for anomalies and if possible cross reference the file in question with another version of the original file to find discrepancies.

More information on ClamAV signatures.

Thursday, April 19, 2012

YARA + Volatility ... the beginning

YARA - the sleeping giant.  There's been mention of it over the last few years but as far as adoption - I think it's still lacking in the tool set of many analysts. I personally like to leverage YARA on its own, within pescanner and most definitely within volatility's malfind.  I've recently encountered two obstacles (1) Converting ClamAV to YARA signatures and (2) How to process multiple YARA rule files.  If you take a look at page 26 of YARA's v1.6 User's Manual you'll see it outlines an option to include multiple rule files from within a single file (thanks Par).  In other words, if you use the standard syntax for calling YARA from the cli "yara /path/to/rules.yara <file>" you can't specify multiple rule files (without some foo of course).  Another prime example is within MHL's pescanner where you define the location of your rules file at the bottom, but again, a single rules file:


The above image shows the configuration within pescanner where you define the path to your YARA rules.  This particular example is taken from REMnux and is already filled out, generally it's left blank for your own configuration.

The use of the 'include' feature is one way of circumventing such a restriction because by placing this with the path to your other rule files to the top of the main rules file you're invoking, YARA will automatically process those additional rule files as well.  Here's an example of what I mean:

Simple and straightforward.  Just pop that syntax into the top of your main rule file and you're good to go.

So.. cool right?  Sort of... maybe useful if you have certain rule files you want to use for certain things, like pescanner, but I have a lot of files :/ . If you don't have many rule files then sure... but what if you have a bunch of different ones and foresee yourself continuing to split up or create new ones?  Having to constantly update the main rule file with an "include /path/to/new/rules.yara" every time just sounds like too much upkeep.  Say what..you don't see yourself having that many rule files for it to be a concern you say? ... Well what if, for example, you convert the ClamAV signatures to YARA rules?

The Malware Analysts Cookbook provides such a means with clamav_to_yara.py.  At the time of writing this there is an open issue with this script but there are a couple modified versions which work a bit better - still produce some errors, but not nearly as many.  There are a few tutorials out there on how to convert ClamAV signatures to YARA rules and it looks pretty straight forward, but I found some things have either changed or people just left out details.  If you have a fresh install of ClamAV you need to make sure you unpack its signature file before you can use the conversion script on it.  This can be done using ClamAV's sigtool:
$ sigtool -u /var/lib/clamav/main.cvd
which when complete will leave present you with the following:


Once you have the .ndb file you can proceed to converting as follows:
$ python clamav_to_yara.py -f main.ndb -o clamav.yara
Based on what I've encountered I believe depending on what version of the ClamAV signature DB you have and which version of the clamav_to_yara.py script you have, you may or may not get some signatures which YARA won't process.  I happened to get the problem child this time around and if get errors relating to invalid jumps etc. you can just remove those rules as needed since the errors are nice enough to tell you which lines it doesn't like.

The resulting file was ~18 MB of newly generated YARA rules based off the ClamAV signatures.. fwe.. that's a lot.  I tried multiple ways/attempts to get YARA to use this rule file but failed every time.  My assumption was that it's just too big to process in a timely manner like all of the other (smaller) rule files.  But I had a thought... so I started to split this big ol' file into smaller chunks and wanted to see at about what size would be ideal.  Finally at ~512K it seemed to be pretty fast and effective.  To split the file in an easy fashion you can use some form of the 'split' command... i.e :
$ split -d -b 512k clamav.yara
* if you split based on size like I did here you need to realize that it's going to cut the top/bottom signatures into pieces because you're only taking size into consideration and not splitting based on the signatures' structure. This can be easily fixed by going through each one and re-assembling just those two rules but if you don't do this, it's going to scream about the broken rules.

If you did the math, now you can see where I'm going.  This little workaround produced (33) YARA rule files and no, I don't want to add them all statically in case something changes.  When I'm doing some Volatility automation I usually define the path to my YARA rules in the beginning, i.e. :
YARA_Rules="/path/to/capabilities.yara"
but because of what we've just found out, a simple workaround to use instead is:
YARA_Rules=(`find /path/to/rules/ -type f -iname *.yara -exec ls {} \;`);
What this essentially does (in Bash) is point to the location where you keep all of your YARA rules and then it will list them all so that you don't have list them one-by-one... you can then parse them in an array:
for rule in "${YARA_Rules[@]}"; do 
and then pass them to the normal volatility syntax from within your volatility automation script, i.e:
YARA_Rules=(`find /path/to/rules/ -type f -iname *.yara -exec ls {} \;`);  

for rule in "${YARA_Rules[@]}"; do
   vol.py -f <mem.raw> --profile=<profile> malfind -Y $rule -D /path/to/dump/directory >> log
done 
Hopefully my troubles and workarounds will help someone else out there.. as always, ping me for feedback, tips etc.