ZBTrap: a Virtual Appliance for Botnet Detection

I have created a Virtualbox VM named "ZBTrap" (shortened from ZomBie Trap), which can be used to detect spam sending computers with greylisting. Anyone can download and configure this VM, and then start detecting malware-infected computers themselves.

ZBTrap is licensed under Creative Commons Attribution-ShareAlike 4.0 International License.

Warning: Even though this appliance seems to work correctly when I create and test it, I can not guarantee it will work for you. Use at your own risk.

Prerequisite: ZBTrap's operation is based on greylisting, so one or more heavily spammed domains with no active mail accounts are necessary for it to work. All incoming mail will be discarded, so don't use this for any domain which you intend to receive mail.

Built-in accounts/password:

root/cbf7b598 (root account)
zbtrap/8ad49a99

"zbtrap" is the account used for retrieving IP list of suspected zombie computers and associated spam mail headers.

Here is how to set up your own ZBTrap:

• Download the "ZBTrap" virtual appliance (download link here). That file is almost 600MB, so be patient.
• The appliance is in OVA format. Import it (File/Import Appliance...) into your Virtualbox manager.
• Configure the network setting of your ZBTrap so that it is able to accept incoming SMTP connections. There are many ways to achieve this. When I tested it at home, I configured my ZBTrap to be "bridged" within my Virtualbox, so that it gets its own IP address. Then I set up my gateway to forward TCP port 25 (SMTP) of its external IP to my ZBTrap.
• Start your ZBTrap VM. For each domain you use to detect botnets, you have to add one line to /usr/exim/grey_domains as the root user (here I assume that the domain to be added is "spammeddomain3.com"):
  
  echo spammeddomain3.com >> /usr/exim/grey_domains
  
  likewise, one line has to be appended to /usr/exim/domain_aliases, too:
  
  echo "*@spammeddomain3.com: greyhole" >> /usr/exim/domain_aliases
  
  
• Point the MX of your mail domain(s) to your ZBTrap, and it should be the sole MX for those domains. In my case described above, I created a domain name mapping to the public IP of my gateway, then used that for the MX.
• Start the SMTP daemon with the root account by issuing this command:
  
  /usr/exim/bin/exim -bd

Now your ZBTrap is ready to collect malware-infected computers' IP addresses.

The VM's timezone is currently set to UTC, which is important when you want to generate the IP list of the malware-infected computers for the previous day. Follow the procedure below to get the information you want.

• If you want the IP list for a certain day, issue the following commands the next day as the zbtrap user.
  
  cd;
/bin/sh ~/bin/get_expired_entries.sh > ~/expired_list.txt
  
  The file "expired_list.txt" contains the md5 string, message accepting time in UNIX epoch and source IP of expired messages (blocked by greylisting) from the previous day. If you are only interested in botnets' IPs, the 3rd column is what you want.
• If you want to report these suspected bots to someone who can check and clean them, you will need the mail header. I have recorded the header data in the mail server's mainlog file, located under /var/spool/exim/log. At 00:00 UTC, log rotation script will rename the old mainlog to mainlog.01, which is our input data. Issue the following commands to match the expired_list's md5 strings to the corresponding mail headers from mainlog.01, and print them out.
  
  cd;
< /var/spool/exim/log/mainlog.01 awk -v expired_list=expired_list.txt -f bin/generate_daily_logs_gl.awk > ~/conn_log.txt
  
  To view the mail headers from IP xxx.yyy.zzz.www, issue the following commands.
  
  cd;
< ~/conn_log.txt grep xxx.yyy.zzz.www | awk -f bin/show_mail_header.awk

If you encounter any difficulties when using ZBTrap, please post your problem here. I will do my best to answer them.

How Many Bots Can a Greylisting System Detect?

According to a paper presented at TANET 2005（Taiwan Academic Network Conference 2005), titled "Spam Filtering with Open Source Software, and Some Hard Facts from NTU's Email System" (title translated by me, not necessarily correct), their daily mail volume was 800K, of which 58 per cent were blocked by greylisting.

I myself detected 3651 suspected bots while blocking 211178 spams yesterday, so if the folks at NTU (National Taiwan University) had a similiar bots/spam ratio, they would be able to detect (800K * 58% * 3651 / 211178) = 8021 bots daily in 2005. Though I believe greylisting should do much better than my current detection setup.

Now imagine what will happen if some large greylisting users, like Texas A&M University, SpamCop, and Computer Science in Aarhus (DAIMI), all contribute their logs: tens of thousands of bots will be uncovered every day.

For those of you interested in the paper mentioned above, remember that it is written in Chinese. If you can not read Chinese yourself, don't forget to find someone who can to help you:).

Botnet Detection with Greylisting

So how do we uncover botnets? If you google for botnet tracking, you will find that honeypots are often listed in the first page of search results. The strength of honeypots lies in their effectiveness at collecting malware binaries, which are needed to gain a deeper understanding of botnets. As I only track botnets so as to notify unsuspecting victims, and have no interest nor expertise to study the inner working of malware, honeypots are not really made for me. How to attract botnets to interact with honeypots is also a problem.

As I said before, my botnet detection strategy is to follow the spam upstream. According to the Q2/June edition of the MessageLabs Intelligence monthly report, 83.2% of all spam was sent via botnets. The fact is, botnets has accounted for more than half of global spam for several years. Isn't that convenient for us if we want to find botnets? If you manage your own mail server, and get a lot of spam every day, perhaps your server have been interacting with botnets all the time. The remaining problem, is how to identify botnet computers.

Thanks to greylisting, mail servers can easily filter out incoming spam from botnets. The SMTP engines built within malware often are not full blown SMTP servers, as malware authors tend to cut off the retry function of SMTP protocol. Greylisting takes advantage of that and is able to differentiate botnet computers from real mail servers by their lack of retry behavior. To extend from this, mail sending hosts which could not pass greylisting are very likely to be botnet computers, which are exactly what we look for.

Greylisting is a very powerful botnet tracking technique. Once a botnet computer begins to send out malicious mail, be it spam, virus or phishing mail, it will soon be detected by greylisting. If mail servers deploying greylisting could contribute their mail logs to compile the IP address list of suspected botnet computers, the useful life of botnets to cyber criminals will be greatly shortened, which will eventually lead to the demise of botnets. At least it should reduce bot-sent spam to a minimun, I hope.

I have posted my greylisting implementation, comments or questions are welcomed.