The modern botnet is one of the most powerful attack techniques available to today’s cybercriminal. Conceptually, a botnet is a collection of compromised workstations distributed over the public internet that leverages the untapped processing power of a multitude of endpoints, usually to accomplish a malicious agenda.

Each of these endpoints or 'bots' typically link back to a command & control (C&C) server and the whole botnet can be used to power huge DDoS (distributed denial of service) attacks, as well as undertake data theft/fraud or spam-marketing on a mass scale.

Because of their sheer size and the difficulty involved in detecting them, botnets can operate under the radar for long periods of them. As an example, the Zeus botnet operated for over three years in this fashion, netting the perpetrators an estimated $70 million in stolen funds before the FBI arrested over 100 individuals in 2010.

And it wasn’t until March, 2012 that Microsoft announced it had finally succeeded in shutting down the 'majority' of Zeus’ C&C servers, though the botnet itself still has not been fully eradicated.

> See also: How Anthem was breached – and how you can prevent it

As you might have guessed from the length of Zeus’ tenure – which is still ongoing – organisations that own compromised workstations often aren’t even aware of this until considerable damage has been done.

Over time, the number of botnets has grown significantly and increased in value while also becoming more sophisticated in their targets, infiltration, anti-detection, and attack techniques. So today, it’s increasingly important for security professionals to be well versed in a variety of botnet detection techniques and tools.

Botnet detection: ferreting out one or more bots on your network

There are several initial signs and symptoms that can help IT teams recognise that a botnet may have infiltrated their network. These often manifest shortly after botnet infiltration as the compromised machine begins executing instructions.

Signs of botnet infilitration can include : linking to established C&C servers to receive instructions; generating Internet Relay Chat (IRC) traffic via a specific range of ports; generating simultaneous identical DNS requests; generating Simple Mail Transfer Protocol (SMTP) traffic/e-mails; geducing workstation performance/Internet access to the point it is obvious to end users

Host-based botnet detection begins with client-side anti-viral solutions, since the infiltration itself nearly always happens via malware. Unfortunately, anti-viral technology that relies on signature-based detection alone will fail to identify new variants of malware simply because that exact code hasn’t been seen/researched before.

Host-based botnet detection includes monitoring of things like rootkit installations, unexpected pop-ups while browsing over HTTP (though this may simply be spyware), or any sudden change to the Windows Hosts file, which can be used (or abused) to restrict outbound server access.

Also, of course, if the default DNS servers have been modified, then that’s likely a sign that traffic is going places that the organisation doesn’t want it to go.

Botnet detection on the network

Network-based botnet detection is a bit more complex. One approach lies in detecting and monitoring internet relay chat (IRC) traffic, which in normal circumstances shouldn’t exist on a company network.

IRC traffic is also sent unencrypted, meaning that keywords can be detected with a packet sniffer. The default IRC port is 6667, but the entire port range (from 6660-6669 and 7000) could be utilised by bots.

> See also: The three golden rules for software security in the IoT

Best practise guidelines for effectively monitoring for botnet threats on the network include deploying both host and network-based botnet detection tools, ensuring your host-based IDS or anti-malware solution is capable of detecting the common endpoint signs of bot net infection and is frequently updated with the latest C&C server information, and implementing a honeypot (or several) if necessary.

Static vs. behavioral botnet detection

Botnet detection falls into two categories: Static Analysis and Behavioral Analysis. Static analyses are simplistic, fast, and resource friendly. Behavioral analyses are more thorough, but also more resource-intensive.

Static analysis is your first line of defense because it looks for a highly specific match to something like a malware signature or specific executable or C&C connection address. Unfortunately, this approach often doesn’t work, because if there is any alteration to a signature, however minute, the malware will not be detectable through this method.

For this reason, behavioral analysis is an essential approach to botnet detection as well. For instance, often factors like the timing of an attack is a dead giveaway; a C&C server usually orders bots to take specific actions and this generates enormous network activity at a single point in time (usually, of the types described above under network-based detection). Suspicious behaviors can be detected with SIEM / Network IDS rules to expand an organization’s botnet detection capabilities.

One development in botnets that throws a wrinkle in detection methods is the rise of a P2P management architecture. This works in a decentralised way, such that there is no central C&C; bot commands are instead issued by peers.

Such botnets are harder to detect, though infected bots will usually act in much the same ways as in a traditional botnet because the bot herder still has the same goals.

While attack campaigns in the past might have passed up 'lower priority' systems and devices that do not store payment or other sensitive information, we are seeing botnets designed specifically to go after them.

We can attribute this to today’s lower compute cost as well as the pervasiveness of devices deployed with vendor default passwords such as network devices, wireless access points, and even CCTV cameras.

When it comes to best practises in terms of static vs. behavioural analysis of botnet detection, use static analysis at a minimum and also behavioural analysis if possible, talk to in-house and external analysts about P2P botnet detection techniques, and ensure the rules for your behavioural network-based botnet detection system take into account less common systems.

Botnet tools and the future of botnet detection

The news isn’t all bad. As botnets have evolved, so too have the tools to detect and eradicate them. Today, focused open-source solutions like Snort and more integrated security intelligence offerings can help with determining when network activity is unusual in predefined ways, identifying its network origin, analysing its nature and impact, and directly quarantining, limiting or eradicating local bots.

> See also: DDoS ransom notes: why paying up will get you nowhere

And going forward, botnet detection solutions are getting smarter – fast. This is happening in a variety of ways, some tech-centric (such as machine learning being implemented for botnet pattern recognition), some human-centric, and some that combine the two.

Encouraging the IT security community to work together by exchanging information on cyber threats is our best opportunity to stay one step of the attackers and successfully defeat the threats posed by botnets.

Sourced from Garrett Gross, Senior Technical Product Marketing Manager, AlienVault