Advanced malware protection: Application Whitelisting

Posted: April 8, 2011 in Security

Many security professionals agree that the signature based approach used by anti-virus, anti-malware and anti-spyware products, also known as ‘blacklisting’, doesn’t work. Blacklisting products tend to be in a constant state of chasing new types and examples and malware, usually a few steps behind. They are also inherently ill-suited to handle sophisticated malware, polymorphic threats (those that automatically alter their code) and recently exposed or published malware (0-day threats).

To the rescue comes a relatively new category of software product, application whitelisting. Sometimes called application control, this product category introduces a low level driver on client systems which monitors activity on the system and can prevent execution of programs that have not been approved in advance. The approval is a combination of multiple policy components delivered by a management server. Since the approach requires all executed software to be approved, it has no need to predict which software is malicious.

Also as a result, this type of approach and software has the potential to be much more disruptive, especially in organizations that support a relatively open client platform.

Once you understand the problem and identify a need, it’s time to take a look at potential products that meet the need. There are several vendors that deliver products in this space and they vary quite a bit in their approach, cost and complexity. As a result it helps to first establish your selection criteria. Key factors to consider should include:

  • Flexibility of policy controls – the primary challenge in implementing application whitelisting is the crafting of an approval configuration that meets constantly changing security objectives while minimizing disruption of constantly changing productivity tasks. In order to allow the organization to strike this delicate balance and maintain it over time, the selected solution must have great flexibility in configuring approvals. Typical mechanisms include approvals based on digital signatures, file metadata, file hash, file path, properties of running process, trusted software installers, trusted software directories and even external ratings of known applications.
  • Management and administration tools – as a key security system, application control solutions must complement protection capabilities with enterprise class tools for management and administration. Provided consoles, tools and APIs must be flexible and easy to use to support operators with different privilege levels, easy access to information and configuration controls and easy to use ‘master switches’ in the event of an emergency.
  • Monitoring and auditing – A vital complement to management tools is a robust auditing and monitoring capability. This capability can be delivered either within the solution or as ready integration interfaces into existing monitoring and auditing frameworks. The system must audit and record any administrative changes as well as key events on protected systems and agents. Monitoring compliance and current issues must be possible and delivered in a format that serves the needs of engineering and management staff.
  • Agent tamper protection – most organizations have many processes, including interactive user sessions, that run with administrative privileges on client systems. To prevent malicious code or curious users from disabling or tampering with the protective agent, the solution architecture must include sophisticated tamper protection to make such changes more difficult to make.
  • Operational modes – in my experience, all of the products in this space offer a ‘monitor only’ mode that allows the administrator to monitor the environment, determine how applications are used throughout the environment and assess what the impact of policy enforcement would be. In addition, almost all of the products I’ve reviewed also offer an enforcement mode that ensures that the client operates within the policy based framework. Some products differentiate themselves by offering more than one type of enforcement mode that allows the user to interact with the system (for example, prompting the user for action in some cases).
  • Supported platforms – in today’s IT environment, many organizations support multiple client operating systems and while they may not represent the same risk, most information security strategies strive to achieve parity in the security of supported platforms. Identifying a solution that addresses all support operating systems should be a key objective.
  • Vendor support and viability – since this product category is very young, the amount of available public knowledge on the technology and leading products is minimal and any implementation will rely heavily on the vendor. It is therefore important to make sure that the vendor will continue to be around and provide the required level of support.

When looking for products to include in the selection process, a good place to start is this set of introductory articles from InfoWorld magazine: The article and associated reviews are somewhat out of date so the details about each product should be validated using additional sources but the article provides a good starting point to identify the key vendors in this space. For a recent effort, I focused on the following three products:

  • Microsoft AppLocker – For organizations that are either on or migrating to current Microsoft operating systems (Windows Server 2008 R2 and Windows 7), this option is attractive primarily because it is free. AppLocker is a component of these operating systems and is easy to configure using group policies. Another strength is a powerful approval configuration mechanism that leverages digital signatures and file metadata to provide granular control for approving applications that are properly signed and contain metadata.
    The product’s primary weakness is a lack of administration console and poor visibility into compliance and auditing – the data is there in the event logs of each client system but there is no easy native way to collect and present the data.
  • Bit9 Parity Suite – The solution from a leading specialty vendor in this space provides flexible policy controls and an excellent searchable  knowledge base called FileAdvisor that provides sophisticated ratings for millions of applications based on several factors to help determine if the application is malicious or safe.
  • McAfee Application Control – This product is the result of an acquisition of SolidCore Systems in 2009. The product approaches the solution from the perspective of creating an initial trusted configuration of a system and closely managing any changes from that point forward. This approach is a great fit for servers and for environments that allow a restricted number of applications on client systems.

After selecting the product that best fits your needs, the project can start. I recommend focusing on the following key high level tasks:

  • Platform design – this system will be collecting information about any application that is written and executed on all managed systems. Since the volume of data can be significant, the system must be designed and architected correctly. In most cases only 1 or 2 servers are required but the capacity, placement and component configuration should be discussed with the vendor, designed on paper and then tested in a lab environment.
  • Approval configuration – all of the solutions in this space have several mechanisms to approve applications. Work with the platform in a lab/test configuration in order to understand the behavior of each mechanism and design the configuration of the platform to meet security objectives and business requirements. I recommend focusing on the ability to identify and correctly configure applications and processes that will automatically create or deploy other approved applications – compilers, software distribution systems, updaters from Adobe/Google/Mozilla, etc.
  • Process design – due to the potentially disruptive nature of an application control system, the administration and support processes around the platform must be designed carefully and thoroughly to ensure a successful project. The processes must involve teams from support departments, client engineering, security and IT management. The interaction between users and the systems must be clearly understood and incorporated into proposed processes. Input from various business departments and users should be solicited to make sure that different needs, job responsibilities and user environments are identified and addressed by the solution.
  • Client deployment and monitoring – the first step in testing the proposed design and configuration in the ‘field’ is to deploy the client agent to managed systems in a ‘monitor only’ mode and taking the time to collect and analyze the resulting data while stabilizing the infrastructure. Leverage the information collected in this phase to validate assumptions and adjust plans as needed.
  • Enforcement testing and pilot – planning is vital but it can only take you so far in determining the best way to approach a deployment and the impact to your user community. The rest must come from careful testing and a staged deployment. Carefully select early adopter users to ensure that there is a good distribution of test scenarios. Over-allocate resources for the testing and pilot phases to ensure that the phases are concluded quickly and that participating users have a positive experience with the system and the support processes. And finally, when needed, make changes and corrections to designs, plans, configurations and processes to incorporate lessons learned from early adopters. Assume that this feedback cycle will continue throughout the deployment of the system and its lifecycle

Hopefully you can use this information to help kick start your whitelisting project (I know it would have been useful to me at the start of some recent projects) and please comment if you have any questions or anything to add.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s