@ISACA Volume 7  4 April 2018

Five Key Considerations When Evaluating Security Control Effectiveness


John P. Pironti “Trust but verify” is a concept and approach that should be applied when evaluating security control effectiveness. It is often the case that organizations inherently trust the security controls they implement without verifying their capabilities or consistently monitoring and appreciating both their efficacy and limitations. It is inevitable that security controls will depreciate in their effectiveness of mitigating the risk they are intended to address over time if they are not properly monitored, maintained and enhanced.

Information security is an evolutionary concept where defenders are challenged by increasingly sophisticated attacks carried out by motivated and capable adversaries. These individuals are constantly developing methods, practices and techniques to circumvent both the detective and protective security controls that organizations rely on to protect them. To combat this threat, organization’s must evolve and tune their security controls to adapt to the ever-evolving threat and vulnerability landscape. The following are 5 key considerations when evaluating security control effectiveness:

  1. Are security controls designed properly—The design of a security control is the key factor that establishes its effectiveness in mitigating security risk. Security controls should be mapped to clearly defined and achievable control objectives that identify their intended benefits to the security posture of an organization, risk factors they are expected to mitigate and realistic expectations of their capabilities. This information allows security control designers to use an informed approach when ensuring that their security risk management goals and expectations are being met with the controls they introduce.
    A common oversight in the design of technical security controls is to create controls that cannot be implemented or maintained by technical operations staff. What may look good on a whiteboard or security control design document may not work in operation. It is important to ensure that staff who will be responsible for the ongoing operation of the security controls are included the design process. This will allow them to provide informed insights into the implementation, operation and sustainability of the security controls that they will ultimately have to maintain.
  2. Monitoring security control effectiveness—Security controls should be monitored and measured for their effectiveness on a continuing basis. Their effectiveness is likely to change over time as both operating environments and attacks evolve and change. To monitor effectiveness, clearly defined metrics and associated reporting for both the success and failure of security controls should be defined as part of the design process.
    An example of a useful metric for the effectiveness of a security control is its ability to support the reduction of the risk it is intended to mitigate to the defined target level (i.e., high risk to low risk). This is often represented in reporting as the residual risk level once the security control is in place. This type of metric can assist organizations in identifying and evaluating the value provided by the security control on an ongoing basis.
    It is also important to recognize that the configuration and management of security controls will need to be adjusted as organizations and the threats they face change and evolve. When designing security controls, it is important to understand how they can be can be used appropriately at the time of implementation and how they can be adjusted in the future. The design of security controls should be revisited to ensure their effectiveness as part of an organization’s change management processes and should be reviewed for effectiveness on an annual basis at minimum.
  3. Security instrumentation—Security instrumentation tools can assist an organization in the assessment of the effectiveness of its technical security controls. These tools continuously simulate known or expected attack behaviors and activities against the technical security controls. The results of this testing provide valuable intelligence to the organization and assists it in answering vital control effectiveness questions, such as what portions of the attack can the existing security controls identify and block? What information would be generated by attacks by detective security controls and capabilities? And how effective are the organization’s security event and incident response processes and capabilities in defending against probable and realistic attack scenarios?
    For this type of security control testing to be effective and yield the maximum benefit, it is important to ensure these activities are performed in the context of an organization’s actual operating environment if possible. The use of tabletop exercises or simulated environments can introduce inaccurate testing due to the possibility of missing components or assumptions of components that are in place vs. what is actually in place. This results in misrepresentations of the security controls’ ability to effectively mitigate the intended risk.
  4. Assumption vs. assurance—A common mistake that often occurs during the evaluation of security controls is assuming they are effective by their mere existence. This is often based on the assumption that the security controls have been properly designed, implemented, maintained and are performing their intended activities. Without regular and continuous positive assurance testing and verification, it is not possible for an organization to have independent verification that its security controls are operating as designed and are providing effective risk mitigation.
    In many cases, organizations rely on targeted penetration tests, vulnerability scanning and security assessments to provide assurance of the effectiveness of security controls. While these techniques can be effective, they are often conducted on a point-in-time basis with significant windows between testing periods. They also may be targeted in nature and do not provide a comprehensive evaluation of the entire security stack. In today’s quickly changing threat and vulnerability landscape, it is important for organizations to have regular insights and intelligence about the effectiveness of their security controls.
    It is also important for organizations to implement assurance testing to ensure business activities and technical operations do not create new vulnerabilities. For instance, as part of a change management action, a network firewall may have a rule added that instructs it to restrict particular network traffic. At the same time, another rule with a higher priority may already exist in the firewall policy that allows this same type of traffic to pass and it supersedes the new firewall rule. The security control designer may not be aware of these conflicting rules and may assume once the new rule is in place that it is working as expected. Unless an assurance test exists to test effectiveness of the firewall rule after its implementation, the control designer may not be aware that this condition exists.
  5. Security controls supported by threat and vulnerability analysis—When considering security control effectiveness, one key question that should always be asked is what risk is intended to be mitigated or reduced through the use of the security control? An effective way of answering this question is to use threat and vulnerability analysis. This type of analysis uses scenario-based analysis to identify the threats and vulnerabilities that have both a high likelihood of occurrence and, if realized, a materially negative impacting result.
    The security controls that are implemented should be mapped to this analysis. This will demonstrate how they will assist in the mitigation of the risk that would result from the realization of identified threats and vulnerabilities that concern the organization. This information can then be used by the organization to make informed decisions on the implementation of new security controls or adjustments to existing ones.

There are many ways to evaluate security control effectiveness. Ensuring proper design; continuously monitoring effectiveness; and using security instrumentation rules, assurance testing, and threat and vulnerability analysis all can be used by an organization to verify current security controls capabilities, efficacy and limitations. While even the best security controls may still be subject to attack, if controls are properly monitored, maintained and enhanced, they will continue to mitigate risk to the best level possible.

John P. Pironti, CISA, CRISC, CISM, CGEIT, CISSP, ISSAP, ISSMP, is the president of IP Architects LLC.


Webinar: Cyberthreat Intelligence Best Practice Defined

Webinar:  Cyberthreat Intelligence Best Practice Defined
Source: Victor
Habbick Visions;
Getty Images

Threat intelligence is an ongoing challenge in today’s networks. If threat intelligence is not consistently updated and accurate, it impacts the effectiveness of incident response. What can you do to minimize incomplete and stale threat intelligence that can reduce the efficacy of your organization’s security operations?

To help you learn the best solution to eliminate silos and accelerate threat response in your organization, ISACA and Infoblox have teamed together to present the “Best Practices in Threat Intelligence for Containing Cyber Threats” webinar. This webinar will address threat intelligence best practices and next steps for your organization to take. This webinar takes place on 17 April at 11AM CDT (UTC -5 hours). ISACA members can earn 1 continuing professional education (CPE) hour by attending this webinar and completing a related survey.

Anne Hiller, senior director of strategic partnerships at Farsight Security, and Jonathan Abbe, senior product manager at Infoblox, will lead the webinar. They will use their combined cybersecurity experience to illustrate today’s best practices for threat intelligence and cyberresponse.

To learn more about this webinar or to register for it, visit the Best Practices in Threat Intelligence for Containing Cyber Threats page of the ISACA website.


Tech Brief: The Impact of Networked Biomedical Devices


Source: uchar;
Getty Images

Data-driven biomedical devices monitor and manage patient care functions via permanent or temporary connections to the Internet, a mobile application (app) or to a healthcare facility’s network. Connecting to a network can impose certain risk and, late in 2016, the US Food and Drug Administration (FDA) released healthcare industry security controls guidance to manage the risk and vulnerabilities associated with these devices. The ISACA Tech Brief: Networked Biomedical Device Security serves to explain how these devices benefit patients and also bring about a new required security awareness for healthcare organizations.

This complimentary tech brief details to the lay person how networked biomedical devices can compromise patient safety and security, and details considerations to keep in mind when evaluating their impact. Networked biomedical devices include computerized tomography (CT) scanners, magnetic resonance imaging (MRI) machines, cardiac monitors, infusion pumps, ventilators, dialysis machines and even hospital beds. These devices serve a significant life-saving or life-supporting purpose for hundreds of thousands of patients across the globe, but the very aspects that makes them effective, including their network connectivity and copious data collection, are also the sources of their potential vulnerabilities, which can pose risk to both the medical professionals and patients who depend on them.

The networked biomedical device tech brief, like other tech briefs in the series, is intended to offer a quick overview of a topic at a nontechnical level. Tech briefs are a great resource for IT professionals to use when educating their business partners on the basics of a technology that might hold potential in their industry.

To learn more and download this tech brief, visit the ISACA Tech Brief: Networked Biomedical Device Security page of the ISACA website.


Prep for the CSX Exam


Immerse yourself in the 40-hour Cybersecurity Nexus™ (CSX) Practitioner Exam Prep Course to build and hone critical cybersecurity skills you can use in the next level of your career. Because the course is conducted in an adaptive, live cyberlaboratory environment, learners are able to practice applying industry-leading methods and learn real-world cybersecurity technical skills. Students will learn to utilize the latest open-source tools and will have access to Packet Hunters Evalu8R, a learning and development tool that measures a professional’s ability to perform specific cybersecurity job tasks and enables the instructor to provide the student with immediate feedback.

To learn more about the CSX Practitioner Exam Prep Course, please visit the CSX Practitioner Exam Prep Course page on the CSX website.