Test Methodologies
Test Methodology: Software-Defined Wide Area Network (SD-WAN) v3.0
Q4 2020
Test Methodology: Enterprise Firewall V1.0
Q1 2021
Test Methodology: Secure Access Service Edge (SASE) – Part 1 Firewall as a Service (FWaaS)
Q1 2021
Test Methodology: Web Browser Protection - Phishing & Malware Download
Jul 13, 2021
Ratings Matrix
A Product Rating is a forecast, an expert opinion about a product’s capacity to meet its obligations to consumers over time. Product ratings inform consumers — enhancing transparency and enabling them to focus on considerations that are most critical to their organizations.
Ratings range from AAA – D. A product rated ‘AAA’ has the highest rating assigned by CyberRatings.org. The product’s capacity to meet its commitments to consumers is extremely strong. A product rated ‘D’ is actively being breached by known threats and is unable to protect consumers.
Ratings offer forward looking guidance on a product’s ability to meet future commitments to customers and is based on multiple factors including technology and business leadership, employee turnover, customer satisfaction, financial strength, test results, and market conditions. Test results included security effectiveness, performance, SSL/TLS functionality, management, and customer feedback.
Rating Definition
AAA
A product rated ‘AAA’ has the highest rating assigned by CyberRatings.org. The product’s capacity to meet its commitments to consumers is extremely strong.
AA
A product rated ‘AA’ differs from the highest-rated products only to a small degree. The product’s capacity to meet its commitments to consumers is very strong.
A
A product rated ‘A’ is somewhat less capable than higher-rated categories. However, the product’s capacity to meet its commitments to consumers is still strong.
BBB
A product rated ‘BBB’ exhibits adequate stability and reliability. However, previously unseen events and use cases are more likely to negatively impact the product’s capacity to meet its commitments to consumers.
A product rated ‘BB,’ ‘B,’ ‘CCC,’ ‘CC,’ and ‘C’ is regarded as having significant risk characteristics. ‘BB’ indicates the least degree of risk and ‘C’ the highest. While such products will likely have some specialized capability and features, these may be outweighed by large uncertainties or major exposure to adverse conditions.
BB
A product rated ‘BB’ is more susceptible to failures than products that have received higher ratings. The product has the capacity to meet its commitments to consumers. However, it faces minor technical limitations that have a potential to be exposed to risks.
B
A product rated ‘B’ is more susceptible to failures than products rated ‘BB’; however, it has the minimum capacity. Adverse conditions will likely expose the product’s technical limitations that lead to an inability to meet its commitments to consumers.
CCC
A product rated ‘CCC’ is susceptible to failures and is dependent upon favorable conditions to perform expected functions. In the event of adverse conditions, the product is not likely to have the capacity to meet its commitments to consumers.
CC
A product rated ‘CC’ is highly susceptible to failures. The ‘CC’ rating is used when a failure has not yet occurred, but CyberRatings considers it a virtual certainty.
C
A product rated ‘C’ is highly susceptible to failures. The product is expected to fail under any abnormal operating conditions and does not offer a useful management systems and logging information compared with products that are rated higher.
D
A product rated ‘D’ is actively underperforming and failing and does not meet the use-case. The ‘D’ rating is used when the product is not operational without a major technical overhaul. Unless CyberRatings believes that such technical fixes will be made within a stated grace period (typically 30-90 calendar days), the ‘D’ rating also is an indicator that existing customers using the product have already experienced a failure and should take immediate action.
TESTED
Phishing and Malware tests for Web Browser Security ran for 20 days, four times per day, totaling 80 discrete test runs. The reports include measurements of protection against fresh new attacks, consistency of protection over time, and how effective the browser protection was overall. Web Browser protection is free, and consumers should take advantage of it.