Ratings in CyberRatings.org Tests

When CyberRatings.org tests products, we begin with a methodology that is published before the test. After extensive testing of a product, the test report will show a variety of important metrics on how a product defended against exploits, how many evasions could bypass protection, and if the device would remain stable under adverse conditions. An overall Protection Rate (Security Effectiveness) score for the tested product is the outcome of a number of key metrics including exploits, malware and evasions, TLS/SSL functionality, and product stability. Rated performance is based on connection rates, transaction rates, the number of simultaneous sessions, unencrypted raw data (UDP), HTTP, and HTTPS using the top cipher suites. When appropriate, we run other real-world applications.

At the end of the test, a Comparative Report is written with a Security Value Map™ (SVM) showing the test results for all products in the test and how they performed against the other products with the focus on Security Effectiveness, Performance and Total Cost of Ownership (TCO). This view for the enterprise has three ratings: Recommended, Neutral and Caution. Typically, those products receiving the highest Security Effectiveness scores would be Recommended.

There will be certain times during a test cycle that CyberRatings will publish individual test reports before a Comparative Report is produced. In those instances, a Rating may be given based on the overall score in the test. A Rating is calculated on a scale from 0 to 800, based Security Effectiveness, Performance, Functionality, and Management.

Rating Scores

A Product Rating is a forecast about a product’s capacity to meet its obligations to consumers over time. Product ratings inform consumers — enhancing transparency and enabling them to focus on considerations that are most critical to their organizations. Ratings range from AAA – D. A product rated ‘AAA’ has the highest rating assigned by CyberRatings.org. The product’s capacity to meet its commitments to consumers is extremely strong. A product rated ‘D’ is actively being breached by known threats and is unable to protect consumers. Ratings offer forward looking guidance on a product’s ability to meet future commitments. Test results included security effectiveness, performance, TLS/SSL functionality, management, and customer feedback.