Independent tests show why orgs should use third-party cloud security services  – CyberScoop

Businesses don’t always get what they pay for in cybersecurity. Some of the most expensive cloud network firewall vendors are among the worst performers against exploits and evasions, according to the most comprehensive, independent testing CyberRatings.org has conducted to date.

Cisco, by far the most expensive cloud network firewall offering across the top 10 vendors on price per megabits per second, ranked seventh with an overall security effectiveness score of 53.5%, according to CyberRatings.org research released Wednesday. 

The trio of big cloud providers — Amazon Web Services, Microsoft Azure and Google Cloud Platform — fared even worse, each landing at the bottom of the pack with a 0% security effectiveness score. 

“We’ve been told to use cloud-native technologies, that they’re better suited than using bolt-ons. Well, that’s clearly not the case here,” CyberRatings.org CEO Vikram Phatak told CyberScoop.

“Any of the third-party firewalls you pick are going to be better at protecting you than what you have today with the AWS firewall, but also frankly Azure and GCP today as well,” he said.

Fortinet and Check Point earned the highest rating of 100%, followed by Versa Networks, Palo Alto Networks and Juniper Networks — each landing in the upper end of the 99th percentile, according to CyberRatings.org’s tests. Forcepoint’s security effectiveness score was 96.6%.

CyberRatings.org tested cloud network firewalls against more than 2,000 widely exploited vulnerabilities. The nonprofit, which paid for the tests and research in Q1 2025 without any vendor involvement, then applied 2,500 attacks spanning 27 evasion techniques across multiple network layers to bypass firewall defenses.

“This is what I consider to be the equivalent of an open-book test. It’s not super hard stuff,” Phatak said. 

“We want to know what a buyer, purchaser of the technology can count on in an adversarial situation where things are not always going their way,” he said. “This is not a Category 5 hurricane, and it’s also not a sunny day on the beach.”

CyberRatings.org’s tests showed wide disparities in cloud network firewalls’ ability to defend against publicly available exploits. Protecting organizations against exploits is the first line of defense, a core selling point and purpose of firewalls. 

AWS performed the worst on this front, blocking only 0.59% of exploits. The big problem for AWS is that its signature set for exploits is mismatched, Phatak said.

“If you put all your eggs in the AWS basket, you’re going to end up regretting it from a cybersecurity perspective at least,” Phatak said. 

Rounding out the bottom of the field, Microsoft Azure blocked 55.28%, Cisco blocked 90.68%, GCP blocked 96.6% and Forecepoint blocked 97.63% of exploits. Fortinet and Check Point blocked all of the exploits CyberRatings.org threw at their cloud network firewalls. Versa Networks, Juniper Networks and Palo Alto Networks each scored in the high 99th percentile on exploit prevention.

The overall results and rankings diverged further when CyberRatings.org measured cloud network firewalls’ performance against evasions.

Cisco, AWS, GCP and Microsoft Azure each failed to defend against evasion tactics between layer 3 and layer 7, network traffic originating from IP addresses and the content of application data.

Ultimately, the 0% security effectiveness score applied to AWS and GCP was due to the ease with which CyberRatings.org bypassed their firewalls with evasions. Both vendors earned a 0% score in preventing evasions.

Microsoft performed better than its cloud counterparts on evasions, scoring 78%. Yet, Microsoft’s “big issue is that if anything comes across encrypted with HTTPS, they’re blind. [It’s] the only firewall that doesn’t have HTTPS decryption built in,” Phatak said.

Microsoft’s lack of transport layer security (TLS) and secure sockets layer (SSL) support resulted in its overall 0% security effectiveness score, according to CyberRatings.org’s benchmarks. Cisco prevented 59% of CyberRatings.org’s evasion tests.

Forcepoint blocked 99% of evasions while Palo Alto Networks, Check Point, Juniper Networks and Versa Networks all blocked 100%, according to CyberRatings.org’s tests.

CyberRatings.org explained its testing framework, including why and the extent to which it deducted points from firewall vendors’ score across all categories tested. In many cases, it was the combination of exploit and evasion prevention tests, and other factors unique to specific factors that resulted in low security effectiveness scores.

In the case of AWS, its firewall didn’t block any live attacks, so CyberRatings.org couldn’t test it against evasions. With Microsoft’s firewall, CyberRatings.org evaded defenses by encrypting traffic or targeting a web server that’s encrypted.

Phatak directed his harshest criticism at AWS, which has consistently performed poorly in CyberRatings.org exploit prevention tests since 2014. “Amazon’s lack of improvement was shocking to us,” he said. “It just says that it’s not taking this seriously.”

The post Independent tests show why orgs should use third-party cloud security services appeared first on CyberScoop.

  –

Read More  – CyberScoop