The Data Quality Crisis: Your Fragmented Security Stack is Lying to You

A recent analysis of enterprise security tools revealed a shocking truth: each tool’s data accuracy varies wildly depending on what it’s measuring, and none tell the complete story. This means that there are gaps in many organizations’ attack surfaces, and especially for organizations that rely heavily on connected devices and the Internet of Things (IoT), which is often a blind spot for vulnerability scanners.

The numbers bear this out; 58% of enterprises say detecting vulnerabilities is getting more difficult as their attack surface increases in size and complexity. This is a data problem: with some parts of their attack surface invisible to scanners, it’s not surprising that vulnerabilities are going unseen. For defenders, catching all vulnerabilities with limited visibility is like trying to get to a destination with only part of a map. 

What is the Data Quality Problem?

This is the crux of the data quality problem; fragmented cybersecurity tools often don’t have the full picture when it comes to an organization’s assets. Security tools — like vulnerability scanners and EDR agents — collect enormous amounts of data from networks, endpoints, and applications. But if that data is wrong, outdated, or misunderstood, any insights built on top of it will also be flawed. For example, a vulnerability scanner might misidentify a device’s operating system or miss a critical patch, leading security teams to prioritize the wrong risks or overlook real threats.

Poor data quality also causes fragmentation: different tools may report conflicting information about the same asset, or label risks using incompatible terms. This makes it difficult for analysts to get a single source of truth about an organization’s security posture. It can lead to wasted effort, duplicated work, or false confidence in the organization’s security stance.

What is a Vulnerability Scanner? 

A vulnerability scanner is a tool that automatically searches computers, networks, and connected devices for weaknesses that attackers could exploit.

Scanners collect information about systems, such as software versions, open ports, and configurations, and compare that data against databases of known vulnerabilities. The scanner then produces a report highlighting potential risks, ranking them by severity, and often suggesting ways to fix them. Vulnerability scanners are essential for identifying security gaps before hackers find them, but they only report the issues. They don’t fix the issues themselves. 

What is an EDR Agent? 

EDR agents (Endpoint Detection and Response agents) are specialized software programs installed on endpoints such as laptops, desktops, servers, or cloud instances. EDR agents continuously monitor, detect, and respond to security threats. Unlike traditional antivirus software, which primarily scans for known malware signatures, EDR agents are designed to:

1. Detect Suspicious Behavior: They use behavioral analytics, machine learning, and threat intelligence to identify abnormal activity that might indicate malware, ransomware, or intrusions.
2. Record Forensic Data:  They capture detailed event logs and process histories for investigation and threat hunting.
3. Respond to Threats: Many EDRs can isolate infected endpoints, kill malicious processes, or block suspicious network activity in real time.
4. Integrate with Security Operations: They feed data into SIEMs, SOAR platforms, and central dashboards, enabling coordinated incident response across the enterprise.

What Vulnerability Scanners Get Wrong

Scanners’ accuracy depends on the quality of the data they collect. Even though they can perform deep scans, vulnerability scanners can — and do — make mistakes. 

Scanners Can Make Assumptions

Scanners typically collect information from limited sources, which can lead to assumptions about what’s running on a system. For example, if a device responds like a Windows 10 machine, the scanner may assume it’s vulnerable to every Windows 10 CVE, even if patches are already applied. Without accurate, context-rich data from the operating system, configuration files, or other trusted sources, scanners may generate massive, noisy reports that make prioritization difficult.

Scanners Only Know Known Vulnerabilities

Scanners rely heavily on pattern matching and databases of known vulnerabilities. This can lead to trouble, especially when it comes to zero-day vulnerabilities and outdated databases. Zero-day vulnerabilities aren’t known, and so they won’t get caught at all. When scanners are using older databases, they may generate false positives (flagging non-issues as risks) or false negatives (missing real threats), both of which can distort an organization’s view of its security posture.

Scanners Can’t Always Identify Devices

Because vulnerability scanners rely on network probes, service banners, and limited credentials to identify assets, they’re only seeing a narrow slice of an organization’s network. Firewalls, segmentation, or nonstandard items like IoT devices can evade detection. Even authenticated scans may miss assets if credentials are outdated or privileges restricted, and cloud-native or ephemeral resources can disappear before a scan ever reaches them. As a result, scanners frequently label devices incorrectly or fail to detect them altogether, creating blind spots.

Scanners Often Lack Context

Most scanners operate at the asset level, rather than the business or exploitability level.
They’re missing context about the asset; they can’t tell if it’s : 

• Externally accessible
• Behind a firewall
• Actively exploitable

Without context, every potential vulnerability is given equal weight, leading to poor prioritization of risks and alert fatigue for team members who are pinged every time any potential risk is flagged.

Scanners Might Be Focusing on Theoretical Risk

Because they assess vulnerabilities in isolation, scanners often focus on theoretical risk rather than real risk. For example, a scanner might flag every instance of a CVE based solely on version numbers, assuming the system is vulnerable even if the vulnerable component isn’t exposed, the function isn’t used, or compensating controls like firewalls or WAFs neutralize the threat. This approach pulls attention from exploitable issues that truly matter. 

What EDR Agents Get Wrong

While vulnerability scanners operate on theoretical vulnerabilities, EDR agents tell you what is happening on a device. Scanners and agents work hand in hand to detect risk. Because of the way they work, there is an assumption that agent-based insights are accurate. However, that’s a dangerous assumption. Like vulnerability scanners, agents can make mistakes. 

Agents Often Misclassify Threats

EDR agents are extremely good at some tasks. For example, they usually identify the operating system and basic system details of a device with high accuracy. When it comes to threats, behaviors, or alerts, however, they tend to make mistakes. That’s because the two tasks rely on fundamentally different types of data and inference: 

• OS detection is fact-based and structured: The agent queries known system APIs, registry keys, kernel signatures, or build metadata — all values that are stable and verifiable. 
• Risk classification requires context and pattern recognition: EDRs rely on behavioral heuristics, machine learning models, and threat intelligence correlations. Actions like “process spawning PowerShell” or “file modification in system32” might be malicious, but could also be part of normal admin behavior.

Agent-Based Tools Don’t Always Understand Data 

EDRs have deep visibility into a system, but that’s not the same as understanding. They may know what happened, but not why. Without contextual intelligence like network exposure, exploitability, and compensating controls, they can easily misclassify benign or irrelevant findings as critical.

Agents Can’t See Across an Entire Environment

EDR tools work within one system, so like scanners, they’re only seeing a piece of what’s happening.  Without organization-wide visibility, EDR agents may make false assumptions. For example, they may believe a vulnerability is exploitable when it’s not reachable externally. 

Agents May Create a False Sense of Security

The problem with relying too much on EDR agents is that they can cause a false sense of security. Teams may feel that if an agent says an organization is clean, they’re safe. However, without verifying those results, teams leave themselves vulnerable to blind spots, especially when agents aren’t installed, are misconfigured, or overlook nontraditional attack paths.

Too Much Trust in Disparate Tools

Organizations often place a lot of trust in patch management tools, assuming that if an agent reports all endpoints are up to date, the environment is secure. This confidence can be misleading, however. Some systems may never have been rebooted after patches were applied, or agents may have lost contact before completing the update process. As a result, vulnerabilities remain unpatched despite what the reporting indicates.

 An example of this is the 2017 WannaCry ransomware attack, which exploited the SMBv1 vulnerability on systems that had technically received patches but were never fully updated or rebooted. The incident highlights a crucial lesson: agent data shows reporting status, not actual state. Blind reliance on single-source patch data can give organizations a false sense of security, leaving them exposed to preventable attacks.

Unified Asset Management with Asimily

It’s dangerous to rely on tools that don’t have the complete picture of your organization’s risk. To minimize risk, your team needs to be able to see your entire attack surface, including everything from your networks to IoT devices.

Asimily unifies data from across your entire organization’s environment, building a single, trusted view of assets and vulnerabilities across IT, IoT, and OT devices. Asimily normalizes and correlates data to ensure that prioritization decisions reflect reality, accounting for device context and current security controls. 

To see how Asimily can help your organization gain the complete visibility necessary for effective vulnerability prioritization, contact us today for a demo.