Internet-Scale Attack Surface Discovery
Cyber Crawler autonomously scans the entire public internet to discover exposed assets, map vulnerabilities, and detect risks — before attackers do. Passive, responsible, and fully RFC-compliant.
How Cyber Crawler Works
AI-assisted discovery across the entire public internet
Discover
DNS enumeration, subdomain discovery, and TLS fingerprinting across the internet.
Classify
AI-driven server fingerprinting, technology detection, and cloud exposure mapping.
Detect
Passive vulnerability mapping via version fingerprints, metadata, and security headers.
Alert
Real-time visibility with actionable findings for shadow IT and unknown exposures.
What We Detect
Non-intrusive vulnerability & misconfiguration detection
Identify deprecated protocols, insecure cipher suites, and certificate issues across your entire internet-facing footprint.
Data Collection
Only publicly visible information — never behind authentication
We Collect (Public Only)
- Domain names and subdomains
- DNS records and IP address mappings
- Public IPs and exposed ports
- Service banners and protocol metadata
- TLS certificate info and cipher details
- HTTP headers and tech fingerprints
- Public cloud storage visibility
We Do NOT Collect
- User data or private content
- Credentials or internal system data
- Information behind authentication
- Data behind access controls
Cyber Crawler never attempts to bypass security controls, authenticate, or exploit vulnerabilities.
Responsible Crawling
Passive, legal, safe — like a search engine, not an attacker
Opt-Out Instructions
Two mechanisms to exclude your assets from scanning
Email Opt-Out
Send a request to:
crawler-optout@cyberarmy.techInclude:
- Organization name
- Contact name and role
- Domains, subdomains, IPs, or CIDR ranges
- Proof of ownership (may be required)
robots.txt Exclusion
Add to your robots.txt:
User-agent: CyberCrawler Disallow: /
Note: robots.txt applies only to web crawling, not basic IP visibility or port discovery.
Important Notes
- Opt-out removes assets from future scans
- You may stop receiving exposure notifications
- Data already collected from public sources cannot be erased retroactively
- Minimal audit/compliance records may be retained
Questions About Cyber Crawler?
Learn how Cyber Crawler helps organizations discover and address their public attack surface exposure.