Ctrlk
For the complete documentation index, see llms.txt. This page is also available as Markdown.

Search Engines

Cybersecurity-focused search engines are powerful tools that enable targeted research on specific assets, vulnerabilities, and threat intelligence. By searching for IP addresses, domains, CVEs, and other indicators, security professionals can perform tasks ranging from passive reconnaissance during penetration testing to comprehensive threat intelligence analysis.

Search.html - Your OSINT Starting Point

When beginning any OSINT investigation, consider starting with Search.html, a comprehensive HTML interface created by OSINT expert Michael Bazzell. This tool provides a unified interface for searching multiple sources simultaneously, allowing researchers to efficiently query various search engines and services with a single action. The tool's design facilitates manual analysis and enables investigators to follow multiple leads as they emerge during research.

Included Search Engines: Google, Google Date, Bing, Yahoo, Searx, Yandex, Baidu, DuckDuckGo, Startpage, Newsgroups, Blogs, FTP Servers, Indexes, Scholars, Patents, Qwant, News, Wayback Machine, and Ahmia. Some older Search.html entries, such as Exalead, are no longer active public web search engines.

The resources listed below provide a solid foundation for search engine-based investigation techniques, though they represent only a starting point for comprehensive OSINT research.

Local helper copy: Search.html

Google

Google remains one of the most comprehensive information gathering tools available to security professionals. While basic keyword searches are useful, leveraging advanced search operators (commonly known as "Google Dorks") can significantly refine results and uncover valuable intelligence about targets, including exposed files, vulnerable systems, and sensitive information inadvertently published online.

Google Search Utilities

  • Google Advanced Search - Google's native advanced search interface with multiple filtering options for refining search parameters

  • Keyword Tool - Analyzes keyword effectiveness and generates related search terms to improve search strategy

  • Google Alerts - Automated monitoring service that sends notifications when new content matching specified keywords appears in Google's index

  • ISearchFrom - Simulates Google searches from different geographic locations to analyze regional variations in search results

Google Dorking

pagodo - Passive Google Dork tool that automates Google Hacking Database scraping and searching against target domains

## Install
git clone https://github.com/opsdisk/pagodo.git
cd pagodo
virtualenv -p python3.7 .venv           # If using a virtual environment
source .venv/bin/activate               # If using a virtual environment
pip install -r requirements.txt

## Get Latest GHDB
python ghdb_scraper.py -s -j -i

## Run All Dorks
python pagodo.py -d example.com -g dorks/all_google_dorks.txt

Utility Search Engines

Archived Page Search Engines

Web pages frequently become unavailable due to deletion, site restructuring, or domain expiration. However, archived versions often persist in web archives and cached data repositories. These resources are essential when investigating historical website content or tracking changes over time.

  • Quick Cache and Archive Search - Unified interface for searching multiple web archives and cached versions of websites simultaneously

    • Wayback Machine - The most comprehensive web archive, containing billions of archived pages dating back to 1996. Essential for retrieving historical versions of web pages at specific dates and times

      • waybackurls - Command-line tool for retrieving all URLs archived for a domain from the Wayback Machine

    • archive.ph - On-demand archiving service that creates permanent snapshots of web pages

    • Common Crawl - Open repository of web crawl data collected since 2008, containing petabytes of archived web content

  • gau (Get All URLs) - Command-line tool that fetches all known URLs for a domain from AlienVault's Open Threat Exchange, Wayback Machine, and Common Crawl

Privacy-Focused Search Engines

  • Swisscows - Privacy-centric search engine that does not track, store, or profile users. All searches are encrypted and no data is logged

  • Gigablast - Open-source private search engine. Note: Service availability can be intermittent

  • Startpage - Privacy-focused search engine that delivers Google search results without tracking or storing user information. Based in the Netherlands with strong privacy protections

  • DuckDuckGo - Privacy-focused search engine that doesn't track users or personalize search results

Metasearch Engines

  • FaganFinder - Comprehensive metasearch tool providing unified access to numerous search engines, social media platforms, encyclopedias, libraries, news sources, government databases, and document repositories

  • All-io.net - Customizable metasearch engine aggregating results from major search engines. Allows users to create and configure personalized search engine interfaces

Miscellaneous Search Engines and Utilities

Additional specialized search engines and utilities for comprehensive OSINT research.

  • Search Engine Colossus - Comprehensive directory of search engines worldwide, organized by country and region

  • Million Short - Unique search engine that filters out the top 100, 1,000, 10,000, 100,000, or 1 million most popular websites, revealing lesser-known sources and obscure content

  • LeakIX - Specialized search engine indexing publicly exposed systems and data leaks, combined with an open reporting platform for security findings

International Search Engines

  • Russia - Yandex - Leading Russian search engine, essential for research targeting Russian-language content

  • China - Baidu - Dominant Chinese search engine, necessary for accessing content within China's internet ecosystem

  • Japan - Goo - Popular Japanese search engine and web portal

  • Korea - Daum - Major South Korean search engine and web portal

  • Iran - Parseek - Persian-language news aggregator and content portal (primarily aggregates news rather than providing web search)

  • Additional resources - OH SHINT! Search Engine List - Extensive compilation of international and specialized search engines

Specialized Content Search

  • Blog Search Engine - Dedicated blog search platform with options to search, submit, and subscribe to RSS feeds by topic

  • Boardreader - Multi-forum search engine indexing discussions across thousands of forums and message boards

  • Firebounty - Search engine specifically for bug bounty programs and vulnerability disclosure information

Deprecated and Inactive Tools

The following tools and services are no longer maintained or have been discontinued. They are listed here for historical reference and to prevent wasted time attempting to use them.

Google Dorking Tools

  • ASHOK - OSINT Swiss Army knife. Repository archived and no longer maintained

  • Googd0rker - Automated Google dork launcher for domain-specific OSINT. Repository last updated in 2018

Archive Services

  • WebCite - On-demand archive service formerly used for preserving cited web content in academic and scientific publications. No longer accepting new archiving requests as of 2020. Existing archives remain accessible

Search Engines

  • Exalead - Enterprise search platform acquired by Dassault Systèmes and transitioned to Netvibes. The public web search interface is no longer available

Training and Educational Resources

Hands-on training has been moved to the Training section.

Training and Resources
PreviousOSINTNextGoohak.sh Code

Last updated