Skip to main content

Replication and Web Crawler Policy

Note: This is a boilerplate document. It must be reviewed by legal counsel.

We value an open ecosystem, but excessive unfettered scraping can destabilize the SigmaShake Hub for all developers.

  1. Rate Limits: Web crawlers must respect our standard Hub rate limits to prevent denial-of-service conditions.
  2. robots.txt: Crawlers must respect rules specified in standard robots.txt implementations.
  3. Mirrors: If you intend to operate a full mirror of the SigmaShake Hub rulesets, you must contact us for an approved, federated data flow process.