EqualifyEverything / crawler
🐳 Stuff
Currently, the crawler will look for a database for the source of domains, so this will likely break.
To Do:
- Add docker env vars
- Sitemap Crawling
- Database Connection
- Improve Logging & Error Handling
- Documentation
Bentley Hensel
committed on Mar 30, 2023, 07:36 PM
Showing 2 changed files
+60 additions
-0 deletions