Unnamed repository; edit this file 'description' to name the repository.

Jeremy D Jeremy D Update the scrutinizer file 50226db @ 2022-08-10 18:57:27
.scrutinizer.yml 5.78 KB Update the scrutinizer file 2022-08-10 18:57:27
AddMoreSpiders.php 11.5 KB Initial commit 2022-03-06 14:24:07
DCO.txt 1.21 KB Add DCO and scrutinizer files 2022-04-04 18:57:57
LICENSE 1.48 KB Add a license 2022-03-06 14:26:11
package-info.xml 1.03 KB SMF Customize site doesn't detect versions without hints. 2022-03-06 14:38:01
readme.md 2.84 KB SMF Customize site doesn't detect versions without hints. 2022-03-06 14:38:01
readme.txt 3.11 KB SMF Customize site doesn't detect versions without hints. 2022-03-06 14:38:01


By Karl Benson | Link to Customization | Support Topic


Adds 88 more spiders/crawlers/bots to your Spiders section in SMF

SMF Support

Version Supported
2.1.x Yes
2.0.x Yes
1.x.x No


  • 83 Spiders belonging to search engines, validators, checkers, crawlers, bots, software, etc.
  • Including; Facebook, Ask, Baidu, GigaBot, Google-AdSense, Google-Adwords, Google-SA, Google-Image, Bing, InternetArchive, Alexa, Omgili, Speedy Spider, Yahoo, Yahoo JP, DeadLinksChecker, W3C Validator, W3C CSSValidator, W3C FeedValidator, W3C LinkChecker, W3C mobileOK, W3C P3PValidator, Bloglines, Feedburner, SnapBot, Picsearch, Websnapr, AllTheWeb, Altavista, Asterias, 192bot, AbachoBot, Abdcatos, Acoon, Accoona, BecomeBot, BlogRefsBot, Daumoa, DuckDuckBot, Exabot, Furl, FyperSpider, Geona, GirafaBot, GoSeeBot, Ichiro, LapozzBot, Looksmart, Lycos, Majestic12, MLBot, MSRBOT, MSR-ISRCCrawler, Naver, Naver, NoxTrumBot, OmniExplorer, OnetSzukaj, ScrubTheWeb, SearchSight, Seeqpod, Shablast, SitiDiBot, Slider, Sogou, Sosospider, StackRambler, SurveyBot, Touche, Walhello, WebAlta, Wisponbot, YacyBot, YodaoBot, Charlotte, DiscoBot, EnaBot, Gaisbot, Kalooga, ScoutJet, TinEye, Twiceler, GSiteCrawler, HTTrack, Wget (Remember SMF detects most Google/Yahoo/MSN bots by default)

Spider List

It appears that most sites offering spider/bot lists have tonnes of inactive ones. I'm putting together my own lists by detecting them in the wild on my own sites. Plus any that get reported to me (after I've checked them out). So if there are any ACTIVE ones I'm missing? Please let me know in the support topic.


Any previous versions of the mod does NOT need to be uninstalled. This mod adds a row for each spider in the database only.

  • No theme edits required.
  • No language strings to translate or editing.
  • It adds database rows only.
  • It will ignore adding ones which already exist.

Install and your done. Note: Click uninstall to remove the mod from your mod list. But it won't remove the spiders from your database. You'll need to remove each one from your SMF Search Engines/Spider section.

Manual Installation

For manual installation, just upload AddMoreSpiders.php to your SMF directory and run it in your browser (then delete the file).

Useful Links


Please use the modification thread for support with this modification. (Please don't ask me to do the edits for you)