Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
Merge pull request #146 from perusiongreg/master
Update robots.cfg
  • Loading branch information
jonjensen committed Nov 30, 2023
2 parents f456cc3 + 1ea7b49 commit 7ad5760
Showing 1 changed file with 5 additions and 2 deletions.
7 changes: 5 additions & 2 deletions dist/robots.cfg
Expand Up @@ -3,6 +3,7 @@ RobotUA <<EOR
AdsBot-Google,
agent,
AltaVista,
Amazonbot,
Apache (internal dummy connection),
appie,
AppleSyndication,
Expand All @@ -20,6 +21,7 @@ RobotUA <<EOR
Awasu,
BackRub,
bender,
BingPreview,
bingbot,
Bookdog,
BookmarkSync,
Expand Down Expand Up @@ -89,13 +91,15 @@ RobotUA <<EOR
MFC_Tear_Sample,
Microsoft Data Access,
Microsoft Office,
Microsoft Outlook,
Microsoft URL Control,
Microsoft-WebDAV,
MimeLive,
mirago,
Miva,
moget,
MSFrontPage,
MSOffice,
Nazilla,
NetMechanic,
NetScoop,
Expand All @@ -108,6 +112,7 @@ RobotUA <<EOR
panscient.com,
ParaSite,
pavuk,
Pinterestbot,
POE-Component,
Pokey,
Pompos,
Expand Down Expand Up @@ -171,8 +176,6 @@ NotRobotUA <<EOR
AskTB,
Seekmo,
SearchToolbar,
MSIE,
Gecko,
EOR

RobotIP <<EOR
Expand Down

0 comments on commit 7ad5760

Please sign in to comment.