> To help cut down on some of the webcrawler traffic, create a robots.txt file
> and put it in your webroot directory.  If you're using webv4 that would be
> /sbbs/webv4/root.  In the robots.txt file, all you need really is three
> lines:

> User-agent: *
> User-agent: AdsBot-Google
> Disallow: /

> It won't eliminate all of the webcrawler traffic as there are some that
> ignore it, but it will stop many from indexing everything on your site.

> Chris (CJ)
> SysOp - CJ's Place BBS

> ... Gone crazy, be back later, please leave message.
> --- MultiMail/Linux v0.52
>  ¨ Synchronet ¨ CJ's Place, Orange City, FL - cjsplace.thruhere.net


Thank you. I will do that.

---
 þ Synchronet þ Ricks BBS - RICKSBBS.SYNCHRO.NET