All Packages  Class Hierarchy  This Package  Previous  Next  Index

Class websphinx.RobotExclusion

java.lang.Object
   |
   +----websphinx.RobotExclusion

public class RobotExclusion
extends Object

Constructor Index

 o RobotExclusion(String)
Make a RobotExclusion object.

Method Index

 o clear()
Clear the cache of robots.txt entries.
 o disallowed(URL)
Check whether a URL is disallowed by robots.txt.
 o main(String[])

Constructors

 o RobotExclusion
 public RobotExclusion(String userAgent)
Make a RobotExclusion object.

Parameters:
userAgent - name of the robot using this object, as shown in the User-Agent header fields of its HTTP requests. Use null for anonymous robots.

Methods

 o disallowed
 public boolean disallowed(URL url)
Check whether a URL is disallowed by robots.txt.

Parameters:
url - URL to test
Returns:
true if url's Web site denies robot access to the url
 o clear
 public void clear()
Clear the cache of robots.txt entries.

 o main
 public static void main(String argv[]) throws Exception

All Packages  Class Hierarchy  This Package  Previous  Next  Index