Checkout
Alex
committed
on 22 Mar 07
spider support changed back to robots_list.txt file support