On Mon, Oct 3, 2011 at 8:55 AM, Ed Summers <ehs_at_pobox.com> wrote: > Respecting robots.txt is key. It's normally easy to eyeball, but when > in doubt something like robotparser [1] can be helpful. A bit of > reflection on the Golden Rule probably is probably more important than > pondering the legality of what you are doing. s/probably is/is/ //EdReceived on Mon Oct 03 2011 - 09:05:59 EDT