#Skip to menu


robots is a library written in golang for parsing the robots.txt file format.


First install the library with

go get xojoc.pw/crawl/robots

…then run…

go test

if something fails open an issue.

See godoc for the complete documentation.


robots was written by Alexandru Cojocaru (https://xojoc.pw).

Get the code

See all the discussions around the web about this page.