python-protego: python-protego (robots.txt parser)
python-protego:
python-protego: Protego is a pure-Python robots.txt parser with support for modern
python-protego: conventions.
python-protego:
python-protego:
python-protego:
python-protego:
python-protego:
python-protego:
python-protego: