urllib.parse — Parse URLs into components
urllib.parse
urllib.robotparser — Parser for robots.txt
urllib.robotparser