![]() Server : LiteSpeed System : Linux premium84.web-hosting.com 4.18.0-553.44.1.lve.el8.x86_64 #1 SMP Thu Mar 13 14:29:12 UTC 2025 x86_64 User : claqxcrl ( 523) PHP Version : 8.1.32 Disable Function : NONE Directory : /opt/alt/python312/lib64/python3.12/urllib/__pycache__/ |
� ��h�$ � � � d Z ddlZddlZddlZddlZdgZ ej dd� Z G d� d� Z G d� d� Z G d � d � Zy)a% robotparser.py Copyright (C) 2000 Bastian Kleineidam You can choose between two licenses when using this package: 1) GNU GPLv2 2) PSF license for Python 2.2 The robots.txt Exclusion Protocol is implemented as specified in http://www.robotstxt.org/norobots-rfc.txt � N�RobotFileParser�RequestRatezrequests secondsc �Z � e Zd ZdZdd�Zd� Zd� Zd� Zd� Zd� Z d� Z d � Zd � Zd� Z d� Zd � Zy)r zs This class provides a set of methods to read, parse and answer questions about a single robots.txt file. c �z � g | _ g | _ d | _ d| _ d| _ | j |� d| _ y )NFr )�entries�sitemaps� default_entry�disallow_all� allow_all�set_url�last_checked��self�urls �9/opt/alt/python312/lib64/python3.12/urllib/robotparser.py�__init__zRobotFileParser.__init__ s; � ������ �!���!���������S����� c � � | j S )z�Returns the time the robots.txt file was last fetched. This is useful for long-running web spiders that need to check for new robots.txt files periodically. )r �r s r �mtimezRobotFileParser.mtime&