nyawc.http.Handler.
Handler
(options, queue_item)[source]¶Bases: object
The Handler class executes HTTP requests.
_Handler__content_type_matches
(content_type, available_content_types)[source]¶Check if the given content type matches one of the available content types.
Parameters: |
|
---|---|
Returns: | True if a match was found, False otherwise. |
Return type: | bool |
_Handler__get_all_scrapers
()[source]¶Find all available scraper references.
Returns: | The scraper references. |
---|---|
Return type: | list(obj) |
_Handler__get_all_scrapers_modules
()[source]¶Find all available scraper modules.
Returns: | The scraper modules. |
---|---|
Return type: | list(obj) |
_Handler__make_request
(url, method, data, auth, cookies, headers, proxies, timeout, verify)[source]¶Execute a request with the given data.
Parameters: |
|
---|---|
Returns: | The response object. |
Return type: | obj |
__init__
(options, queue_item)[source]¶Construct the HTTP handler.
Parameters: |
|
---|
get_new_requests
()[source]¶Retrieve all the new request that were found in this request.
Returns: | A list of request objects. |
---|---|
Return type: | list(nyawc.http.Request ) |
nyawc.http.Request.
Request
(url, method='get', data=None, auth=None, cookies=None, headers=None, proxies=None, timeout=30, verify=True)[source]¶Bases: object
The Request class contains details that were used to request the specified URL.
The (requests module) cookie jar to use for the request.
Type: | obj |
---|
verify
[source]¶True or False based on if certificates should be checked or else a path to a trusted bundle.
Type: | mixed |
---|
METHOD_DELETE
= 'delete'[source]METHOD_GET
= 'get'[source]METHOD_HEAD
= 'head'[source]METHOD_OPTIONS
= 'options'[source]METHOD_POST
= 'post'[source]METHOD_PUT
= 'put'[source]__init__
(url, method='get', data=None, auth=None, cookies=None, headers=None, proxies=None, timeout=30, verify=True)[source]¶Constructs a Request instance.
Parameters: |
|
---|
nyawc.http.Response.
Response
(url)[source]¶Bases: object
Response placeholder class for before request is finished.
Note
This class will be replaced with the response class of Python’s requests module when the request is finished. For more information check http://docs.python-requests.org/en/master/api/#requests.Response.