3
�Pf�A � @ s d dl mZ d dlZd dlZd dlZddlmZ ddlmZm Z ddlm
Z
ddl m Z m
Z
mZ ddlmZ dd lmZ dd
lmZ dd lmZ d d
dgZeje�Zd4Zd5Z ej d-e �Z d.d/� Z ej e e �ej e e �d0�Z!ee d0�Z"G d1d � d e�Z#G d2d
� d
e#�Z$d3d� Z%dS )6� )�absolute_importN� )�RecentlyUsedContainer)�HTTPConnectionPool�HTTPSConnectionPool)�port_by_scheme)�LocationValueError�
MaxRetryError�ProxySchemeUnknown)�urljoin)�RequestMethods)� parse_url)�Retry� PoolManager� ProxyManager�proxy_from_url�key_file� cert_file� cert_reqs�ca_certs� ssl_version� ca_cert_dir� ssl_context�
key_scheme�key_host�key_port� key_timeout� key_retries�
key_strict� key_block�key_source_address� key_key_file�
key_cert_file�
key_cert_reqs� key_ca_certs�key_ssl_version�key_ca_cert_dir�key_ssl_context� key_maxsize� key_headers�
key__proxy�key__proxy_headers�key_socket_options�key__socks_options�key_assert_hostname�key_assert_fingerprint�PoolKeyc C s� |j � }|d j� |d< |d j� |d< x4d D ],}||kr.|| dk r.t|| j� �||< q.W |jd�}|dk r|t|�|d< x&t|j� �D ]}|j|�|d| < q�W x | j D ]}||kr�d||< q�W | f |�S )
a�
Create a pool key out of a request context dictionary.
According to RFC 3986, both the scheme and host are case-insensitive.
Therefore, this function normalizes both before constructing the pool
key for an HTTPS request. If you wish to change this behaviour, provide
alternate callables to ``key_fn_by_scheme``.
:param key_class:
The class to use when constructing the key. This should be a namedtuple
with the ``scheme`` and ``host`` keys at a minimum.
:type key_class: namedtuple
:param request_context:
A dictionary-like object that contain the context for a request.
:type request_context: dict
:return: A namedtuple that can be used as a connection pool key.
:rtype: PoolKey
�scheme�host�headers�_proxy_headers�_socks_optionsNZsocket_optionsZkey_)r3 r4 r5 )
�copy�lower� frozenset�items�get�tuple�list�keys�pop�_fields)Z key_class�request_context�context�keyZ socket_optsZfield� rC �!/usr/lib/python3.6/poolmanager.py�_default_key_normalizer9 s
rE )�http�httpsc @ sx e Zd ZdZdZd dd�Zdd� Zdd � Zd d
d �Zd d
� Z d dd�Z
dd� Z d dd�Z d dd�Z
dd� Zd!dd�ZdS )"r a$
Allows for arbitrary requests while transparently keeping track of
necessary connection pools for you.
:param num_pools:
Number of connection pools to cache before discarding the least
recently used pool.
:param headers:
Headers to include with all requests, unless other headers are given
explicitly.
:param \**connection_pool_kw:
Additional parameters are used to create fresh
:class:`urllib3.connectionpool.ConnectionPool` instances.
Example::
>>> manager = PoolManager(num_pools=2)
>>> r = manager.request('GET', 'http://google.com/')
>>> r = manager.request('GET', 'http://google.com/mail')
>>> r = manager.request('GET', 'http://yahoo.com/')
>>> len(manager.pools)
2
N�
c K s8 t j| |� || _t|dd� d�| _t| _tj� | _d S )Nc S s | j � S )N)�close)�prC rC rD �� s z&PoolManager.__init__..)Z dispose_func)r �__init__�connection_pool_kwr �pools�pool_classes_by_scheme�key_fn_by_schemer6 )�self� num_poolsr3 rM rC rC rD rL � s zPoolManager.__init__c C s | S )NrC )rQ rC rC rD � __enter__� s zPoolManager.__enter__c C s | j � dS )NF)�clear)rQ �exc_typeZexc_valZexc_tbrC rC rD �__exit__� s zPoolManager.__exit__c C sf | j | }|dkr | jj� }xdD ]}|j|d� q"W |dkrXxtD ]}|j|d� qDW |||f|�S )a�
Create a new :class:`ConnectionPool` based on host, port, scheme, and
any additional pool keyword arguments.
If ``request_context`` is provided, it is provided as keyword arguments
to the pool class used. This method is used to actually create the
connection pools handed out by :meth:`connection_from_url` and
companion methods. It is intended to be overridden for customization.
Nr1 r2 �portrF )r1 r2 rW )rO rM r6 r> � SSL_KEYWORDS)rQ r1 r2 rW r@ Zpool_clsrB �kwrC rC rD � _new_pool� s
zPoolManager._new_poolc C s | j j� dS )z�
Empty our store of pools and direct them all to close.
This will not affect in-flight connections, but they will not be
re-used after completion.
N)rN rT )rQ rC rC rD rT � s zPoolManager.clearrF c C sT |s t d��| j|�}|p d|d< |s:tj|d j� d�}||d< ||d< | j|�S )a�
Get a :class:`ConnectionPool` based on the host, port, and scheme.
If ``port`` isn't given, it will be derived from the ``scheme`` using
``urllib3.connectionpool.port_by_scheme``. If ``pool_kwargs`` is
provided, it is merged with the instance's ``connection_pool_kw``
variable and used to create the new connection pool, if one is
needed.
zNo host specified.rF r1 �P rW r2 )r �_merge_pool_kwargsr r: r7 �connection_from_context)rQ r2 rW r1 � pool_kwargsr@ rC rC rD �connection_from_host� s
z PoolManager.connection_from_hostc C s, |d j � }| j| }||�}| j||d�S )z�
Get a :class:`ConnectionPool` based on the request context.
``request_context`` must at least contain the ``scheme`` key and its
value must be a key in ``key_fn_by_scheme`` instance variable.
r1 )r@ )r7 rP �connection_from_pool_key)rQ r@ r1 Zpool_key_constructor�pool_keyrC rC rD r] � s
z#PoolManager.connection_from_contextc
C s` | j j�N | j j|�}|r |S |d }|d }|d }| j||||d�}|| j |< W dQ R X |S )z�
Get a :class:`ConnectionPool` based on the provided pool key.
``pool_key`` should be a namedtuple that only contains immutable
objects. At a minimum it must have the ``scheme``, ``host``, and
``port`` fields.
r1 r2 rW )r@ N)rN �lockr: rZ )rQ ra r@ Zpoolr1 r2 rW rC rC rD r` � s
z$PoolManager.connection_from_pool_keyc C s t |�}| j|j|j|j|d�S )a�
Similar to :func:`urllib3.connectionpool.connection_from_url`.
If ``pool_kwargs`` is not provided and a new pool needs to be
constructed, ``self.connection_pool_kw`` is used to initialize
the :class:`urllib3.connectionpool.ConnectionPool`. If ``pool_kwargs``
is provided, it is used instead. Note that if a new pool does not
need to be created for the request, the provided ``pool_kwargs`` are
not used.
)rW r1 r^ )r
r_ r2 rW r1 )rQ �urlr^ �urC rC rD �connection_from_url
s z PoolManager.connection_from_urlc C sZ | j j� }|rVxF|j� D ]:\}}|dkrJy
||= W qR tk
rF Y qRX q|||< qW |S )a
Merge a dictionary of override values for self.connection_pool_kw.
This does not modify self.connection_pool_kw and returns a new dict.
Any keys in the override dictionary with a value of ``None`` are
removed from the merged dictionary.
N)rM r6 r9 �KeyError)rQ �overrideZbase_pool_kwargsrB �valuerC rC rD r\ s
z PoolManager._merge_pool_kwargsTc K sd t |�}| j|j|j|jd�}d|d< d|d< d|krD| jj� |d< | jdk rj|jdkrj|j||f|�}n|j||j f|�}|o�|j
� }|s�|S t ||�}|j dkr�d }|j
d
�} t| t�s�tj| |d �} | jo�|j|� � r�x | jD ]}
|d j|
d� q�W y| j||||d �} W n tk
�r4 | j�r0� |S X | |d
< ||d< tjd
||� | j||f|�S )a]
Same as :meth:`urllib3.connectionpool.HTTPConnectionPool.urlopen`
with custom cross-host redirect logic and only sends the request-uri
portion of the ``url``.
The given ``url`` parameter must be absolute, such that an appropriate
:class:`urllib3.connectionpool.ConnectionPool` can be chosen for it.
)rW r1 FZassert_same_host�redirectr3 NrF i/ ZGET�retries)ri )�responseZ_poolzRedirecting %s -> %s)r
r_ r2 rW r1 r3 r6 �proxy�urlopenZ request_uriZget_redirect_locationr Zstatusr: �
isinstancer Zfrom_intZremove_headers_on_redirectZ is_same_hostr> Z incrementr Zraise_on_redirect�log�info) rQ �methodrc ri rY rd Zconnrk Zredirect_locationrj �headerrC rC rD rm - s@
zPoolManager.urlopen)rH N)N)NrF N)N)N)T)�__name__�
__module__� __qualname__�__doc__rl rL rS rV rZ rT r_ r] r` re r\ rm rC rC rC rD r y s
c sH e Zd ZdZd� fdd� Zd� fdd� Zdd d
�Zd� fd d
� Z� ZS )r aw
Behaves just like :class:`PoolManager`, but sends all requests through
the defined proxy, using the CONNECT method for HTTPS URLs.
:param proxy_url:
The URL of the proxy to be used.
:param proxy_headers:
A dictionary contaning headers that will be sent to the proxy. In case
of HTTP they are being sent with each request, while in the
HTTPS/CONNECT case they are sent only once. Could be used for proxy
authentication.
Example:
>>> proxy = urllib3.ProxyManager('http://localhost:3128/')
>>> r1 = proxy.request('GET', 'http://google.com/')
>>> r2 = proxy.request('GET', 'http://httpbin.org/')
>>> len(proxy.pools)
1
>>> r3 = proxy.request('GET', 'https://httpbin.org/')
>>> r4 = proxy.request('GET', 'https://twitter.com/')
>>> len(proxy.pools)
3
rH Nc s� t |t�r d|j|j|jf }t|�}|jsFtj|jd�}|j|d�}|jdkrZt |j��|| _
|pfi | _ | j
|d< | j |d< t t
| �j||f|� d S ) Nz
%s://%s:%ir[ )rW rF rG �_proxyr4 )rF rG )rn r r1 r2 rW r
r r: �_replacer
rl �
proxy_headers�superr rL )rQ � proxy_urlrR r3 ry rM rl rW )� __class__rC rD rL � s
zProxyManager.__init__rF c sD |dkr t t| �j||||d�S t t| �j| jj| jj| jj|d�S )NrG )r^ )rz r r_ rl r2 rW r1 )rQ r2 rW r1 r^ )r| rC rD r_ � s
z!ProxyManager.connection_from_hostc C s0 ddi}t |�j}|r ||d< |r,|j|� |S )z�
Sets headers needed by proxies: specifically, the Accept and Host
headers. Only sets headers not provided by the user.
ZAcceptz*/*ZHost)r
�netloc�update)rQ rc r3 Zheaders_r} rC rC rD �_set_proxy_headers� s
z ProxyManager._set_proxy_headersTc sN t |�}|jdkr0|jd| j�}| j||�|d< tt| �j||fd|i|��S )z@Same as HTTP(S)ConnectionPool.urlopen, ``url`` must be absolute.rF r3 ri )r
r1 r: r3 r rz r rm )rQ rq rc ri rY rd r3 )r| rC rD rm � s
zProxyManager.urlopen)rH NN)NrF N)N)T) rs rt ru rv rL r_ r rm �
__classcell__rC rC )r| rD r i s
c K s t f d| i|��S )Nr{ )r )rc rY rC rC rD r � s )r r r r r r r )r r r r r r r r r! r" r# r$ r% r& r' r( r) r* r+ r, r- r. r/ )&Z
__future__r � collections� functoolsZlogging� _collectionsr Zconnectionpoolr r r �
exceptionsr r r
Z packages.six.moves.urllib.parser Zrequestr Zutil.urlr
Z
util.retryr �__all__Z getLoggerrs ro rX Z _key_fields�
namedtupler0 rE �partialrP rO r r r rC rC rC rD � s`
6
qW