�
��abc @` s� d d l m Z m Z m Z d d l m Z d d l m Z d d l m
Z
d d l m Z d d l m Z m
Z
d d l m Z m Z m Z d d l m Z m Z d d
l m Z d d l m Z d d l m Z e e � Z d
e f d � � YZ d S( i ( t absolute_importt divisiont unicode_literals( t unichr( t dequei ( t spaceCharacters( t entities( t asciiLetterst asciiUpper2Lower( t digitst hexDigitst EOF( t
tokenTypest
tagTokenTypes( t replacementCharacters( t HTMLInputStream( t Triet
HTMLTokenizerc B` s� e Z d Z dJ d � Z d � Z d � Z dJ e d � Z d � Z d � Z
d � Z d � Z d � Z
d
� Z d � Z d � Z d
� Z d � Z d � Z d � Z d � Z d � Z d � Z d � Z d � Z d � Z d � Z d � Z d � Z d � Z d � Z d � Z d � Z! d � Z" d � Z# d � Z$ d! � Z% d" � Z& d# � Z' d$ � Z( d% � Z) d& � Z* d' � Z+ d( � Z, d) � Z- d* � Z. d+ � Z/ d, � Z0 d- � Z1 d. � Z2 d/ � Z3 d0 � Z4 d1 � Z5 d2 � Z6 d3 � Z7 d4 � Z8 d5 � Z9 d6 � Z: d7 � Z; d8 � Z< d9 � Z= d: � Z> d; � Z? d< � Z@ d= � ZA d> � ZB d? � ZC d@ � ZD dA � ZE dB � ZF dC � ZG dD � ZH dE � ZI dF � ZJ dG � ZK dH � ZL dI � ZM RS(K u This class takes care of tokenizing HTML.
* self.currentToken
Holds the token that is currently being processed.
* self.state
Holds a reference to the method to be invoked... XXX
* self.stream
Points to HTMLInputStream object.
c K` sb t | | � | _ | | _ t | _ g | _ | j | _ t | _ d | _
t t | � j
� d S( N( R t streamt parsert Falset
escapeFlagt
lastFourCharst dataStatet statet escapet Nonet currentTokent superR t __init__( t selfR R t kwargs( ( sC /usr/lib/python2.7/site-packages/pip/_vendor/html5lib/_tokenizer.pyR " s c c` s} t g � | _ xg | j � rx x6 | j j rV i t d d 6| j j j d � d 6Vq! Wx | j rt | j j � VqZ Wq Wd S( u� This is where the magic happens.
We do our usually processing through the states and when we have a token
to return we yield the token which pauses processing until the next token
is requested.
u
ParseErroru typei u dataN( R t
tokenQueueR R t errorsR t popt popleft( R ( ( sC /usr/lib/python2.7/site-packages/pip/_vendor/html5lib/_tokenizer.pyt __iter__1 s * c % C` s� t } d } | r! t } d } n g } | j j � } x8 | | k rp | t k rp | j | � | j j � } q9 Wt d j | � | � } | t k r� t | } | j j i t
d d 6d d 6i | d 6d 6� n�d
| k o� d k n s� | d k r3d
} | j j i t
d d 6d d 6i | d 6d 6� nrd | k oJd k n s d | k ofd k n s d | k o�d k n s d | k o�d k n s | t d d d d d d d d d d d d! d" d# d$ d% d& d' d( d) d* d+ d, d- d. d/ d0 d1 d2 d3 d4 d5 d6 d7 d g# � k rQ| j j i t
d d 6d d 6i | d 6d 6� n y t | � } WnA t
k
r�| d8 } t d
| d ?B� t d9 | d: @B� } n X| d; k r�| j j i t
d d 6d< d 6� | j j | � n | S(= u� This function returns either U+FFFD or the character based on the
decimal or hexadecimal representation. It also discards ";" if present.
If not present self.tokenQueue.append({"type": tokenTypes["ParseError"]}) is invoked.
i
i u u
ParseErroru typeu$ illegal-codepoint-for-numeric-entityu datau charAsIntu datavarsi � i�� i�� u �i i i i i i� i�� i�� i i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i�� i��
i��
i�� i�� i�� i�� i��
i��
i�� i�� i�� i�� i�� i i � i� u ;u numeric-entity-without-semicolon( R R
R t charR t appendt intt joinR R R t frozensett chrt
ValueErrort unget( R t isHext allowedt radixt charStackt ct charAsIntR% t v( ( sC /usr/lib/python2.7/site-packages/pip/_vendor/html5lib/_tokenizer.pyt consumeNumberEntityA s`
* c C` s� d } | j j � g } | d t k s] | d t d d f k s] | d k rt | | d k rt | j j | d � n�| d d k rpt } | j | j j � � | d d k r� t } | j | j j � � n | r� | d t k s� | r"| d t
k r"| j j | d � | j | � } q7| j j i t
d d 6d
d 6� | j j | j � � d d j | � } n�xF | d t k r�t j d j | � � s�Pn | j | j j � � qsWy, t j d j | d � � } t | � } Wn t k
r�d } n X| d k r�| d d
k r@| j j i t
d d 6d d 6� n | d d
k r�| r�| | t k s�| | t
k s�| | d k r�| j j | j � � d d j | � } q7t | } | j j | j � � | d j | | � 7} nK | j j i t
d d 6d d 6� | j j | j � � d d j | � } | r[| j d d d c | 7u
ParseErroru' expected-tag-name-but-got-right-bracketu
Charactersu <>u ?u' expected-tag-name-but-got-question-marku expected-tag-nameu <( R R% t markupDeclarationOpenStateR t closeTagOpenStateR R R R t tagNameStateR R& R R, t bogusCommentStateR5 ( R RJ ( ( sC /usr/lib/python2.7/site-packages/pip/_vendor/html5lib/_tokenizer.pyRH i s6
" " c C` s? | j j � } | t k rS i t d d 6| d 6g d 6t d 6| _ | j | _ n� | d k r� | j j i t d d 6d d 6� | j
| _ n� | t k r� | j j i t d d 6d d 6� | j j i t d
d 6d d 6� | j
| _ nL | j j i t d d 6d d 6i | d 6d
6� | j j | � | j
| _ t S( Nu EndTagu typeu nameu datau selfClosingu >u
ParseErroru* expected-closing-tag-but-got-right-bracketu expected-closing-tag-but-got-eofu
Charactersu u
ParseErroru typeu eof-in-tag-nameu datau /u u invalid-codepointu nameu �( R R% R t beforeAttributeNameStateR RF R R R& R R t selfClosingStartTagStateR R5 ( R RJ ( ( sC /usr/lib/python2.7/site-packages/pip/_vendor/html5lib/_tokenizer.pyRV � s"
c C` su | j j � } | d k r3 d | _ | j | _ n> | j j i t d d 6d d 6� | j j | � | j | _ t
S( Nu /u u
Charactersu typeu | j j i t d d 6d d 6� | j j | � | j
| _ t S( Nu
Charactersu typeu u
Charactersu | j j i t d d 6d d 6� | j j | � | j | _ t
S( Nu /u u
Charactersu typeu | j j i t d d 6d d 6� | j j | � | j
| _ t S( Nu
Charactersu typeu u
Charactersu | j j i t d d 6d d 6� | j j | � | j
| _ t S( Nu /u u !u
Charactersu typeu | j j i t d d 6d d 6� | j j | � | j
| _ t S( Nu
Charactersu typeu u
Charactersu u u
ParseErroru invalid-codepointu �( R R% R R& R Rg R RR Rh R R R5 ( R RJ ( ( sC /usr/lib/python2.7/site-packages/pip/_vendor/html5lib/_tokenizer.pyRe � s& % " " c C` s� | j j � } | d k r3 d | _ | j | _ n� | t k r} | j j i t d d 6d | d 6� | | _ | j | _ n> | j j i t d d 6d d 6� | j j
| � | j | _ t S( Nu /u u
Charactersu typeu | j j i t d d 6d d 6� | j j | � | j
| _ t S( Nu
Charactersu typeu u
Charactersu u
Charactersu typeu datau script( u /u >( R R% R R) R R& R RZ R] t scriptDataDoubleEscapedStateR Rh R R, R5 ( R RJ ( ( sC /usr/lib/python2.7/site-packages/pip/_vendor/html5lib/_tokenizer.pyRj � s " " c C` s? | j j � } | d k rL | j j i t d d 6d d 6� | j | _ n� | d k r� | j j i t d d 6d d 6� | j | _ n� | d k r� | j j i t d d 6d d 6� | j j i t d d 6d d 6� n_ | t k r| j j i t d d 6d
d 6� | j | _ n" | j j i t d d 6| d 6� t
S( Nu -u
Charactersu typeu datau u u
ParseErroru invalid-codepointu �u eof-in-script-in-script( R R% R R& R Rn R RR Rl R R R5 ( R RJ ( ( sC /usr/lib/python2.7/site-packages/pip/_vendor/html5lib/_tokenizer.pyRo s, % " " " c C` su | j j � } | d k rU | j j i t d d 6d d 6� d | _ | j | _ n | j j | � | j | _ t
S( Nu /u
Charactersu typeu datau ( R R% R R& R RZ t scriptDataDoubleEscapeEndStateR R, Rl R5 ( R RJ ( ( sC /usr/lib/python2.7/site-packages/pip/_vendor/html5lib/_tokenizer.pyRn 0 s " c C` s� | j j � } | t t d � Bk rz | j j i t d d 6| d 6� | j j � d k rk | j | _
q� | j | _
n\ | t k r� | j j i t d d 6| d 6� | j | 7_ n | j j
| � | j | _
t S( Nu /u >u
Charactersu typeu datau script( u /u >( R R% R R) R R& R RZ R] Rh R Rl R R, R5 ( R RJ ( ( sC /usr/lib/python2.7/site-packages/pip/_vendor/html5lib/_tokenizer.pyRp ; s " " c C` s� | j j � } | t k r1 | j j t t � nz| t k rf | j d j | d g � | j | _ nE| d k r | j
� n,| d k r� | j | _ n| d k r� | j j i t
d d
6d d 6� | j d j | d g � | j | _ n� | d k rH| j j i t
d d
6d
d 6� | j d j d d g � | j | _ nc | t k r�| j j i t
d d
6d d 6� | j | _ n& | j d j | d g � | j | _ t S( Nu datau u >u /u 'u "u =u u /u u
ParseErroru typeu invalid-codepointu �u 'u "u u datau u /u u
ParseErroru typeu invalid-codepointu �u 'u "u u
ParseErroru typeu. expected-attribute-value-but-got-right-bracketu datau u invalid-codepointi����i u �u =u
c C` s� | j j � } | d k r* | j | _ n� | d k rF | j d � n� | d k r� | j j i t d d 6d d 6� | j d d d c d
7u "u 'u =u u "u 'u =u | j j i t
d d 6d d 6� | j j | � | j | _ t
S( Nu >u /u
ParseErroru typeu$ unexpected-EOF-after-attribute-valueu datau* unexpected-character-after-attribute-value( R R% R RX R RF RY R R R& R R, R R5 ( R RJ ( ( sC /usr/lib/python2.7/site-packages/pip/_vendor/html5lib/_tokenizer.pyR{ s"
c C` s� | j j � } | d k r5 t | j d <| j � n� | t k r� | j j i t d d 6d d 6� | j j | � | j
| _ n> | j j i t d d 6d d 6� | j j | � | j | _ t S( Nu >u selfClosingu
ParseErroru typeu# unexpected-EOF-after-solidus-in-tagu datau) unexpected-character-after-solidus-in-tag(
R R% R5 R RF R R R& R R, R R RX ( R RJ ( ( sC /usr/lib/python2.7/site-packages/pip/_vendor/html5lib/_tokenizer.pyRY 4 s
c C` sc | j j d � } | j d d � } | j j i t d d 6| d 6� | j j � | j | _ t S( Nu >u u �u Commentu typeu data(
R RI t replaceR R& R R% R R R5 ( R RJ ( ( sC /usr/lib/python2.7/site-packages/pip/_vendor/html5lib/_tokenizer.pyRW F s
c C` sB | j j � g } | d d k rv | j | j j � � | d d k r�i t d d 6d d 6| _ | j | _ t Snw| d d k r(t } xP d d d! d" d# d$ f D]6 } | j | j j � � | d | k r� t } Pq� q� W| r�i t d d 6d d 6d d 6d d 6t d 6| _ | j
| _ t Sn� | d d k r�| j d k r�| j j j
r�| j j j
d j | j j j k r�t } xP d d d d d d g D]6 } | j | j j � � | d | k r�t } Pq�q�W| r�| j | _ t Sn | j j i t d d 6d d 6� x | r1| j j | j � � qW| j | _ t S(% Ni����u -u Commentu typeu u datau du Du ou Ou cu Cu tu Tu yu Yu pu Pu eu Eu Doctypeu nameu publicIdu systemIdu correctu [u Au
ParseErroru expected-dashes-or-doctype( u du D( u ou O( u cu C( u tu T( u yu Y( u pu P( u eu E( R R% R& R R t commentStartStateR R5 R R t doctypeStateR t treet openElementst namespacet defaultNamespacet cdataSectionStateR R, R" RW ( R R0 t matchedt expected( ( sC /usr/lib/python2.7/site-packages/pip/_vendor/html5lib/_tokenizer.pyRT U sR
% c C` s1 | j j � } | d k r* | j | _ n| d k rn | j j i t d d 6d d 6� | j d c d 7u incorrect-commentu eof-in-comment( R R% t commentStartDashStateR R R& R R R R t commentStateR5 ( R RJ ( ( sC /usr/lib/python2.7/site-packages/pip/_vendor/html5lib/_tokenizer.pyR} � s( c C` s5 | j j � } | d k r* | j | _ n| d k rn | j j i t d d 6d d 6� | j d c d 7u incorrect-commentu eof-in-comment( R R% t commentEndStateR R R& R R R R R� R5 ( R RJ ( ( sC /usr/lib/python2.7/site-packages/pip/_vendor/html5lib/_tokenizer.pyR� � s( c C` s� | j j � } | d k r* | j | _ n� | d k rn | j j i t d d 6d d 6� | j d c d 7u u
ParseErroru typeu invalid-codepointu datau --�u !u, unexpected-bang-after-double-dash-in-commentu -u, unexpected-dash-after-double-dash-in-commentu eof-in-comment-double-dashu unexpected-char-in-commentu --( R R% R R& R R R R R� t commentEndBangStateR R5 ( R RJ ( ( sC /usr/lib/python2.7/site-packages/pip/_vendor/html5lib/_tokenizer.pyR� � s6 c C` s2 | j j � } | d k r= | j j | j � | j | _ n� | d k rk | j d c d 7<| j | _ n� | d k r� | j j i t d d 6d d 6� | j d c d 7<| j | _ ns | t
k r | j j i t d d 6d
d 6� | j j | j � | j | _ n# | j d c d | 7<| j | _ t S( Nu >u -u datau --!u u
ParseErroru typeu invalid-codepointu --!�u eof-in-comment-end-bang-state( R R% R R& R R R R� R R� R R5 ( R RJ ( ( sC /usr/lib/python2.7/site-packages/pip/_vendor/html5lib/_tokenizer.pyR� � s( c C` s� | j j � } | t k r* | j | _ n� | t k r� | j j i t d d 6d d 6� t | j
d <| j j | j
� | j | _ n> | j j i t d d 6d d 6� | j j | � | j | _ t
S( Nu
ParseErroru typeu! expected-doctype-name-but-got-eofu datau correctu need-space-after-doctype( R R% R t beforeDoctypeNameStateR R R R& R R R R R, R5 ( R RJ ( ( sC /usr/lib/python2.7/site-packages/pip/_vendor/html5lib/_tokenizer.pyR~ s
c C` s? | j j � } | t k r n | d k r{ | j j i t d d 6d d 6� t | j d <| j j | j � | j | _ n� | d k r� | j j i t d d 6d d 6� d | j d
<| j
| _ nv | t k r"| j j i t d d 6d d 6� t | j d <| j j | j � | j | _ n | | j d
<| j
| _ t S( Nu >u
ParseErroru typeu+ expected-doctype-name-but-got-right-bracketu datau correctu u invalid-codepointu �u nameu! expected-doctype-name-but-got-eof(
R R% R R R& R R R R R t doctypeNameStateR R5 ( R RJ ( ( sC /usr/lib/python2.7/site-packages/pip/_vendor/html5lib/_tokenizer.pyR� s.
c C` ss | j j � } | t k rG | j d j t � | j d <| j | _ n(| d k r� | j d j t � | j d <| j j | j � | j
| _ n� | d k r� | j j i t d d 6d d 6� | j d c d 7<| j | _ n� | t
k r\| j j i t d d 6d d 6� t | j d
<| j d j t � | j d <| j j | j � | j
| _ n | j d c | 7u u
ParseErroru typeu invalid-codepointu datau �u eof-in-doctype-nameu correct( R R% R R RD R t afterDoctypeNameStateR R R& R R R� R R R5 ( R RJ ( ( sC /usr/lib/python2.7/site-packages/pip/_vendor/html5lib/_tokenizer.pyR� 6 s,
c C` s� | j j � } | t k r n�| d k rL | j j | j � | j | _ n�| t k r� t | j d <| j j
| � | j j i t d d 6d d 6� | j j | j � | j | _ n9| d k r)t } xB d d! d" d# d$ f D]+ } | j j � } | | k r� t } Pq� q� W| r�| j
| _ t Snp | d% k r�t } xB d& d' d( d) d* f D]+ } | j j � } | | k rQt } PqQqQW| r�| j | _ t Sn | j j
| � | j j i t d d 6d d 6i | d 6d 6� t | j d <| j | _ t S(+ Nu >u correctu
ParseErroru typeu eof-in-doctypeu datau pu Pu uu Uu bu Bu lu Lu iu Iu cu Cu su Su yu Yu tu Tu eu Eu mu Mu* expected-space-or-right-bracket-in-doctypeu datavars( u pu P( u uu U( u bu B( u lu L( u iu I( u cu C( u su S( u yu Y( u su S( u tu T( u eu E( u mu M( R R% R R R& R R R R R R, R R5 t afterDoctypePublicKeywordStatet afterDoctypeSystemKeywordStatet bogusDoctypeState( R RJ R� R� ( ( sC /usr/lib/python2.7/site-packages/pip/_vendor/html5lib/_tokenizer.pyR� O sT
c C` s� | j j � } | t k r* | j | _ n� | d k rw | j j i t d d 6d d 6� | j j | � | j | _ ny | t k r� | j j i t d d 6d d 6� t
| j d <| j j | j � | j | _ n | j j | � | j | _ t
S(
Nu 'u "u
ParseErroru typeu unexpected-char-in-doctypeu datau eof-in-doctypeu correct( u 'u "( R R% R t" beforeDoctypePublicIdentifierStateR R R& R R, R R R R R5 ( R RJ ( ( sC /usr/lib/python2.7/site-packages/pip/_vendor/html5lib/_tokenizer.pyR� � s"
c C` sg | j j � } | t k r nE| d k rF d | j d <| j | _ n | d k rn d | j d <| j | _ n� | d k r� | j j i t d d 6d d 6� t
| j d
<| j j | j � | j | _ n� | t k r(| j j i t d d 6d d 6� t
| j d
<| j j | j � | j | _ n; | j j i t d d 6d d 6� t
| j d
<| j
| _ t S(
Nu "u u publicIdu 'u >u
ParseErroru typeu unexpected-end-of-doctypeu datau correctu eof-in-doctypeu unexpected-char-in-doctype( R R% R R t( doctypePublicIdentifierDoubleQuotedStateR t( doctypePublicIdentifierSingleQuotedStateR R& R R R R R� R5 ( R RJ ( ( sC /usr/lib/python2.7/site-packages/pip/_vendor/html5lib/_tokenizer.pyR� � s4
c C` s? | j j � } | d k r* | j | _ n| d k rn | j j i t d d 6d d 6� | j d c d 7u unexpected-end-of-doctypeu correctu eof-in-doctype( R R% t! afterDoctypePublicIdentifierStateR R R& R R R R R R5 ( R RJ ( ( sC /usr/lib/python2.7/site-packages/pip/_vendor/html5lib/_tokenizer.pyR� � s*
c C` s? | j j � } | d k r* | j | _ n| d k rn | j j i t d d 6d d 6� | j d c d 7u unexpected-end-of-doctypeu correctu eof-in-doctype( R R% R� R R R& R R R R R R5 ( R RJ ( ( sC /usr/lib/python2.7/site-packages/pip/_vendor/html5lib/_tokenizer.pyR� � s*
c C` s� | j j � } | t k r* | j | _ nZ| d k rX | j j | j � | j | _ n,| d k r� | j j i t d d 6d d 6� d | j d <| j
| _ n� | d k r� | j j i t d d 6d d 6� d | j d <| j | _ n� | t k rI| j j i t d d 6d
d 6� t
| j d <| j j | j � | j | _ n; | j j i t d d 6d d 6� t
| j d <| j | _ t S( Nu >u "u
ParseErroru typeu unexpected-char-in-doctypeu datau u systemIdu 'u eof-in-doctypeu correct( R R% R t- betweenDoctypePublicAndSystemIdentifiersStateR R R& R R R t( doctypeSystemIdentifierDoubleQuotedStatet( doctypeSystemIdentifierSingleQuotedStateR R R� R5 ( R RJ ( ( sC /usr/lib/python2.7/site-packages/pip/_vendor/html5lib/_tokenizer.pyR� � s6
c C` s8 | j j � } | t k r n| d k rL | j j | j � | j | _ n� | d k rt d | j d <| j | _ n� | d k r� d | j d <| j | _ n� | t
k r� | j j i t d d 6d d 6� t | j d
<| j j | j � | j | _ n; | j j i t d d 6d d 6� t | j d
<| j
| _ t S( Nu >u "u u systemIdu 'u
ParseErroru typeu eof-in-doctypeu datau correctu unexpected-char-in-doctype( R R% R R R& R R R R� R� R R R R� R5 ( R RJ ( ( sC /usr/lib/python2.7/site-packages/pip/_vendor/html5lib/_tokenizer.pyR� s.
c C` s� | j j � } | t k r* | j | _ n� | d k rw | j j i t d d 6d d 6� | j j | � | j | _ ny | t k r� | j j i t d d 6d d 6� t
| j d <| j j | j � | j | _ n | j j | � | j | _ t
S(
Nu 'u "u
ParseErroru typeu unexpected-char-in-doctypeu datau eof-in-doctypeu correct( u 'u "( R R% R t" beforeDoctypeSystemIdentifierStateR R R& R R, R R R R R5 ( R RJ ( ( sC /usr/lib/python2.7/site-packages/pip/_vendor/html5lib/_tokenizer.pyR� s"
c C` sg | j j � } | t k r nE| d k rF d | j d <| j | _ n | d k rn d | j d <| j | _ n� | d k r� | j j i t d d 6d d 6� t
| j d
<| j j | j � | j | _ n� | t k r(| j j i t d d 6d d 6� t
| j d
<| j j | j � | j | _ n; | j j i t d d 6d d 6� t
| j d
<| j
| _ t S( Nu "u u systemIdu 'u >u
ParseErroru typeu unexpected-char-in-doctypeu datau correctu eof-in-doctype( R R% R R R� R R� R R& R R R R R� R5 ( R RJ ( ( sC /usr/lib/python2.7/site-packages/pip/_vendor/html5lib/_tokenizer.pyR� / s4
c C` s? | j j � } | d k r* | j | _ n| d k rn | j j i t d d 6d d 6� | j d c d 7u unexpected-end-of-doctypeu correctu eof-in-doctype( R R% t! afterDoctypeSystemIdentifierStateR R R& R R R R R R5 ( R RJ ( ( sC /usr/lib/python2.7/site-packages/pip/_vendor/html5lib/_tokenizer.pyR� L s*
c C` s? | j j � } | d k r* | j | _ n| d k rn | j j i t d d 6d d 6� | j d c d 7u unexpected-end-of-doctypeu correctu eof-in-doctype( R R% R� R R R& R R R R R R5 ( R RJ ( ( sC /usr/lib/python2.7/site-packages/pip/_vendor/html5lib/_tokenizer.pyR� d s*
c C` s� | j j � } | t k r n� | d k rL | j j | j � | j | _ n� | t k r� | j j i t d d 6d d 6� t
| j d <| j j | j � | j | _ n. | j j i t d d 6d d 6� | j | _ t S( Nu >u
ParseErroru typeu eof-in-doctypeu datau correctu unexpected-char-in-doctype(
R R% R R R& R R R R R R R� R5 ( R RJ ( ( sC /usr/lib/python2.7/site-packages/pip/_vendor/html5lib/_tokenizer.pyR� | s
c C` s | j j � } | d k r= | j j | j � | j | _ n> | t k r{ | j j | � | j j | j � | j | _ n t S( Nu >(
R R% R R& R R R R R, R5 ( R RJ ( ( sC /usr/lib/python2.7/site-packages/pip/_vendor/html5lib/_tokenizer.pyR� � s c C` s` g } x� t r� | j | j j d � � | j | j j d � � | j j � } | t k r` Pq | d k sr t � | d d d k r� | d d | d i����i����u ]]u u i u
ParseErroru typeu invalid-codepointu datau �u
Characters( R5 R& R RI R% R t AssertionErrorR( t countt rangeR R R| R R ( R RJ R% t nullCountRw ( ( sC /usr/lib/python2.7/site-packages/pip/_vendor/html5lib/_tokenizer.pyR� � s0 N(N t __name__t
__module__t __doc__R R R$ R4 R RB RC RF R RG RN RL RP RR RS RH RU RV RM R[ R\ RO R_ R` RQ Ra Rc Rb Rd Rh Rf Re Rg Ri Rk Rj Rl Rm Ro Rn Rp RX Rq Rs Rr Rx Rz Ry R{ RY RW RT R} R� R� R� R� R� R~ R� R� R� R� R� R� R� R� R� R� R� R� R� R� R� R� ( ( ( sC /usr/lib/python2.7/site-packages/pip/_vendor/html5lib/_tokenizer.pyR s� HP #
6 " - 3 N( t
__future__R R R t pip._vendor.sixR R* t collectionsR t constantsR R R R R R
R R R
R t _inputstreamR t _trieR R6 t objectR ( ( ( sC /usr/lib/python2.7/site-packages/pip/_vendor/html5lib/_tokenizer.pyt s