o k=[?Pvp@sdZdZddlZddlZddlZddlZddlZddlZddlZddl m Z ddl m Z m Z mZddlmZddlmZmZdd lmZd!d d ZGd ddeZddZddZddZddZddZGdddeZGdddeZ GdddeZ!Gdd d e Z"dS)"zRefactoring framework. Used as a main program, this can refactor any number of files and/or recursively descend down directories. Imported as a module, this provides infrastructure to write your own refactoring tool. z#Guido van Rossum N)chain)drivertokenizetoken) find_root)pytreepygram) btm_matcherTcCsTt|ggdg}g}t|jD]\}}}|dr'|r"|dd}||q|S)zEReturn a sorted list of all available fix names in the given package.*fix_N) __import__pkgutilZ iter_modules__path__ startswithappend)Z fixer_pkgZ remove_prefixZpkgZ fix_namesfindernameZispkgr6/app/.heroku/python/lib/python3.10/lib2to3/refactor.pyget_all_fix_namess   rc@ eZdZdS) _EveryNodeN__name__ __module__ __qualname__rrrrr+rcCst|tjtjfr|jdurt|jhSt|tjr$|jr"t|jStt|tj rAt }|jD]}|D] }| t|q4q0|St d|)zf Accepts a pytree Pattern Node and returns a set of the pattern types which will match first. Nz$Oh no! I don't understand pattern %s) isinstancerZ NodePatternZ LeafPatterntyperZNegatedPatternZcontent_get_head_typesZWildcardPatternsetupdate Exception)patrpxrrrr!/s       r!c Cstt}g}|D];}|jr1zt|j}Wnty#||Yq w|D] }|||q&q |jdur?||j|q ||q tt j j t j j D] }|||qPt|S)z^ Accepts a list of fixers and returns a dictionary of head node type --> fixer list. N) collections defaultdictlistpatternr!rrZ _accept_typerr python_grammarZ symbol2numbervaluestokensextenddict)Z fixer_listZ head_nodesZeveryfixerZheadsZ node_typerrr_get_headnode_dictKs(     r3csfddtdDS)zN Return the fully qualified names for fixers in the package pkg_name. csg|]}d|qS.r).0fix_namepkg_namerr hsz+get_fixers_from_package..F)rr8rr8rget_fixers_from_packageds r;cCs|SNr)objrrr _identityksr>csjd}tt|jfdd}ttjtjtj h}t }z |\}}||vr*q|tj kr5|r2ncd}n_|tj kr|dkr|\}}|tj ksL|dkrMnR|\}}|tj ks[|dkr\nH|\}}|tj kro|dkro|\}}|tj kr|||\}}|tj ks|d krn |\}}|tj kstnnq Wt|SWt|SWt|SWt|StyYt|Sw) NFcst}|d|dfS)Nrr)next)tokgenrradvancersz(_detect_future_features..advanceTfromZ __future__import(,)rgenerate_tokensioStringIOreadline frozensetrNEWLINENLCOMMENTr"STRINGNAMEOPadd StopIteration)sourceZhave_docstringrCignorefeaturestpvaluerrAr_detect_future_featuresos\             rZc@seZdZdZdS) FixerErrorzA fixer could not be loaded.N)rrr__doc__rrrrr[sr[c@seZdZddddZdZdZd4ddZdd Zd d Zd d Z ddZ ddZ d5ddZ d5ddZ ddZd5ddZddZd6ddZddZd d!Z  d7d"d#Zd8d$d%Zd&Zd'Zd(d)Zd*d+Zd,d-Zd.d/Zd0d1Zd2d3ZdS)9RefactoringToolF)print_function exec_functionwrite_unchanged_filesZFixr NcCsH||_|pg|_|j|_|dur|j|tj|_|jdr)|jj d=n |jdr3|jj d=|j d|_ g|_ t d|_g|_d|_tj|jtj|jd |_|\|_|_g|_t|_g|_g|_t|j|jD]#}|j r~|j!|qr||jvr|j"|qr||jvr|j"|qrt#|j|_$t#|j|_%dS) zInitializer. Args: fixer_names: a list of fixers to import options: a dict with configuration. explicit: a list of fixers to run even if they are explicit. Nr^printr_execr`r]F)convertlogger)&fixersexplicit_default_optionscopyoptionsr#r r-grammarkeywordsgetr`errorslogging getLoggerrd fixer_logwroterZDriverrrc get_fixers pre_order post_orderfilesbmZ BottomMatcherBMZ bmi_pre_orderZbmi_post_orderrZ BM_compatibleZ add_fixerrr3bmi_pre_order_headsbmi_post_order_heads)selfZ fixer_namesrirfr2rrr__init__sD              zRefactoringTool.__init__c CsJg}g}|jD]}t|iidg}|ddd}||jr(|t|jd}|d}|jddd |D}zt ||}Wnt yQt d ||fdw||j |j } | jrm|jd urm||jvrm|d |q|d || jdkr~|| q| jdkr|| qt d| jtd} |j| d|j| d||fS)aInspects the options to load the requested patterns and handlers. Returns: (pre_order, post_order), where pre_order is the list of fixers that want a pre-order AST traversal, and post_order is the list that want post-order traversal. r r5rN_cSsg|]}|qSr)title)r6r'rrrr:sz.RefactoringTool.get_fixers..zCan't find %s.%sTzSkipping optional fixer: %szAdding transformation: %spreZpostzIllegal fixer order: %rZ run_orderkey)rerrsplitr FILE_PREFIXlensplit CLASS_PREFIXjoingetattrAttributeErrorr[rirprf log_message log_debugorderroperator attrgettersort) rzZpre_order_fixersZpost_order_fixersZ fix_mod_pathmodr7parts class_nameZ fix_classr2Zkey_funcrrrrrs<             zRefactoringTool.get_fixerscOs)zCalled when an error occurs.r)rzmsgargskwdsrrr log_errorszRefactoringTool.log_errorcGs|r||}|j|dS)zHook to log a message.N)rdinforzrrrrrrszRefactoringTool.log_messagecGs|r||}|j|dSr<)rddebugrrrrr szRefactoringTool.log_debugcCsdS)zTCalled with the old version, new version, and filename of a refactored file.Nr)rzold_textnew_textfilenameequalrrr print_outputszRefactoringTool.print_outputcCs8|D]}tj|r||||q||||qdS)z)Refactor a list of files and directories.N)ospathisdir refactor_dir refactor_file)rzitemswrite doctests_onlyZ dir_or_filerrrrefactors  zRefactoringTool.refactorc Cstjd}t|D]@\}}}|d||||D]}|ds>tj|d|kr>tj||} | | ||qdd|D|dd<q dS)zDescends down a directory and refactor every Python file found. Python files are assumed to have a .py extension. Files and subdirectories starting with '.' are skipped. pyzDescending into %sr5rcSsg|] }|ds|qSr4)r)r6Zdnrrrr:2z0RefactoringTool.refactor_dir..N) rextsepwalkrrrrsplitextrr) rzZdir_namerrZpy_extdirpathdirnames filenamesrfullnamerrrr s   zRefactoringTool.refactor_dirc Cszt|d}Wnty!}z|d||WYd}~dSd}~wwzt|jd}W|n|wtj|d|dd}||fWdS1sPwYdS) zG Do our best to decode a Python source file correctly. rbzCan't open %s: %sNNNrr&r~encodingnewline) openOSErrorrrdetect_encodingrKcloserIread)rzrferrrrrr_read_python_source4s $z#RefactoringTool._read_python_sourcecCs||\}}|dur dS|d7}|r9|d||||}|js&||kr1||||||dS|d|dS|||}|jsG|rX|jrX|jt|dd|||ddS|d|dS)zRefactors a file.N zRefactoring doctests in %szNo doctest changes in %sr|)rrzNo changes in %s)rrrefactor_docstringr`processed_filerefactor_string was_changedstr)rzrrrinputroutputtreerrrrDs     zRefactoringTool.refactor_filec Cst|}d|vr tj|j_z3z|j|}Wn#ty9}z|d||jj |WYd}~W|j|j_dSd}~wwW|j|j_n|j|j_w||_ | d|| |||S)aFRefactor a given input string. Args: data: a string holding the code to be refactored. name: a human-readable name for use in error/log messages. Returns: An AST corresponding to the refactored input stream; None if there were errors during the parse. r^zCan't parse %s: %s: %sNzRefactoring %s) rZr Z!python_grammar_no_print_statementrrjZ parse_stringr$r __class__rfuture_featuresr refactor_tree)rzdatarrWrrrrrr[s&      zRefactoringTool.refactor_stringcCstj}|r)|d||d}|js||kr"||d|dS|ddS||d}|js7|rB|jrB|t |d|dS|ddS)NzRefactoring doctests in stdinzzNo doctest changes in stdinzNo changes in stdin) sysstdinrrrr`rrrr)rzrrrrrrrrefactor_stdinvs    zRefactoringTool.refactor_stdinc Cst|j|jD]}|||q||j|||j||j| }t | r|jj D]}||vr||r||j tjjdd|jrV||j tjjdt||D]o}|||vrk|||zt|Wn tyzYq\w|jr||jvrq\||}|r|||}|dur|||D]}|jsg|_|j|q|j| }|D]} | |vrg|| <|| || qq\q4t | s0t|j|jD]}|||q|jS)aRefactors a parse tree (modifying the tree in place). For compatible patterns the bottom matcher module is used. Otherwise the tree is traversed node-to-node for matches. Args: tree: a pytree.Node instance representing the root of the tree to be refactored. name: a human-readable name for this tree. Returns: True if the tree was modified, False otherwise. T)rreverserN)rrsrtZ start_tree traverse_byrxryrwrunZleavesanyr.rerrZBasedepthZkeep_line_orderZ get_linenor+remover ValueErrorZfixers_appliedmatch transformreplacerr0Z finish_treer) rzrrr2Z match_setnoderesultsnewZ new_matchesZfxrrrrrsP          1zRefactoringTool.refactor_treecCsV|sdS|D]"}||jD]}||}|r'|||}|dur'|||}q qdS)aTraverse an AST, applying a set of fixers to each node. This is a helper method for refactor_tree(). Args: fixers: a list of fixer instances. traversal: a generator that yields AST nodes. Returns: None N)r rrr)rzreZ traversalrr2rrrrrrs    zRefactoringTool.traverse_bycCs|j||dur||d}|durdS||k}||||||r0|d||js0dS|r<|||||dS|d|dS)zR Called when a file has been refactored and there may be changes. NrzNo changes to %szNot writing changes to %s)rurrrrr` write_file)rzrrrrrrrrrrs  zRefactoringTool.processed_filec Csz tj|d|dd}Wnty%}z|d||WYd}~dSd}~ww|)z||WntyI}z |d||WYd}~nd}~wwWdn1sTwY|d|d|_dS) zWrites a string to a file. It first shows a unified diff between the old text and the new text, and then rewrites the file; the latter is only done if the write option is set. wr~rzCan't create %s: %sNzCan't write %s: %szWrote changes to %sT)rIrrrrrrq)rzrrrrfprrrrrs$  zRefactoringTool.write_filez>>> z... c Csg}d}d}d}d}|jddD]d}|d7}||jr?|dur-|||||||}|g}||j} |d| }q|dur\|||jsV|||jdkr\| |q|durk||||||d}d}| |q|dur||||||d |S)aRefactors a docstring, looking for doctests. This returns a modified version of the input string. It looks for doctests, which start with a ">>>" prompt, and may be continued with "..." prompts, as long as the "..." is indented the same as the ">>>". (Unfortunately we can't use the doctest module's parser, since, like most parsers, it is not geared towards preserving the original source.) NrTkeependsrrr~) splitlineslstriprPS1r0refactor_doctestfindPS2rstriprr) rzrrresultblockZ block_linenoindentlinenolineirrrrsD        z"RefactoringTool.refactor_docstringc s(z ||}Wn4ty=}z(jtjr&|D] }d|dqd|||j j ||WYd}~Sd}~ww ||rt |j dd}|d|d||dd} }| dg|dksjJ| |ddsy|dd7<j|d g}|r|fd d |D7}|S) zRefactors one doctest. A doctest is given as a block of lines, the first of which starts with ">>>" (possibly indented), while the remaining lines start with "..." (identically indented). z Source: %srz+Can't parse docstring in %s line %s: %s: %sNTrrr|rcsg|] }j|qSr)r)r6rrrzrrr:^rz4RefactoringTool.refactor_doctest..) parse_blockr$rd isEnabledForrnDEBUGrrrrrrrrendswithrpop) rzrrrrrrrrZclippedrrrrDs,  "z RefactoringTool.refactor_doctestcCs|jrd}nd}|js|d|n|d||jD]}||q|jr6|d|jD]}||q.|jrdt|jdkrF|dn |dt|j|jD]\}}}|j|g|Ri|qRdSdS) Nwerez need to bezNo files %s modified.zFiles that %s modified:z$Warnings/messages while refactoring:rzThere was 1 error:zThere were %d errors:)rqrurrprmr)rzrfilemessagerrrrrr summarizeas(       zRefactoringTool.summarizecCs"|j||||}t|_|S)zParses a block into a tree. This is necessary to get correct line number / offset information in the parser diagnostics and embedded into the parse tree. )rZ parse_tokens wrap_toksrLr)rzrrrrrrrrxszRefactoringTool.parse_blockc csft|||j}|D]#\}}\}}\} } } ||d7}| |d7} ||||f| | f| fVq dS)z;Wraps a tokenize stream to systematically modify start/end.rN)rrH gen_lines__next__) rzrrrr/r rYZline0Zcol0Zline1Zcol1Z line_textrrrrs  zRefactoringTool.wrap_toksccsx||j}||j}|}|D]'}||r |t|dVn||dkr,dVntd||f|}q dVq8)zGenerates lines as expected by tokenize from a list of lines. This strips the first len(indent + self.PS1) characters off each line. Nrzline=%r, prefix=%rTr~)rrrrrAssertionError)rzrrprefix1Zprefix2prefixrrrrrs   zRefactoringTool.gen_linesr)FF)F)NFNr<)rrrrgrrr{rrrrrrrrrrrrrrrrrrrrrrrrrrrrr]sB 7(    O + r]c@r)MultiprocessingUnsupportedNrrrrrrrrcsFeZdZfddZ  d fdd ZfddZfd d ZZS) MultiprocessRefactoringToolcs&tt|j|i|d|_d|_dSr<)superrr{queue output_lockrzrkwargsrrrr{s z$MultiprocessRefactoringTool.__init__Frc s2|dkrtt|||SzddlWn tytwjdur'td_ _ fddt |D}z8|D]}| q@tt|||Wj t |D]}jdq[|D] }|rp| qfd_dSj t |D]}jdq|D] }|r| qd_w)Nrrz already doing multiple processescsg|] }jjdqS))target)Process_child)r6rmultiprocessingrzrrr:sz8MultiprocessRefactoringTool.refactor..)rrrr  ImportErrorrr RuntimeError JoinableQueueLockrrangestartrputis_alive)rzrrrZ num_processes processesr'rrr rrsL            z$MultiprocessRefactoringTool.refactorcsf|j}|dur1|\}}ztt|j|i|W|jn|jw|j}|dus dSdSr<)rrlrrr task_done)rzZtaskrrrrrrs   z"MultiprocessRefactoringTool._childcs4|jdur|j||fdStt|j|i|Sr<)rrrrrrrrrrs  z)MultiprocessRefactoringTool.refactor_file)FFr)rrrr{rrr __classcell__rrrrrs   r)T)#r\ __author__rIrrrrnrr) itertoolsrZpgen2rrrZ fixer_utilrr~rr r rvrr$rr!r3r;r>rZr[objectr]rrrrrrs:     (