관리-도구
편집 파일: __init__.cpython-39.pyc
a �)g�' � @ s� d Z ddlmZmZmZ eZddlZddlZddl Z ddl Z ddl Z ddl Z ddl Z ddlZddlZddlZddlZddlZddlZddlZddlZddlZddlmZ ddlmZ ddl m Z m Z ddl m!Z! ddl"m#Z# dd l$m%Z% dd l&m'Z' zdd l(m)Z* W n& e+�y* G d d � d �Z*dZ,Y n0 dZ,z ddl-m.Z. ddl/m0Z0 W n e+�yd dZ1Y n0 dZ1ej2�r4ddl3m4Z4 ej5d Z6ej5d Z7ej5dddddddd d d d d d!d"e7f Z8ej9e8ej:e;e<ej=e< ej9e<e<f df f Z>ej9e6e>f Z?ej9e7ej:e<e;df f Z@ej9ej5d# ej:ej=e@ e;f f ZAddlBmCZD dd$lEmFZF dd%lGmHZH dd&l3mIZImJZJmKZKmLZLmMZM dd'lNmOZO dd(lPmQZQmRZRmSZSmTZT z4dd)lUmVZV dd*lWmXZXmYZY dd+lZm[Z[m\Z\m]Z] W n e+�y� dZ^Y n0 dZ^dd,l_m`Z`m)Z)maZa dd-lbmcZc dd.ldmeZe dd/lfmgZg dd0lhmiZimjZjmkZk dd1llmmZm dd2lnmoZo dd3lpmqZq dd4lrmsZs dd5ltmuZumvZv dd6lwmxZx es� ayd7Zzd8Z{ed9g d:��Z|d;Z}e G d<d=� d=��Z~G d>d?� d?e�Z�G d@dA� dA�Z�dBdC� Z�dDdE� Z�dFdG� Z�dHdI� Z�dJdK� Z�dLdM� Z�dNdO� Z�dPdQ� Z�dRdS� Z�dTdU� Z�edVdW� �Z�ed�dXdY��Z�dZd[� Z�d\d]� Z�d�d_d`�Z�dadb� Z�dcdd� Z�dedf� Z�dgdh� Z�didj� Z�dkdl� Z�dmdn� Z�dodp� Z�dqdr� Z�dsdt� Z�dudv� Z�dwdx� Z�dydz� Z�d�d{d|�Z�d}d~� Z�dd�� Z�d�d�� Z�d�d�� Z�d�d�d��Z�d�d�� Z�dS )�z)Installed collections management package.� )�absolute_import�division�print_functionN)� namedtuple)�contextmanager)� dataclass�fields)�sha256)�BytesIO)� distribution)�chain)� Requirementc @ s e Zd ZdS )�PkgReqN)�__name__� __module__� __qualname__� r r �F/usr/lib/python3.9/site-packages/ansible/galaxy/collection/__init__.pyr % s r FT)�Manifest)�DistlibException)�ConcreteArtifactsManager��collection_info�file_manifest_file�format��name�ftype� chksum_type� chksum_sha256r � namespacer �version�authors�readme�tags� description�license� license_file� dependencies� repository� documentation�homepage�issues��filesr )� AnsibleError)� GalaxyAPI)� _consume_file�_download_file� _get_json_from_installed_dir�_get_meta_from_src_dir�_tarfile_extract)�MultiGalaxyAPIProxy)�run_gpg_verify�parse_gpg_errors�get_signature_from_source� GPG_ERROR_MAP)�$build_collection_dependency_resolver)�(CollectionDependencyResolutionImpossible�)CollectionDependencyInconsistentCandidate)�RESOLVELIB_VERSION�RESOLVELIB_LOWERBOUND�RESOLVELIB_UPPERBOUND)� Candidater � _is_installed_collection_dir)�meets_requirements)�get_all_plugin_loaders)� raise_from)�to_bytes� to_native�to_text)� is_sequence)� yaml_dump)�AnsibleCollectionRef)�Display)� secure_hash� secure_hash_s)�Sentinel� z MANIFEST.json�ModifiedContent��filename�expected� installedz0^(?P<strict>\+)?(?:(?P<count>\d+)|(?P<all>all))$c s: e Zd ZU dZee ed< dZeed< � fdd�Z � Z S )�ManifestControlN� directivesF�omit_default_directivesc s6 t | �D ](}t| |j�d u rt� �|j|�� � qd S �N)� dc_fields�getattrr �super� __setattr__�type)�selfZfield�� __class__r r � __post_init__� s z ManifestControl.__post_init__) r r r rW �list�str�__annotations__rX �boolrb � __classcell__r r r` r rV � s rV c @ s6 e Zd Zd dd�Zdd� Zdd� Zd d � Zd d � ZdS )�CollectionSignatureErrorNFc C s" || _ || _|| _|| _d | _d S rY )�reasons�stdout�rc�ignore�_reason_wrapper)r_ ri rj rk rl r r r �__init__� s z!CollectionSignatureError.__init__c C s d|� d| j � d| j� �S )NzUnexpected error for 'z<': GnuPG signature verification failed with the return code z and output )rk rj �r_ �collection_namer r r �_report_unexpected� s ���z+CollectionSignatureError._report_unexpectedc C s d|� d| j � d�}|| �� S )N�#Signature verification failed for 'z' (return code z):)rk �_format_reasons)r_ rp �headerr r r �_report_expected� s z)CollectionSignatureError._report_expectedc s< � j d u rtjddd�� _ � fdd�� jD �}dd�|� S )Nz * z )�initial_indent�subsequent_indentc s g | ]}d � � j�|���qS )� )�joinrm �wrap)�.0�reason�r_ r r � <listcomp>� s �z<CollectionSignatureError._format_reasons.<locals>.<listcomp>rx )rm �textwrapZ TextWrapperri ry )r_ Zwrapped_reasonsr r} r rs � s � �z(CollectionSignatureError._format_reasonsc C s | j r| �|�S | �|�S rY )ri ru rq ro r r r �report� s z CollectionSignatureError.report)NNNF)r r r rn rq ru rs r� r r r r rh � s rh c @ s e Zd Zdd� ZdS )�CollectionVerifyResultc C s || _ d| _d S )NT)rp �successro r r r rn � s z CollectionVerifyResult.__init__N)r r r rn r r r r r� � s r� c C s t | j�}t| jdd�}t�dj| d�� t�djt| j�d�� g }|du }t�t |�}t�t |�}|s�| j |j kr�dj| j| j |j d �} t�| � d |_ |S t j�t|dd�t�} t| j�} |r�| jdur�d d � | jd D �| } n |�s|j�rt|j�| } |jdu} | �s&| �r&t�d� n<| �rbt| j| | |j|j|j��sRd |_ |S t�d| � �� |�rr|t�} nt| �r�|j�r�|t�} n\|j�r�|jn|j |�}t� djt|�d�� t�t |�}t�t |�}|t�} t |t| |� t�dj| d�� |t�}|d }|d }|d|d }t ||||� ||�}t!� }t j�|d�t j�|d�h}|d D ]�}|d }|d dk�r�|�"t j�|t|dd��� |d|d }t ||||� |d d k�r`|�"t j�|t|dd��� �q`t �#|�D ]�\}}}|D ]N}t j�||�}t|t$|�d d� dd�} ||v�r�|�%t&| d d d �� �q�|D ]N}t j�||�}t|t$|�d d� dd�} ||v�rN|�%t&| d!d"d �� �qN�q�|�r�d |_ t�d#jt| j�d$�� |D ],} t�d%| j' � t�(d&| j)| j*f � �q�n"|�rd'nd(} t�d)j| | d*�� |S )+a5 Verify integrity of the locally installed collection. :param local_collection: Collection being checked. :param remote_collection: Upstream collection (optional, if None, only verify local artifact) :param artifacts_manager: Artifacts manager. :return: a collection verify result object. �surrogate_or_strict��errorszVerifying '{coll!s}'.��collz(Installed collection found at '{path!s}'��pathNzX{local_fqcn!s} has the version '{local_ver!s}' but is being compared to '{remote_ver!s}')Z local_fqcnZ local_verZ remote_verFc S s g | ] }|d �qS )� signaturer )r{ �infor r r r~ � � z+verify_local_collection.<locals>.<listcomp>� signaturesz�The GnuPG keyring used for collection signature verification was not configured but signatures were provided by the Galaxy server. Configure a keyring for ansible-galaxy to verify the origin of the collection. Skipping signature verification.z>GnuPG signature verification succeeded, verifying contents of z&Remote collection cached as '{path!s}'z#MANIFEST.json hash: {manifest_hash})� manifest_hashr r z chksum_%sr � MANIFEST.json� FILES.jsonr. r �file�dirrP zthe file does not existzthe file existsrR z the directory does not existzthe directory existszECollection {fqcn!s} contains modified content in the following files:��fqcnz %sz Expected: %s Found: %sz+are internally consistent with its manifestzmatch the remote collectionz=Successfully verified that checksums for '{coll!s}' {what!s}.)r� �what)+r� r� rF �src�displayr rH � functools�partialr3 �_get_file_hash�verr� �osr� ry �MANIFEST_FILENAMErc r� Z source_info�keyring�warning�verify_file_signatures�#required_successful_signature_count�ignore_signature_errors�vvvv�is_concrete_artifact�get_artifact_path�get_galaxy_artifact_path�vvv�_get_json_from_tar_file�_get_tar_file_hash�_verify_file_hash�set�add�walk�len�appendrQ rS �vrT rU ) �local_collection�remote_collection�artifacts_manager�result�b_collection_pathZmodified_contentZverify_local_onlyZ get_json_from_validation_sourceZ get_hash_from_validation_source�err� manifest_filer� Zkeyring_configuredr� Zb_temp_tar_path�manifestZfile_manifest_dataZfile_manifest_filename� expected_hash� file_manifestZcollection_dirsZcollection_filesZ manifest_datar �root�dirsr. � full_pathr� Zcontent_changer� r r r �verify_local_collection� s� �� �� �� � �� � �� � � �� ��r� c C s\ d}g }t �t|��� }|d p"d} |d } |d } | d urDt| �} |D ]�} t| dd�} zt|| ||� W nF ty� } z.| jr�W Y d } ~ qH|� | � | �� W Y d } ~ qHd } ~ 0 0 |d7 }| r�qH|| krH q�qH| r�|s�d}t � d | � d �� nN| �r| }|�s<t � d | � d �� n(| �p"| |k}|�s<t � d | � d �� |�sX|D ]}t � |� �qF|S ) Nr �strictF�all�countr� r� rP rr z': no successful signaturesz': some signatures failedz,': fewer successful signatures than required) �re�match�SIGNATURE_COUNT_RE� groupdict�intrH �verify_file_signaturerh rl r� r� r� r� )r� r� Zdetached_signaturesr� Zrequired_successful_countr� Z successfulZerror_messagesZ signature_count_requirementsr� Z require_allZ require_countr� �errorZverified�msgr r r r� y sF &r� c C s� t | ||t�\}}|r�t|�}z t|�}W n ty< Y nz0 g }d} t|g|�D ]B}tt�� �tt� � �� |j � } | |v r�| d7 } |� |� � � qRt|�| k} tt|�||| d��|r�t||d��dS )zURun the gpg command and parse any errors. Raises CollectionSignatureError on failure.r rP )ri rj rk rl )rj rk N)r7 r� r8 �next� StopIterationr rc r: �keys�values�indexra r� Zget_gpg_error_descriptionr� rh r� ) r� Zdetached_signaturer� r� Z gpg_resultZgpg_verification_rcr� r� ri Zignored_reasonsZ status_coderl r r r r� � s& r� c C s t | dd�}z t|�}W n6 tyN } z ttt|��|� W Y d}~n d}~0 0 tf i |��}t||d |d |d |d �}dj|d |d |d d �}t j � t |�t |dd��} t j � | �r�t j � | �r�td t| � ��n|s�td t| � ��t|| ||�} | S ) a� Creates the Ansible collection artifact in a .tar.gz file. :param u_collection_path: The path to the collection to build. This should be the directory that contains the galaxy.yml file. :param u_output_path: The path to create the collection build artifact. This should be a directory. :param force: Whether to overwrite an existing collection build artifact or fail. :return: The path to the collection build artifact. r� r� Nr r � build_ignorer� z {ns!s}-{name!s}-{ver!s}.tar.gzr! )r �nsr� zQThe output collection artifact '%s' already exists, but is a directory - abortingzWThe file '%s' already exists. You can use --force to re-create the collection artifact.)rF r4 � LookupErrorrE r/ rG �_build_manifest�_build_files_manifestr r� r� ry �exists�isdir�_build_collection_tar) �u_collection_pathZ u_output_path�forcer� �collection_metaZ lookup_err�collection_manifestr� Zartifact_tarball_file_name�b_collection_output�collection_outputr r r �build_collection� sB (�� � ��r� c C s� t d��. tt| �|d|||dddd� }W d � n1 s<0 Y t|dd�}g }t dj|d����| |�� �� D ]�\} } | jr�t�d jt | �d �� qvt�d jt | �t |�d �� | j r�|j n|j | �} t j�|t j�| ��} | j�rttt | dd�t |dd�d d�dd�} nt�t| �t| �� t�dj| d �� |�tt j�| ��| jd�� qvt j�|d�} t| dd�}t�djt | �d�� ttd|i�dd�}t|dd��}|�|� W d � n1 �s�0 Y W d � n1 �s�0 Y dS )a� Download Ansible collections as their tarball from a Galaxy server to the path specified and creates a requirements file of the downloaded requirements to be used for an install. :param collections: The collections to download, should be a list of tuples with (name, requirement, Galaxy Server). :param output_path: The path to download the collections to. :param apis: A list of GalaxyAPIs to query when search for a collection. :param validate_certs: Whether to validate the certificate if downloading a tarball from a non-Galaxy host. :param no_deps: Ignore any collection dependencies and only download the base requirements. :param allow_pre_release: Do not ignore pre-release versions when selecting the latest. z Process download dependency mapNF�� galaxy_apis�preferred_candidates�concrete_artifacts_manager�no_deps�allow_pre_release�upgrade�include_signatures�offliner� r� z2Starting collection download process to '{path!s}'r� z {coll!s} is not downloadabler� z/Downloading collection '{coll!s}' to '{path!s}'�r� r� T)r� z1Collection '{coll!s}' was downloaded successfully)r r! zrequirements.ymlzEWriting requirements.yml file of downloaded collections to '{path!s}'� collections�wb��mode)�_display_progress�_resolve_depenency_mapr� rF r �copy�items� is_virtualr� rH r� r� r� r� r� ry �basename�is_dirr� �shutilrG r� r� rJ �open�write)r� � output_path�apisr� r� r� Zdep_mapZ b_output_pathZ requirementsr� �concrete_coll_pin� b_src_pathZ b_dest_pathZrequirements_pathZb_requirements_pathZ yaml_bytesZreq_fdr r r �download_collections� s� �$ ���� ���� � �� ��� ��� �r� c C s� |� | �}|r�d}t|�d��D ]}|r |} q2q |sBtd| ��tdj|d��� |�||� W d � n1 st0 Y t�d|j|j f � nt�d|j|j |f � dS )ay Publish an Ansible collection tarball into an Ansible Galaxy server. :param collection_path: The path to the collection tarball to publish. :param api: A GalaxyAPI to publish the collection to. :param wait: Whether to wait until the import process is complete. :param timeout: The time in seconds to wait for the import process to finish, 0 is indefinite. N�/zoPublishing the collection did not return valid task info. Cannot wait for task status. Returned task info: '%s'zRCollection has been published to the Galaxy server {api.name!s} {api.api_server!s})�apizRCollection has been successfully published and imported to the Galaxy server %s %sz�Collection has been pushed to the Galaxy server %s %s, not waiting until import has completed due to --no-wait being set. Import task results can be found at %s) �publish_collection�reversed�splitr/ r� r Zwait_import_taskr� r Z api_server)�collection_pathr� �wait�timeoutZ import_uriZtask_idZ path_segmentr r r r� d s* ��* � �r� c s dd� t |� �D ��tt�� fdd�| D ���} dd� | D ��| |sH|rNt� n�fdd�| D �8 } | sx|sxt�d� dS �fd d��D �} |r�g n |r�| n�}d d� |D �}td ��, t| ||� |||| | d � }W d � n1 s�0 Y � jdu}td ��� |�� D ]�\}}|j �r6t� dj t |�d�� �q ||v �rZt�dj t |�d�� �q | �sx|j �rx|�sxt�d� |jdk�r�|�� }zt||� � W nL t�y� } z2|�r�t�dj t |�t |�d�� n� W Y d}~n d}~0 0 �q W d � n1 �s0 Y dS )a� Install Ansible collections to the path specified. :param collections: The collections to install. :param output_path: The path to install the collections to. :param apis: A list of GalaxyAPIs to query when searching for a collection. :param validate_certs: Whether to validate the certificates if downloading a tarball. :param ignore_errors: Whether to ignore any errors when installing the collection. :param no_deps: Ignore any collection dependencies and only install the base requirements. :param force: Re-install a collection if it has already been installed. :param force_deps: Re-install a collection as well as its dependencies if they have already been installed. c S s$ h | ] }t |j|j|j|jd ��qS rY )r r� r� r� r^ �r{ r� r r r � <setcomp>� s �z&install_collections.<locals>.<setcomp>c 3 s6 | ].}|j r(� fd d�� �|��� D �n|fV qdS )c 3 s | ]}t �|� �V qd S rY )r � from_dir_path)r{ Zsub_coll�r� r r � <genexpr>� s �z0install_collections.<locals>.<genexpr>.<genexpr>N)Z is_subdirsZ"get_direct_collection_dependenciesr� )r{ Z install_reqr r r r � s �� ��z&install_collections.<locals>.<genexpr>c S s h | ] }|j �qS r r� )r{ �reqr r r r � r� c s4 h | ],}� D ]"}|j |j kr t|j|j�r |�q qS r )r� rC r� )r{ r Zexs)�existing_collectionsr r r � s �zxNothing to do. All requested collections are already installed. If you want to reinstall them, consider using `--force`.Nc s h | ]}|j � vr|�qS r r� r )� requested_requirements_namesr r r � s �c S s$ h | ] }t |j|j|j|jd ��qS rY )rA r� r� r� r^ r r r r r � s �z Process install dependency mapr� z#Starting collection install processz Encountered {coll!s}, skipping.r� z*'{coll!s}' is already installed, skipping.a The GnuPG keyring used for collection signature verification was not configured but signatures were provided by the Galaxy server to verify authenticity. Configure a keyring for ansible-galaxy to use or disable signature verification. Skipping signature verification.�galaxyzeFailed to install collection {coll!s} but skipping due to --ignore-errors being set. Error: {error!s})r� r� )�find_existing_collectionsr� r � from_iterabler� r� r� r� r� r� r� r rH r� r� r^ Zwith_signatures_repopulated�installr/ )r� r� r� � ignore_errorsr� r� Z force_depsr� r� r� Zdisable_gpg_verifyr� Zunsatisfied_requirementsZ"existing_non_requested_collectionsZpreferred_requirementsZpreferred_collectionsZdependency_mapZkeyring_existsr� r� r� r )r� r r r �install_collections� s� � ���� ��� �$ �� ��� �� r c C s. | � d�\}}}t�|�r | S td| ��dS )z�Validates the collection name as an input from the user or a requirements file fit the requirements. :param name: The input name with optional range specifier split by ':'. :return: The input value, required for argparse validation. �:z�Invalid collection name '%s', name must be in the format <namespace>.<collection>. Please make sure namespace and collection name contains characters from [a-zA-Z0-9_] only.N)� partitionrK Zis_valid_collection_namer/ )r � collection�dummyr r r �validate_collection_name s �r c C s& t j�| �d dkr"t j�| d�S | S )z�Ensure a given path ends with 'ansible_collections' :param collection_path: The path that should end in 'ansible_collections' :return: collection_path ending in 'ansible_collections' if it does not already. rP Zansible_collections)r� r� r� ry )r r r r �validate_collection_path' s r c C s2 g }t ||�}t� �� | D �]�}�z�|jr@tdj|jd�d��d|j } |D ]�} ttj � | |j |j �dd�} tj � | �s|qNt| �s�d|j } qNt�| |�} dd � |jp�g D �} t| j| j| j| jt| �d �} q�qNt| d��|r�d }n�|�| �}|�d d � |j�pg D �� t|j|jd k�r&|jn| jd dt|��}z |jd u �sL|�sV|�|� W nP t�y� } z6dj|d�}|j|k�r�tdj|d���� W Y d }~n d }~0 0 t| ||�}|�|� W q t�y } z.|�r�t � dj|t |�d�� n� W Y d }~q d }~0 0 q W d � n1 �s$0 Y |S )aM Verify the integrity of locally installed collections. :param collections: The collections to check. :param search_paths: Locations for the local collection lookup. :param apis: A list of GalaxyAPIs to query when searching for a collection. :param ignore_errors: Whether to ignore any errors when verifying the collection. :param local_verify_only: When True, skip downloads and only verify local manifests. :param artifacts_manager: Artifacts manager. :return: list of CollectionVerifyResult objects describing the results of each collection verification zM'{coll_type!s}' type is not supported. The format namespace.name is expected.)Z coll_type)�messagez>Collection %s is not installed in any of the collection paths.r� r� z�Collection %s does not have a MANIFEST.json. A MANIFEST.json is expected if the collection has been built and installed via ansible-galaxyc S s g | ]}t |t��qS r �r9 r� �r{ �sourcer r r r~ m s �z&verify_collections.<locals>.<listcomp>)r� Nc S s g | ]}t |t��qS r r r r r r r~ � s ��*r z4Failed to find collection {coll.fqcn!s}:{coll.ver!s}r� zHFailed to find remote collection '{coll!s}' on any of the galaxy serverszdFailed to verify collection '{coll!s}' but skipping due to --ignore-errors being set. Error: {err!s})r� r� ) r6 r� r� r/ r r^ r� rF r� r� ry r r r� rB rA r Zsignature_sourcesr� r� � frozensetZget_signatures�extendr� Z get_collection_version_metadatar r� r� r� r� rH )r� Z search_pathsr� r Zlocal_verify_onlyr� �resultsZ api_proxyr Z default_err� search_pathZ b_search_pathr� Zsupplemental_signaturesr� r� �eZexpected_error_msgr� r� r r r �verify_collections4 s� �� �� ����� �� ��� �� ��:r! c c s< t jttjdd�d�} z| V W t�| � n t�| � 0 d S )Nr� r� )r� )�tempfileZmkdtemprF �CZDEFAULT_LOCAL_TMPr� �rmtree)� b_temp_pathr r r �_tempdir� s r& c c s� t j}|d u rtj�� n|}| d ur.t�| � |s<d V d S dd� }G dd� dt�}t}z|z^t�� }||�at j |||fd�}d|_ |� � zd V W d|_ |�� nd|_ |�� 0 W n ty� � Y n0 W |an|a0 d S )Nc S s� |� d� t�� }dD ]�}|j|d dd� t�d� z| jddd�\}}}W n tjyf Y q�Y q60 t ||�}||i |�� q6t |dd�r|� d � d S qqd S ) Nz(Starting display_progress display threadz|/-\�F)�newlineg�������?)�blockr �finishz7Received end signal for display_progress display thread) �debug� threading�current_threadr� �time�sleep�get�queueZEmptyr[ )� display_queueZactual_display�t�c�method�args�kwargs�funcr r r �progress� s z#_display_progress.<locals>.progressc @ s e Zd Zdd� Zdd� ZdS )z(_display_progress.<locals>.DisplayThreadc S s || _ d S rY )r2 )r_ r2 r r r rn � s z1_display_progress.<locals>.DisplayThread.__init__c s � �fdd�}|S )Nc s �j �� | |f� d S rY )r2 Zput)r6 r7 ��attrr_ r r � call_display� s zJ_display_progress.<locals>.DisplayThread.__getattr__.<locals>.call_displayr )r_ r; r< r r: r � __getattr__� s z4_display_progress.<locals>.DisplayThread.__getattr__N)r r r rn r= r r r r � DisplayThread� s r> )�targetr6 T)r# ZGALAXY_DISPLAY_PROGRESS�sysrj �isattyr� �objectr1 ZQueuer, �Thread�daemon�startr* ry � Exception)r� Zconfig_displayZ display_wheelr9 r> Z old_displayr2 r3 r r r r� � s2 � r� c C s� t tj�t| �|�dd�}tj�|�s,d }n4t|dd��}t|�}W d � n1 sV0 Y ||kr||�t |||d�� d S )Nr� r� �rbr� rR ) rF r� r� ry rH �isfiler� r1 r� rQ )�b_pathrS r� Z error_queueZ b_file_path� actual_hashZ file_objectr r r r� s &r� c C s ddd d t d�gt d�S )N�.r� r r- ��MANIFEST_FORMATr r r r �_make_manifest s �� �rN r c C s& d| d|d|r|nd d|� �|dt iS )Nr r r Zchksum_r rL )r r r �chksumr r r � _make_entry" s �rP c C s8 |r|t urtd��|t ur*t| |||�S t| |||�S )Nz4"build_ignore" and "manifest" are mutually exclusive)rO r/ � _build_files_manifest_distlib�_build_files_manifest_walk)r� r r �ignore_patterns�manifest_controlr r r r� , s �r� c C s~ t s td��|d u ri }ztf i |��}W n2 ty\ } ztd|� ���W Y d }~n d }~0 0 t|j�s|td|jjj� ���t|j t �s�td|j jj� ���|j r�|js�td��g }|j r�|� |j� n�|� g d�� t dd� t � D ��}t|�D ].}|d v �rq�q�|tjv r�|�d |� d �� q�|� d d g� |� |j� |� d|� d|� d�ddg� t�d� t�t�d�|�d�� t| dd�} t| �} |D ]|} z| �| � W nf t�y� } ztd| � ���W Y d } ~ n<d } ~ 0 t�y } ztd| � ���W Y d } ~ n d } ~ 0 0 �q�t � } | jdd�D ]T}t j � || �}t j � |��rPt!|d �}nt!|d d t"|t#d �d �}| d! �|� �q$| S )"Nz7Use of "manifest" requires the python "distlib" libraryz Invalid "manifest" provided: z+"manifest.directives" must be a list, got: zE"manifest.omit_default_directives" is expected to be a boolean, got: z�"manifest.omit_default_directives" was set to True, but no directives were defined in "manifest.directives". This would produce an empty collection artifact.)zinclude meta/*.ymlz(include *.txt *.md *.rst COPYING LICENSEzrecursive-include tests **zArecursive-include docs **.rst **.yml **.yaml **.json **.j2 **.txtz4recursive-include roles **.yml **.yaml **.json **.j2z2recursive-include playbooks **.yml **.yaml **.jsonz+recursive-include changelogs **.yml **.yamlz!recursive-include plugins */**.pyc s s" | ]\}}|j �d �d V qdS )rK ���N)�packager� )r{ �d�lr r r r h r� z0_build_files_manifest_distlib.<locals>.<genexpr>)�modulesZ module_utilszrecursive-include plugins/z **.yml **.yamlz7recursive-include plugins/modules **.ps1 **.yml **.yamlz;recursive-include plugins/module_utils **.ps1 **.psm1 **.csz8exclude galaxy.yml galaxy.yaml MANIFEST.json FILES.json �-z -*.tar.gzz!recursive-exclude tests/output **z global-exclude /.* /__pycache__zManifest Directives:rx z r� r� z Invalid manifest directive: z-Unknown error processing manifest directive: T)Zwantdirsr� r� r �Z hash_func�r rO r. )$� HAS_DISTLIBr/ rV � TypeErrorrI rW ra r � isinstancerX rf r r� rD �sortedr# ZDOCUMENTABLE_PLUGINSr� r� r� r �indentry rH r Zprocess_directiver rF rN r� r� �relpathr� rP rM r )r� r r rT Zcontrol�exrW ZpluginsZpluginr� �mZ directiver r� Zabs_path�rel_pathZmanifest_entryr r r rQ < s� $ �� � �� � "( �rQ c sf ddddddddt d �||��g ��d d � |D �7 �tg d ���t� �� ���fd d�� � | | � �S )Nr� r� s galaxy.ymls galaxy.yaml� .gits *.pycs *.retrys tests/outputz{0}-{1}-*.tar.gzc S s g | ] }t |��qS r )rF �r{ �pr r r r~ � r� z._build_files_manifest_walk.<locals>.<listcomp>)s CVSs .bzrs .hgrf s .svns __pycache__s .toxc s^ t �| �D �]L� t j�| � �}| |kr*dn| t|�d d � }t j�|� ��t�dd�}t j�|��r t� fdd��D ��s�t�fdd��D ��r�t� dt|� � q t j� |�r�t j� |�}t ||�s�t� d t|� � q �d �t|d �� t j� |��sX�||� q t�fd d��D ���r6t� dt|� � q �d �t|d dt|td�d�� q d S )Nr� rP r� r� c 3 s | ]}� |kV qd S rY r )r{ rI )�b_itemr r r � r� z<_build_files_manifest_walk.<locals>._walk.<locals>.<genexpr>c 3 s | ]}t � � |�V qd S rY ��fnmatch�r{ Z b_pattern�� b_rel_pathr r r � r� z"Skipping '%s' for collection buildzLSkipping '%s' as it is a symbolic link to a directory outside the collectionr. r� c 3 s | ]}t � � |�V qd S rY rj rl rm r r r � r� r� r r[ r\ )r� �listdirr� ry r� rH r� �anyr� r� �islink�realpath�_is_child_pathr� r� rP rM r )rI Zb_top_level_dirZ b_abs_pathZb_rel_base_dirre � b_link_target��_walkZ b_ignore_dirsZb_ignore_patternsr� )ri rn r rv � s@ � � ��z)_build_files_manifest_walk.<locals>._walk)rF r r rN )r� r r rS r ru r rR � s � ) rR c K sB | ||||||| d |pd || | | | d�dddd t d�t d�}|S )Nr& )r r r! r" r# r$ r% r& r' r( r) r* r+ r, � FILES.jsonr� r r r rL )r r r! r" r# r$ r% r' r( r) r* r+ r, r7 r� r r r r� � s0 ���r� c C s8 t tj|dd�dd�}t|td�|d d< t tj|dd�dd�}t� ���}tj�|tj� |��}t j |dd ���D}t |fd |ffD ]F\} } t | �} t �| �} t| �| _tt�� �| _d | _|j| | d � q�|d D ]�} | d dkr�q�t| d dd�}tj�| t |dd��}dd� }tj�|��r�tj�|�}t|| ��r�tjj|tj�|�d�}t �|�} t j | _ t|dd�| _ || �} |j| d� q�|j ttj�|��|d|d� q�W d � n1 �s�0 Y t �!t|�t|�� d|d d |d d f }t"|�}t#�#d||f � |W d � S 1 �s*0 Y dS )z:Build a tar.gz collection artifact from the manifest data.T�ra r� r� r[ r r zw:gzr� rw � )�tarinfoZfileobjr. r rK c S sJ | j tjkr.| jtj@ }|s$| �� r(dnd| _d | _| _d | _ | _ | S )N�� ry r � ) r^ �tarfile�SYMTYPEr� �stat�S_IXUSRr� �uid�gid�uname�gname)rz �existing_is_execr r r � reset_stat! s z)_build_collection_tar.<locals>.reset_stat)rE )rz F)Zarcname� recursive�filterN�%s.%sr r z Created collection for %s at %s)$rF �json�dumpsrN r r& r� r� ry r� r} r� r� r ZTarInfor� �sizer� r. �mtimer� ZaddfilerG rq rr rs rb �dirnamer~ r^ �linknamer� r� r� rH r� )r� Z b_tar_pathr� r� �files_manifest_json�collection_manifest_jsonr% Zb_tar_filepathZtar_filer �b�b_ioZtar_info� file_inforS r� r� rt rn rp Ztar_pathr r r r� sT �( �r� c C s� t j|dd� ttj|dd�dd�}t|td�|d d < ttj|dd�dd�}t|fd |ffD ]�\}}t j� |t|dd��}t |d ��D} t |�� } t � | | � W d � n1 s�0 Y W d � n1 s�0 Y t �|d � q^g } t|d dd� d�D ]�} | d dk�r�q t j� | t| d dd��} t j� |t| d dd��}t �| �jtj@ }|�rfdnd }t j�| ��r�d}| �| � t �||� n t �| |� t �||� �q t|�}|S )z|Build a collection directory from the manifest data. This should follow the same pattern as _build_collection_tar. r{ r� Trx r� r� r[ r r rw r� Nry r. c S s | d S )Nr r )�xr r r �<lambda>[ r� z'_build_collection_dir.<locals>.<lambda>)�keyr rK )r� �makedirsrF r� r� rN r r� r� ry r� r r� � copyfileobj�chmodr` r �st_moder� r� r� �mkdir�copyfilerH )r� r� r� r� r� r� r r� rI Zfile_objr� Zbase_directoriesr� Zsrc_fileZ dest_filer� r� r� r r r �_build_collection_dirG s2 H r� c c s� t | dd�}t�|�D ]�}tj�||�}tj�|�r6qt�|�D ]�}tj�||�}tj�|�s`q@zt�||�}W n2 t y� } zt t |�|� W Y d}~n d}~0 0 t � djt|�t|j�d�� |V q@qdS )z�Locate all collections under a given path. :param path: Collection dirs layout search path. :param artifacts_manager: Artifacts manager. r� r� Nz1Found installed collection {coll!s} at '{path!s}'r� )rF r� ro r� ry rH r� rA Zfrom_dir_path_as_unknown� ValueErrorrE r/ r� r� r rH r� ) r� r� rI Z b_namespace�b_namespace_pathZ b_collectionr� r Zval_errr r r r q s& $��r c C s� | j r |jn|j| �}tj�|| j| j�}t|dd�}t � dj t | �|d�� tj� |�rdt �|� | jrzt| |||� n<t|||j| j|j|j|j� | jr�t| jt�r�t| ||� t � dj t | �d�� dS )z�Install a collection under a given path. :param collection: Collection to be installed. :param path: Collection dirs layout path. :param artifacts_manager: Artifacts manager. r� r� z#Installing '{coll!s}' to '{path!s}'r� z#{coll!s} was installed successfullyr� N) r� r� r� r� r� ry r r rF r� r rH r� r� r$ r� � install_src�install_artifactZ_b_working_directoryr� r� r� r� Zis_online_index_pointerr_ r� r0 �write_source_metadata)r r� r� Zb_artifact_pathr r� r r r r � sF � �� � ���r c C s� |� | �}tt|�dd�}| �|�}tj�|�d }tj�|�rJt� |� zTtj |dd� t |dd��}|� |� W d � n1 s�0 Y t� |d� W n* ty� tj�|�r�t� |� � Y n0 d S )Nr� r� r r{ r� zw+bry )Z get_galaxy_artifact_source_inforF rJ Zconstruct_galaxy_info_pathr� r� r� r� r� r$ r� r� r� r� rF r� )r r� r� Z source_dataZb_yaml_source_dataZ b_info_destZ b_info_dir�fdr r r r� � s ( r� c C sd d}t | dd��tjj�}d|d |d f }t|| ||||�sPtd|� d���t�d |� �� d S ) NFr� r� r� ������zNot installing z- because GnuPG signature verification failed.z+GnuPG signature verification succeeded for ) rH r� r� r� �sepr� r/ r� r� )r� r� r� �required_signature_countr� Z failed_verifyZcoll_path_partsrp r r r �verify_artifact_manifest� s r� c C s~ �z4t j| dd���}dd� |�� D �|_t|t||� |durftj�t |dd�t�}t |||||� |� d�} t || ��*\} } t �t | �� dd��} W d � n1 s�0 Y t|d||� | d D ]H} | d }|d kr�q�| d d k�rt||||| d d� q�t|||� q�W d � n1 �s*0 Y W nB t�yx t�|� tj�|�}t�|��srt�|� � Y n0 dS )a. Install a collection from tarball under a given path. :param b_coll_targz_path: Collection tarball to be installed. :param b_collection_path: Collection dirs layout path. :param b_temp_path: Temporary dir path. :param signatures: frozenset of signatures to verify the MANIFEST.json :param keyring: The keyring used during GPG verification :param required_signature_count: The number of signatures that must successfully verify the collection :param ignore_signature_errors: GPG errors to ignore during signature verification �rr� c S s i | ]}|j �tjj�|�qS r )r � removesuffixr� r� r� )r{ rd r r r � <dictcomp>� s z$install_artifact.<locals>.<dictcomp>Nr� r� rw r. r rK r r� r )r� )r} r� Z getmembers�_ansible_normalized_cache�_extract_tar_filer� r� r� ry rH r� � getmemberr5 r� �loads�read�_extract_tar_dirrF r� r$ r� ro �rmdir)Zb_coll_targz_pathr� r% r� r� r� r� �collection_tarr� Zfiles_member_objr Z files_objr. r� � file_namer� r r r r� � s8 � 4 �2 r� c C sv |� | �}d|vr"g |d< t|d< tf i |��}t||d |d |d |d �}t||||�}t�dj| |d�� dS )a� Install the collection from source control into given dir. Generates the Ansible collection artifact data from a galaxy.yml and installs the artifact to a directory. This should follow the same pattern as build_collection, but instead of creating an artifact, install it. :param collection: Collection to be installed. :param b_collection_path: Collection dirs layout path. :param b_collection_output_path: The installation directory for the \ collection artifact. :param artifacts_manager: Artifacts manager. :raises AnsibleError: If no collection metadata found. r� r� r r z+Created collection for {coll!s} at {path!s}r� N)Zget_direct_collection_metarO r� r� r� r� r )r r� Zb_collection_output_pathr� r� r� r� Zcollection_output_pathr r r r� s( ����r� c C s t |dd��tjj�}z| j| }W n tyB td| ��Y n0 tj�|t |dd��}tj� |�}ztj |dd� W n2 t y� } z|j t jkr�� W Y d}~n d}~0 0 |jtjkr�t |jdd�}t|||d�s�tdt |�|f ��t�||� ntj�|��st�|d� dS ) z- Extracts a directory from a collection tar. r� r� z&Unable to extract '%s' from collectionr{ r� N�� link_name�]Cannot extract symlink '%s' in collection: path points to location outside of collection '%s')rG r� r� r� r� r� �KeyErrorr/ ry rF r� r� �OSError�errnoZEEXISTr^ r} r~ r� rs �symlinkr� r� )�tarr� �b_dest� tar_memberZ b_dir_path� b_parent_pathr � b_link_pathr r r r� C s* �r� c C s� t | |����\}}|jtjkr(t|�}n8tj|dd��}t||�}W d � n1 sV0 Y |r�||kr�tdt|dd�t| j �f ��t j � t j � |t|dd���} t j �| �} t| |�s�tdt|dd� ��t j �| �s�t j| dd � |jtjk�r:t|jdd�} t| || d ��s,td t|�| f ��t �| | � nTt�t|j dd�| � | �t|dd��}d } t�|j�tj@ �r�| d O } t � | | � W d � n1 �s�0 Y dS )z( Extracts a file from a collection tar. F)r� �deleteNz4Checksum mismatch for '%s' inside collection at '%s'r� r� zSCannot extract tar entry '%s' as it will be placed outside the collection directoryr{ r� r� r� ry �I ) �_get_tar_file_memberr^ r} r~ r1 r" ZNamedTemporaryFiler/ rG r r� r� �abspathry rF r� rs r� r� r� r� r� �mover� r �S_IMODEr� r� r� ) r� rS r� r% r� r� �tar_objrJ Z tmpfile_objZb_dest_filepathZ b_parent_dirr� Znew_moder r r r� b s< ( � � �r� c C sN t |dd�}z| �|�}W n( tyB tdt | j�|f ��Y n0 t| |�S )Nr� r� z?Collection tar at '%s' does not contain the expected file '%s'.)rG r� r� r/ r r5 )r� rS Z n_filename�memberr r r r� � s �r� c C s� d}t j| dd��h}t||��>\}}d}|�|�}|rN|t|�7 }|�|�}q2W d � n1 sb0 Y W d � n1 s�0 Y t�|�S )Nr| r� r� i )r} r� r� r� rH r� r� )rI rS Z file_contentsr� r r� �bufsize�datar r r r� � s Hr� c C sv t j| dd��T}t||��*\}}t|�W d � W d � S 1 sJ0 Y W d � n1 sh0 Y d S )Nr� r� )r} r� r� r1 )rI rS r� r r� r r r r� � s r� c C sL t j�| t|dd��}t|d��}t|�W d � S 1 s>0 Y d S )Nr� r� rG )r� r� ry rF r� r1 )rI rS �filepath�fpr r r r� � s r� c C sp t | dd�}|rFtj�|�sFtj�t |dd��}tj�tj�||��}t |dd�}||kpn|�|t tjj� �S )z> Checks that path is a path within the parent_path specified. r� r� ) rF r� r� �isabsr� r� ry � startswithr� )r� � parent_pathr� rI Z b_link_dirr� r r r rs � s rs c st t s td��tstd��d} z�z td�} W n ty< Y n 0 t� fdd�| jpRg D �d�} W | du r�tt krzt k s�n tdt j � dtj � ���n$| j � tj �s�td | j � | j � ���n`| du r�tt kr�t k s�n tdt j � dtj � ���n&| j � tj ��std | j � | j � ���0 t||| || ||||d � } z| j| d d �jW S t�y� } z>d d� | jD �} ttd| ��}ttd�|��| ��W Y d} ~ n�d} ~ 0 t�y< } z�dd� | j�� D �}dj| j|�s�dndjd�|�d�d�g}| j� � D ]} |� dj| d�� �q�ttd�|��| ��W Y d} ~ n<d} ~ 0 t �yn } ztt |��|�W Y d}~n d}~0 0 dS )z#Return the resolved dependency map.zHFailed to import resolvelib, check that a supported version is installedzGFailed to import packaging, check that a supported version is installedNz ansible-corec 3 s$ | ] }t |� � jd kr� V qdS )Z resolvelibN)r r )r{ r� �Zrrr r r � r� z)_resolve_depenency_map.<locals>.<genexpr>z#ansible-galaxy requires resolvelib<z,>=zansible-galaxy requires ) r� r� Zuser_requirementsr� Z with_depsZwith_pre_releasesr� r� r� i�� )Z max_roundsc s s6 | ].}d j |j|jdu r dn dj |jd�d�V qdS )z+* {req.fqcn!s}:{req.ver!s} ({dep_origin!s})N�direct request�dependency of {parent!s}��parent�r Z dep_origin)r Z requirementr� )r{ Zreq_infr r r r � s ����)z_Failed to resolve the requested dependencies map. Could not satisfy the following requirements:rx c S s g | ]}|d urt |��qS rY )rd rg r r r r~ s �z*_resolve_depenency_map.<locals>.<listcomp>z�Failed to resolve the requested dependencies map. Got the candidate {req.fqcn!s}:{req.ver!s} ({dep_origin!s}) which didn't satisfy all of the following requirements:r� r� z, r� r� z* {req.fqcn!s}:{req.ver!s})r ) �HAS_RESOLVELIBr/ � HAS_PACKAGINGr rF r� Zrequiresr? r> r@ ZvstringZ specifier�containsr r; �resolve�mappingr< Zcausesrc r rE ry r= Z criterionZ iter_parentr � candidateZiter_requirementr� r� rG )Zrequested_requirementsr� r� r� r� r� r� r� r� r ZdistZcollection_dep_resolverZdep_excZconflict_causesZerror_msg_lines�parents�excr r� r r� � s� ���� � � � ������ � �r� )N)r N)N)N)��__doc__Z __future__r r r r^ Z __metaclass__r� rk r� r� r� r1 r� r� r r@ r} r" r r, r. �typingr3 r� r � contextlibr Z dataclassesr r rZ Zhashlibr �ior �importlib.metadatar � itertoolsr Zpackaging.requirementsr r � ImportErrorr� Zdistlib.manifestr Zdistlibr r] � TYPE_CHECKINGZ3ansible.galaxy.collection.concrete_artifact_managerr �LiteralZManifestKeysTypeZFileMetaKeysTypeZCollectionInfoKeysType�Dict�Unionr� rd �ListZManifestValueTypeZCollectionManifestTypeZFileManifestEntryTypeZFilesManifestTypeZansible.constantsZ constantsr# Zansible.errorsr/ Zansible.galaxy.apir0 r1 r2 r3 r4 r5 Z*ansible.galaxy.collection.galaxy_api_proxyr6 Z ansible.galaxy.collection.gpgr7 r8 r9 r: Z$ansible.galaxy.dependency_resolutionr; Z+ansible.galaxy.dependency_resolution.errorsr< r= Z.ansible.galaxy.dependency_resolution.providersr> r? r@ r� Z0ansible.galaxy.dependency_resolution.dataclassesrA rB Z/ansible.galaxy.dependency_resolution.versioningrC Zansible.plugins.loaderrD Zansible.module_utils.sixrE Zansible.module_utils._textrF rG rH Z'ansible.module_utils.common.collectionsrI Z ansible.module_utils.common.yamlrJ Z ansible.utils.collection_loaderrK Zansible.utils.displayrL Zansible.utils.hashingrM rN Zansible.utils.sentinelrO r� rM r� rQ r� rV rF rh r� r� r� r� r� r� r� r r r r! r& r� r� rN rP r� rQ rR r� r� r� r r r� r� r� r� r� r� r� r� r� r� rs r� r r r r �<module> s ���� .$ ) ,2 /j% C bC F* 0 8) -