← Back
Editing: launchpad.cpython-38.pyc
U ��8d�N � @ s� d Z ddlZddlZddlZddlZddlZddlZddlZddl Z ddl Z ddl m Z ddl mZ ejjdkr�ddlmZmZmZ ddlmZ ddlmZmZ eeeeeef d Zn4dd lmZmZmZmZ dd l mZ ddl mZ d Zz dd l m Z ddl m!Z! e! W n e"k �r$ dZ!Y nX ddl#Z$ddl$Z$ej%�&d�Z'dd� Z(dd� Z)G dd� de$j*j+�Z+da,G dd� de�Z-G dd� de�Z.d!dd �Z/e0d k�r�ddl1Z1ddl2Z2dd l3m4Z4 da*da5da6da7G d d � d e1j8�Z9e1�:� dS )"z,Crash database implementation for Launchpad.� N)�FailedToDecompressContent)�BytesIO� )� HTTPSHandler�Request� build_opener)�HTTPSConnection)� urlencode�urlopenT)r r r r )r F)� HTTPError)� Launchpadz%~/.cache/apport/launchpad.credentialsc c sd | D ]Z}z|j �� }W n& ttfk r< t�d� Y qY nX |j}|�d�sX|�d�r|V qd S )Nz"Broken attachment on bug, ignoring�.txt�.gz)�data�openr r �apport�error�filename�endswith)� attachments� attachment�f�name� r �?/usr/lib/python3/dist-packages/apport/crashdb_impl/launchpad.py�filter_filename, s r c C s t dd� | D ��S )Nc s s" | ]}t |j�d ��� �V qdS )�/N)�int� self_link�split�pop)�.0�ir r r � <genexpr>: s zid_set.<locals>.<genexpr>)�set)�tasksr r r �id_set8 s r& c @ s e Zd ZdZdd� Zedd� �Zdd� Zedd � �ZdAd d �Z d d� Z dd� Z dd� Z dd� Z dBdd�ZdCdd�Zdd � Zd d � Zd d � Zd!d"� Zd#d$� Zd%d&� Zd'd(� Zd)d*� Zd+d,� Zd-d.� Zd/d0� Zd1d2� Zd3d4� Z dDd5d6�Z d7d8� Z d9d:� Z d;d<� Z d=d>� Z!e"d?d@� �Z#d S )E� CrashDatabasez5Launchpad implementation of crash database interface.c C s t �d�rt �d�|d< |sH|�d�}|rDtd |�dd�d }nt}tjj�| ||� |�d�| _ | j r~d|ks�t d ��nd|ks�t d ��d |kr�d |d | _ nd tj � � | _ || _|| _| js�t �d | _d | _t �d|�d��| _| j�stjdd�| _tjtj| jdd� d S )a Initialize Launchpad crash database. You need to specify a launchpadlib-style credentials file to access launchpad. If you supply None, it will use default_credentials_path (~/.cache/apport/launchpad.credentials). Recognized options are: - distro: Name of the distribution in Launchpad - project: Name of the project in Launchpad (Note that exactly one of "distro" or "project" must be given.) - launchpad_instance: If set, this uses the given launchpad instance instead of production (optional). This can be overriden or set by $APPORT_LAUNCHPAD_INSTANCE environment. - cache_dir: Path to a permanent cache directory; by default it uses a temporary one. (optional). This can be overridden or set by $APPORT_LAUNCHPAD_CACHE environment. - escalation_subscription: This subscribes the given person or team to a bug once it gets the 10th duplicate. - escalation_tag: This adds the given tag to a bug once it gets more than 10 duplicates. - initial_subscriber: The Launchpad user which gets subscribed to newly filed bugs (default: "apport"). It should be a bot user which the crash-digger instance runs as, as this will get to see all bug details immediately. - triaging_team: The Launchpad user/team which gets subscribed after updating a crash report bug by the retracer (default: "ubuntu-crashes-universe") - architecture: If set, this sets and watches out for needs-*-retrace tags of this architecture. This is useful when being used with apport-retrace and crash-digger to process crash reports of foreign architectures. Defaults to system architecture. �APPORT_LAUNCHPAD_INSTANCE�launchpad_instance�.z://� ����distro�projectz/Must not set both "project" and "distro" optionz0Need to have either "project" or "distro" option� architecturezneed-%s-retraceNZAPPORT_LAUNCHPAD_CACHEZ cache_dirzlaunchpadlib.cache.)�prefixT)� ignore_errors)�os�getenv�get�default_credentials_pathr r �crashdbr' �__init__r- �AssertionError�arch_tag� packaging�get_system_architecture�options�auth�_CrashDatabase__launchpad�_CrashDatabase__lp_distro�_CrashDatabase__lpcache�tempfile�mkdtemp�atexit�register�shutil�rmtree)�selfr= r< Z lp_instancer r r r7 @ s4 ! � zCrashDatabase.__init__c C s | j r | j S tdkr<tr&tj�d� n tj�d� t�d� | j�d�rV| j�d�}nd}t j � | j �}|r�t j � |�s�t �|� z"tjd|| jdg| j d d �| _ W nV tk r� } z8t|d �r�|j}nt|�}t�d || j � t�d � W 5 d}~X Y nX | j S )zReturn Launchpad instance.Nz^ERROR: The python-launchpadlib package is not installed. This functionality is not available. z_ERROR: The python3-launchpadlib package is not installed. This functionality is not available. r+ r) Z productionzapport-collectZ WRITE_PRIVATEz1.0)Zlaunchpadlib_dirZallow_access_levelsZcredentials_file�version�contentzZconnecting to Launchpad failed: %s You can reset the credentials by removing the file "%s"�c )r> r �_python2�sys�stderr�write�exitr< r4 r2 �path�dirnamer= �isdir�makedirsZ login_withr@ � Exception�hasattrrI �strr r )rG r) Zauth_dir�e�msgr r r � launchpad� s8 � zCrashDatabase.launchpadc c sF | j s t�|D ]2}|j�� | j ks:t�d| j |j�� �r|V qd S )Nz ^.+\(%s.*\)$)r- � StopIteration�bug_target_name�lower�re�match)rG r% �tr r r �_get_distro_tasks� s �z CrashDatabase._get_distro_tasksc C sP | j d krJ| jr"| jj| j | _ n(d| jkrB| jj| jd | _ ntd��| j S )Nr. z:distro or project needs to be specified in crashdb options)r? r- rY � distributionsr< �projects� SystemError�rG r r r � lp_distro� s zCrashDatabase.lp_distroNc C s> | � |�st�| �|�}t||| �� d�}|�� |s:t�|S )a� Upload given problem report return a handle for it. This should happen noninteractively. If the implementation supports it, and a function progress_callback is passed, that is called repeatedly with two arguments: the number of bytes already sent, and the total number of bytes to send. This can be used to provide a proper upload progress indication on frontends. )�hostname)Zacceptsr8 �_generate_upload_blob� upload_blob� get_hostname�close)rG �report�progress_callbackZ blob_file�ticketr r r �upload� s zCrashDatabase.uploadc C s, | j �d�}|r$|dkr d}q(d}nd}|S )z/Return the hostname for the Launchpad instance.r) �stagingzstaging.launchpad.netz launchpad.dev� launchpad.net)r< r4 )rG r) rf r r r ri � s zCrashDatabase.get_hostnamec C s� i }|� d|�� �}|r4t|t�s,|�d�}||d< | �� }d|krN|d }n | j� d�}|s�d|kr�d|| j|d |t|�f S d|| j|t|�f S nd|||t|�f S d S ) a/ Return an URL that should be opened after report has been uploaded and upload() returned handle. Should return None if no URL should be opened (anonymous filing without user comments); in that case this function should do whichever interactive steps it wants to perform.�Title�UTF-8z field.titleZ SnapSourcer. � SourcePackagez,https://bugs.%s/%s/+source/%s/+filebug/%s?%sz!https://bugs.%s/%s/+filebug/%s?%sN) r4 �standard_title� isinstance�bytes�encoderi r< r- r )rG rk Zhandle�args�titlerf r. r r r �get_comment_url� s> � � �z CrashDatabase.get_comment_urlc C s dt |� S )z�Return URL for a given report ID. The report is passed in case building the URL needs additional information from it, such as the SourcePackage name. Return None if URL is not available or cannot be determined. z https://bugs.launchpad.net/bugs/)rV )rG rk �idr r r � get_id_url s zCrashDatabase.get_id_urlc C s� t �� }| jj| }t�d|jtj�}|sBt�d|jtjtjB �}|sNt d��|� d�� d�� dd�� dd �}d |kr�d |kr�|� d d�\}}|� d d �d |� d d�d }n|� d d�d }|�t|�� d |k�r z|j�� |d < W n tk �r |j|d < Y nX d|k�r�d|jk�r,d|d< nTd|jk�rBd|d< n>d|jk�rXd|d< n(d|jk�rnd|d< ntdt|j� ��d�|j�|d< d|k�r�|d |d< |j|d< t|j�D �]}tj� |j �\} } z d || < W n t k �r� Y �q�Y nX | d k�rB|� � || < z|| � d�|| < W n t!k �r> Y nX n~| d k�r�zt"j#|d �� � || < W nJ t$k �r� } z*d t| �k�r�� |�%d � |� � || < W 5 d!} ~ X Y nX nt d"|j ���q�|S )#z>Download the problem report from given ID and return a Report.z(ProblemType:.*)$z^--- \r?$[\r\n]*(.*)z8bug description must contain standard apport format datar+ rr s � s � s s Uname:r ZDate� ProblemTypez apport-bug�Bug� apport-crash�Crashzapport-kernelcrashZ KernelCrashzapport-package�Packagez(cannot determine ProblemType from tags: � �Tagsrq Z OriginalTitle� r r )Zfileobjz Not a gzipNzUnknown attachment type: )&r �ReportrY �bugsr] �search� description�S�Mr8 �grouprw �replacer �loadr Z date_created�ctime�AttributeError�tags� ValueErrorrV �joinry r r r2 rP �splitextr rT �read�decode�UnicodeDecodeError�gzipZGzipFile�IOError�seek) rG r{ rk �b�mr� Zpart1Zpart2r �keyZextrW r r r �download sp � zCrashDatabase.downloadFc C s� | j j| }|r&t|�� �t|� }nd}t�� } |j| |d� | �� | �d� t rbt � | �} n t � | �} | � � } t r�| �� p�| �� } | �� s�t�t r�| �� p�| �� } | �� r�t�| �� dks�t�|�s |jdd� } | �d� d|k�r| | �|d ��� 7 } | |_|�� | j j| }| jdd��d d �}|�rp|�rV|jd | |_|�� n|�sb|j}|j ||d � | D ]2} |j |�p�d | � � d| jdd�| � � dd� �qt| � � dS )ah Update the given report ID with all data from report. This creates a text comment with the "short" data (see ProblemReport.write_mime()), and creates attachments for all the bulk/binary data. If change_description is True, and the crash db implementation supports it, the short data will be put into the description instead (like in a new bug). comment will be added to the "short" data. If attachment_comment is given, it will be added to the attachment uploads. If key_filter is a list or set, then only those keys will be added. N)� skip_keysr � text/plain�apport-collectedr� T�r� rr r� z --- �rI Zsubjectr� F��commentr� Z content_typer r Zis_patch) rY r� r$ �keysrA � TemporaryFile� write_mime�flushr� rK �email�message_from_file�message_from_binary_file�walk�next�__next__� is_multipartr8 �get_content_typer� �append�_filter_tag_namesr �lp_save� get_payloadr� r� ry � newMessage� addAttachment� get_filenamerj )rG r{ rk r� �change_descriptionZattachment_comment� key_filter�bugr� �mimerX �msg_iter�part�x�textr r r �updatea sT � zCrashDatabase.updater� c C s� t jj�| |||� | jj| }d|krn|jD ]>}|jj� d�r.| j j |d d�|_|� � | jj| } qnq.|� � �r�|jD ]0}|jdkr~z |�� W q~ tk r� Y q~X q~z>| �|j�}tr�|�� }n|�� }|jdkr�d|_|� � W n tk �r Y nX |�� }|�r�t�d|j�}|�r�|�d�|k�r�|�d �| |�d � |_z |� � W n tk �rt Y nX | jj| }| � ||� d S ) z�Update the given report ID for retracing results. This updates Stacktrace, ThreadStacktrace, StacktraceTop, and StacktraceSource. You can also supply an additional comment. rs � #distribution�r � CoreDump.gz� Undecided�Mediumz,^(.*crashed with SIG.* in )([^( ]+)(\(\).*$)r r+ � N) r r6 r' � update_tracesrY r� � bug_tasks�target�resource_type_linkr re �getSourcePackager� �has_useful_stacktracer ry � removeFromBugr r` rK r� r� � importancerZ Zstacktrace_top_functionr] r^ r� �_subscribe_triaging_team) rG r{ rk r� r� �task�a�fnr� r r r r� � sJ zCrashDatabase.update_tracesc C s4 | j j| }t�d|j�}|r(|�d�S td��dS )zNGet 'DistroRelease: <release>' from the given report ID and return it.z"DistroRelease: ([-a-zA-Z0-9.+/ ]+)r+ z)URL does not contain DistroRelease: fieldN)rY r� r] r� r� r� r� )rG r{ r� r� r r r �get_distro_release� s z CrashDatabase.get_distro_releasec C s` t �d| j �}| jj| }g }|jD ]4}|�|jj�}|s>q&|j dkrJq&|� |� d�� q&|S )z5Return list of affected source packages for given ID.z5/%s/(?:(?P<suite>[^/]+)/)?\+source/(?P<source>[^/]+)$)�Invalid� Won't Fix� Fix Released�source) r] �compiler- rY r� r� r� r� r �statusr� r� )rG r{ Z bug_target_rer� �resultr� r^ r r r �get_affected_packages� s � z#CrashDatabase.get_affected_packagesc C s | j j| }|jj| j jjkS )z3Check whether the user is the reporter of given ID.)rY r� �ownerr �me�rG r{ r� r r r � is_reporter s zCrashDatabase.is_reporterc C sZ | j j| }|jrdS |jj| j jjkr,dS | j jj}|jjD ]}|d |kr> dS q>dS )aA Check whether the user is eligible to update a report. A user should add additional information to an existing ID if (s)he is the reporter or subscribed, the bug is open, not a duplicate, etc. The exact policy and checks should be done according to the particular implementation. FT� person_link) rY r� � duplicate_ofr� r r� r � subscriptions�entries)rG r{ r� r� �subr r r � can_update s zCrashDatabase.can_updatec C s^ z | j j| jdd�}t|�W S tk rX } z t�dt|�� t� d� W 5 d}~X Y nX dS )z}Return an ID set of all crashes which have not been retraced yet and which happened on the current host architecture.� 2011-08-01�r� Z created_since�"connecting to Launchpad failed: %srJ N) re � searchTasksr9 r& rT r r rV rL rO �rG r� rW r r r �get_unretraced s z CrashDatabase.get_unretracedc C s\ z| j jddd�}t|�W S tk rV } z t�dt|�� t�d� W 5 d}~X Y nX dS )a# Return an ID set of all crashes which have not been checked for being a duplicate. This is mainly useful for crashes of scripting languages such as Python, since they do not need to be retraced. It should not return bugs that are covered by get_unretraced().�need-duplicate-checkr� r� r� rJ N) re r� r& rT r r rV rL rO r� r r r �get_dup_unchecked* s z CrashDatabase.get_dup_uncheckedc C s | j jdd�}t|�S )aO Return an ID set of all crashes which are not yet fixed. The list must not contain bugs which were rejected or duplicate. This function should make sure that the returned list is correct. If there are any errors with connecting to the crash database, it should raise an exception (preferably IOError).r� )r� )re r� r& )rG r� r r r � get_unfixed9 s zCrashDatabase.get_unfixedc C s" | j jjd|| j jd�}|d jS )z�Return the version of given source package in the latest release of given distribution. If 'distro' is None, we will look for a launchpad project . T)Z exact_matchZ source_nameZ distro_seriesr )re Z main_archiveZgetPublishedSourcesZcurrent_seriesZsource_package_version)rG �packageZsourcesr r r �_get_source_versionE s �z!CrashDatabase._get_source_versionc s\ z�j j| }W n tk r& Y dS X |jr2dS t|j�}�j�r$d�j�� � tt� fdd�|��}|s�tt�fdd�|��}|r�dS t |�dkr�t � d�j|� dS |r�|� � }z�� |j�� d �W S tk r� Y dS X n<tt� fd d�|��}|�rXtt� fd d�|��}|�sXdS n4ttd d� |��}|�r@dS ttd d� |���rXdS dS )a Return the package version that fixes a given crash. Return None if the crash is not yet fixed, or an empty string if the crash is fixed, but it cannot be determined by which version. Return 'invalid' if the crash report got invalidated, such as closed a duplicate or rejected. This function should make sure that the returned result is correct. If there are any errors with connecting to the crash database, it should raise an exception (preferably IOError). �invalidz(%s)c s | j dko� | j�� kS �Nr� �r� �bug_target_display_namer\ �r� ��distro_identifierr r �<lambda>o s z1CrashDatabase.get_fixed_version.<locals>.<lambda>c s | j dko | j�� � j�� kS r� )r� r[ r\ r- r� rd r r r� s s r� r+ zVThere is more than one task fixed in %s %s, using first one to determine fixed versionr c s | j dko� | j�� kS �N)r� r� ZExpiredr� r� r� r r r� � s c s | j dko� | j�� kS r� r� r� r� r r r� � s c S s | j dkS r� �r� r� r r r r� � � c S s | j dkS )Nr� r� r� r r r r� � r� N)rY r� �KeyErrorr� �listr� r- r\ �filter�lenr �warningr r� r� r � IndexError) rG r{ r� r% Z fixed_tasksZ fixed_distror� Z invalid_tasksZnon_invalid_tasksr )r� rG r �get_fixed_versionS sV �� � �z CrashDatabase.get_fixed_versionc C s | j j| j}|r|jS dS dS )zcReturn master ID for a duplicate bug. If the bug is not a duplicate, return None. N)rY r� r� r{ )rG r{ r� r r r r� � s zCrashDatabase.duplicate_ofc C s� | j j| }|�r�||ks*tdt|� ��| j j| }|jrd|j}|j}|j|krdt�d||� dS |jD ]0}|j dkrjz |� � W qj t k r� Y qjX qj| j j| }|j d| dd� | j j| }|j r�d|_ |js�||_|j}t|j�d k�r|d | jk�r>| jd |k�r>| j�d d �|k�r>|| jd g |_|�� d | jk�r|| j�d d �|k�r|| j j| jd }|j|d� |�� �rLd|k�s�d|k�rL| j||d| ddddddgd� | j j| }|jdd� } z| �d� W n tk �r� Y nX z| �d� W n tk �r Y nX | |_z |�� W n t k �rJ Y nX dg} | jjD ]"} | j dk�rn�qZ| � | j � �qZt |j�} |j}| � |�} | D ]}|| k�r�|� |� �q�||_|�� n|j�r�d|_|j!�r�|�� dS )zlMark a crash id as duplicate of given master ID. If master is None, id gets un-duplicated. z+cannot mark bug %s as a duplicate of itselfzNBug %i was manually marked as a dupe of newer bug %i, not closing as duplicateN)r� zStacktrace.txtzThreadStacktrace.txtz ProcMaps.txtzProcStatus.txtz Registers.txtzDisassembly.txta� Thank you for taking the time to report this crash and helping to make this software better. This particular crash has already been reported and is a duplicate of bug #%i, so is being marked as such. Please look at the other bug report to see if there is any missing information that you can provide, or to see if there is a workaround for the bug. Additionally, any further discussion regarding the bug should occur in the other report. Please continue to report any other bugs you may find.zThis bug is a duplicater� F� �escalation_tagZ escalated_tagz invalid �escalation_subscription��personzapport-request-retrace�apport-failed-retracez)Updated stack trace from duplicate bug %i� Stacktrace�ThreadStacktracer� � Dependencies�ProcMapsZ ProcCmdline)r� zbugpattern-needed)zActive DevelopmentzCurrent Stable ReleaseZ SupportedzPre-release Freeze)"rY r� r8 rV r� r{ r r r ry r� r r� �privater� r � duplicatesr< r4 r� �people� subscriber� r� �remover� re �seriesr� r� r r$ � differenceZ_dirty_attributes)rG rk r{ Z master_idr� �masterr� Z master_tags�pr� Z tags_to_copyr Z dupe_tagsZ missing_tags�tagr r r �close_duplicate� s� � �� 0 � �� z CrashDatabase.close_duplicatec C sD | j j| }|jd| dd� | j j| }|jdg |_|�� dS )zpMark a crash id as reintroducing an earlier crash which is already marked as fixed (having ID 'master').z�This crash has the same stack trace characteristics as bug #%i. However, the latter was already fixed in an earlier package version than the one in this report. This might be a regression or because the problem is in a dependent package.z Possible regression detectedr� �regression-retracerN)rY r� r� r� r� )rG r{ r r� r r r �mark_regression s �� z CrashDatabase.mark_regressionc C s^ | j j| }| j|jkrZ|jdd� }|�| j� ||_z |�� W n tk rX Y nX dS )zMark crash id as retraced.N)rY r� r9 r� r r� r )rG r{ r� r� r r r � mark_retraced# s zCrashDatabase.mark_retracedc C s� | j j| }|r�z&| �|j�}tr,|�� }n|�� }W n tk rT |jd }Y nX d|_|� � |j |dd� |j D ]0}|j dkrxz |� � W qx tk r� Y qxX qxn d|jkr�|jdg |_|� � dS )z%Mark crash id as 'failed to retrace'.r r� z Crash report cannot be processedr� r� r N)rY r� r` r� rK r� r� rZ r� r� r� r ry r� r r� )rG r{ Z invalid_msgr� r� r� r r r �mark_retrace_failed0 s. � z!CrashDatabase.mark_retrace_failedc C s� | j j| }d|krt|jD ]X}|jj�d�r| jj|d d�|_z|�� | j j| }W n t k rl Y nX qtqd|j kr�|j dd� }|� d� ||_ |�� d|kr�|jD ](}d|jjkr�|j dkr�d|_ |�� q�| � ||� dS ) z/Mark crash id as checked for being a duplicate.rs r� r� r� N� Tracebackr� r� )rY r� r� r� r� r re r� r� r r� r r� r� )rG r{ rk r� r� r� r r r �_mark_dup_checkedN s2 � z CrashDatabase._mark_dup_checkedc C s� t jj�| |�}|s|S ||d< zt|d �}W n tk rH | Y S X |�� }|�d�s`dS |D ]4}|�d�r�d|ks�d|kr� dS q�|�� sd q�qd|S ) a� Check if the crash db already knows about the crash signature. Check if the report has a DuplicateSignature, crash_signature(), or StacktraceAddressSignature, and ask the database whether the problem is already known. If so, return an URL where the user can check the status or subscribe (if available), or just return True if the report is known but there is no public URL. In that case the report will not be uploaded (i. e. upload() will not be called). Return None if the report does not have any signature or the crash database does not support checking for duplicates on the client side. The default implementation uses a text file format generated by duplicate_db_publish() at an URL specified by the "dupdb_url" option. Subclasses are free to override this with a custom implementation, such as a real database lookup. Z DuplicateOfz/+texts bug:Ts tags:s apport-failed-retraces apport-request-retraceN) r r6 r' �knownr r� �readline� startswith�strip)rG rk �urlr �liner r r r n s& zCrashDatabase.knownc C st d|kr |d � � d dkr dS d| jj| j�dd�f }|�t| jj�d��d �d d � |jD �krp|j |d � dS ) z-Subscribe the right triaging team to the bug.� DistroReleaser ZUbuntuNz%s~%sZ triaging_teamzubuntu-crashes-universer� �~c S s g | ]}t |��d �d �qS )r r, )rV r )r! r� r r r � <listcomp>� s z:CrashDatabase._subscribe_triaging_team.<locals>.<listcomp>r ) r rY Z _root_urir< r4 r� rV r# r� r )rG r� rk r r r r r� � s ��z&CrashDatabase._subscribe_triaging_teamc C s� i }d|d � � |d< |�d�}|r.|dkr8|�d�}|rP|d d| 7 < d|krv|d d| �|d � 7 < d|kr�|r�d |ks�d |ks�d |kr�d |d < |�d| j�dd��|d< |d d| 7 < n(d|kr�d |d < d|d< |d d7 < d|k�r"d|d k�r"|d d7 < d|k�r8|d |d< dddddd d d d g }t�� }|j||dd dg|d � |�� |�d!� |S )"z�Generate a multipart/MIME temporary file for uploading. You have to close the returned file object after you are done with it. z apport-%sr r� ZPackageArchitecture�all� Architecturer� r& ZVmCore�CoreDumpZLaunchpadPrivateZyesZPrivateZLaunchpadSubscribeZinitial_subscriberr � Subscribersz need-%s-retracer z need-duplicate-checkZDuplicateSignaturer� ZCheckboxSubmissionzHWDB-Submissionr� Z RegressionZ ReproducibleZTestedUpstreamZProcVersionSignature�Uname�NonfreeKernelModules)Z extra_headersr� Zpriority_fieldsr ) r\ r4 r� r< rA r� r� r� r� )rG rk Zhdrr� �orderr� r r r rg � sL � �� z#CrashDatabase._generate_upload_blobc C s` d}|� � jddd�D ]D}|dks6t|�dkrR|dkrRtrD||7 }qZ|t|�7 }q|d7 }q|S ) zAReplace characters from tags which are not palatable to Launchpadr� �ASCII�ignore)�errorss% abcdefghijklmnopqrstuvwxyz0123456789 r s +-.r* )r\ rw r rK �chr)�klassr� �resZchr r r r� � s z CrashDatabase._filter_tag_names)N)FNN)r� )N)$�__name__� __module__� __qualname__�__doc__r7 �propertyrY r` re rn ri rz r| r� r� r� r� r� r� r� r� r� r� r� r r� r r r r r r r� rg � classmethodr� r r r r r' = sJ B ( # U � S 4 F q 72r' c @ s e Zd ZdZdd� ZdS )�HTTPSProgressConnectionzWImplement a HTTPSConnection with an optional callback function for upload progress.c C s� t st�| |� d S d}t|�}d}||k r�t ||� t�� }t�| |||| � � ||7 }t�� }|dkr$|| dk r�|dK }q$|| dkr$|dL }q$d S )Nr i g �?r+ r )�_https_upload_callbackr �sendr �time)rG r ZsentZtotalZ chunksize�t1�t2r r r r> s" z HTTPSProgressConnection.sendN)r6 r7 r8 r9 r> r r r r r< s r< c @ s e Zd Zdd� ZdS )�HTTPSProgressHandlerc C s | � t|�S )N)Zdo_openr< )rG �reqr r r � https_open$ s z HTTPSProgressHandler.https_openN)r6 r7 r8 rD r r r r rB " s rB rp c C s d}d| }|a tjj�� }tjj�d�}|�dd� |�|� tjj � dd�}|�dd� |� | � � � d �� |�|� t� }tjjd kr�tjj|d d �} ntjj|d d �} | �|� t||�� �} | �d d|�� � tt�} | �| �} | � � � d�}|�st �|S )a Upload blob (file-like object) to Launchpad. progress_callback can be set to a function(sent, total) which is regularly called with the number of bytes already sent and total number of bytes to send. It is called every 0.5 to 2 seconds (dynamically adapted to upload bandwidth). Return None on error, or the ticket number on success. By default this uses the production Launchpad hostname. Set hostname to 'launchpad.dev' or 'staging.launchpad.net' to use another instance for testing. Nzhttps://%s/+storeblob�1zContent-Dispositionz form-data; name="FORM_SUBMIT"Z applicationz octet-streamz*form-data; name="field.blob"; filename="x"�asciir F)Z mangle_from_z Content-Typez multipart/form-data; boundary=zX-Launchpad-Blob-Token) r= r� r� Z multipartZ MIMEMultipartr� ZMIMETextZ add_headerZattach�baseZMIMEBaseZ set_payloadr� r� r rL � version_info�major� generator� GeneratorZBytesGeneratorZflattenr �getvalueZ get_boundaryr rB r �infor4 r8 ) Zblobrl rf rm r$ r ZsubmitZ form_blobZ data_flat�genrC Zopenerr� r r r rh ( s. rh �__main__)�patchc @ sT e Zd ZdZdZdd� Zdd� Zedd� �ZdNd d �Z d d � Z dOd d�Z dd� Z dd� Z dd� Zdd� Zdd� Zdd� Zdd � Zd d � Zd d � Zd!d"� Zd#d$� Zd%d&� Zd'd(� Zd)d*� Zd+d,� Zd-d.� Ze � e d/�d0d1� �Z e d2d3� �Z!d4d5� Z"dPd7d8�Z#d9d:� Z$d;d<� Z%d=d>� Z&d?d@� Z'dAdB� Z(dCdD� Z)dEdF� Z*dGdH� Z+dIdJ� Z,e dQdLdM��Z-d6S )R�_T� coreutilsc C sH t s | �� a t | _ t�� | _| j�� | j�� d| jd< | �d� d S )NrR rs �langpack-o-matic)r6 � _get_instancer r� � ref_report� add_os_info� add_user_info�_create_projectrd r r r �setUpr s z_T.setUpc C s< | j jj| }|s8| j jjj|d |||d |d d� dS )z8Create a project using launchpadlib to be used by tests.r� �summaryry )r� Z display_namer rZ ry N)r6 rY rb Z new_project)rG r r. r r r rX � s �z_T._create_projectc C s | j �� S )z1Get the Launchpad hostname for the given crashdb.)r6 ri rd r r r rf � s z _T.hostnameFc C s� |st dk rt S | �� }|�| j� |�� |�� |�� | �|�� d� d|d< d|d< | � | j |�}| � |� | � ||�}| � |dk� t j�d| j|f � |s�|a |S ) z�Generate SEGV crash report. This is only done once, subsequent calls will return the already existing ID, unless force_fresh is True. Return the ID. N�!crash crashed with SIGSEGV in f()u "]¶" ZShortGibberish� a b c d e ÿÿÿ Z LongGibberishr z*(Created SEGV report: https://%s/bugs/%i) )� _segv_report�_generate_sigsegv_report�add_package_info� test_packagerV � add_gdb_inforW � assertEqualrt �_get_bug_targetr6 � assertTrue� _file_bugrL rM rN rf )rG � force_fresh�r� bug_targetr{ r r r �get_segv_report� s$ z_T.get_segv_reportc C s� t dk r t S t�d�}d|d< d|d< d|d< |�| j� |�� |�� | �|�� d � | � | j |�}| � |� | � ||�}| � |d k� t j�d | j|f � |a |S ) zFGenerate Python crash report. Return the ID. Nr� �/bin/foo�ExecutablePath��Traceback (most recent call last): File "/bin/foo", line 67, in fuzz print(weird) NameError: global name 'weird' is not definedr zboogus pybogusr� �Hfoo crashed with NameError in fuzz(): global name 'weird' is not definedr z,(Created Python report: https://%s/bugs/%i) )�_python_reportr r� r_ r` rV rW rb rt rc r6 rd re rL rM rN rf )rG rg rh r{ r r r �get_python_report� s& � z_T.get_python_reportc C sV |st dk rt S d}| jjjjd|| jjd�}tj�d| j |j f � |sP|j a |j S )a� File a bug report with an uncommon description. This is only done once, subsequent calls will return the already existing ID, unless force_fresh is True. Example taken from real LP bug 269539. It contains only ProblemType/Architecture/DistroRelease in the description, and has free-form description text after the Apport data. Return the ID. Nzbproblem ProblemType: Package Architecture: amd64 DistroRelease: Ubuntu 8.10 more text and more zmixed description bug)ry r� r� z3(Created uncommon description: https://%s/bugs/%i) ) � _uncommon_description_reportr6 rY r� � createBugre rL rM rN rf r{ )rG rf Zdescr� r r r � get_uncommon_description_report� s �z"_T.get_uncommon_description_reportc C s� | j �| �� �}| �|d d� | �|d d� | �|d | jd � | �|d | jd � | �|d | jd � | �|�d�| j�d�� | �|�d �| j�d �� t|d �� �}| �|t| j jd t j � � g�� | �|d d � | � |d � d�� | �|d | j� | � |d �| jd �� | �d|d � | �d|d � | �d|d � | �t|d �d� | �d|� | �d|� | �d|� | j �| �� �}t|d �� �}| �|td d d d t j � � g�� d S ) z download()r r� rq r[ r& r* r- r. Z UserGroupsr� r� �Signal�11rk z/crashrs r� r� zf (x=42)r � StacktraceTopr r+ i� r � Disassembly� RegistersZboogusZpybogusr� N)r6 r� ri rb rU r4 r$ r r9 r r: r; rd r �test_srcpackager" r` �assertInZ assertGreaterr ro )rG rg r� r r r �test_1_download� s> �� �z_T.test_1_downloadc C s� | j �| �� �}| �d|� | �d|� | �d|� | �d|� | �d|� | �d|� | �|d d� d |d < d |d< d |d< d |d< | j �| �� |d� | j �| �� �}| �d|� | �d|� | �d|� | �d|� | �d|� | �d|� | �d|� | �|d d� | j jj| �� j }| �d|� | �d|� d|d < d |d< d |d< | j �| �� |d� | j �| �� �}| �d|� | �d|� | �d|� | �d|� | �d|� | �d|� | �d|� | �|d d� | j jj| �� }d|_ z |� � W n t k �r Y nX d|d < | j �| �� |d� | j �| �� �}| �|d d� | j jj| �� }d|_ z |� � W n t k �r� Y nX d|d < | j �| �� |d� | j �| �� �}| �|d d� d|d< d|d< d |d < | j �| �� |d � d S ) zupdate_traces()r+ r rv rw r r rq r[ z?? ()ru � long trace�thread even longer traceZbogusZFooBarzI can has a better retrace?r� r� �=read () from /lib/libc.6.so foo (i=1) from /usr/lib/libfoo.so� good retrace!z$crash crashed with SIGSEGV in read()z)crash crashed with SIGSEGV in f() on exitz!good retrace with title amendmentz,crash crashed with SIGSEGV in read() on exitzcrash is crashyz good retrace with custom titler� u "]¶" r\ ZStacktraceSourceZtestsN) r6 r� ri ry rb r� � assertNotInrY r� r� ry r� r )rG rg r� r� r r r �test_2_update_traces s| z_T.test_2_update_tracesc C s� d}t �d�}||d< | j�|d�}| �|�d�� |�d�|d< | j�|d�}| �|�d�� d|d< | j�|d�}| �|�d �� d S ) z&get_comment_url() for non-ASCII titless 1ä♥2r� rq �* z1/ubuntu/+filebug/42?field.title=1%C3%A4%E2%99%A52rr rR rs zC/ubuntu/+source/coreutils/+filebug/42?field.title=1%C3%A4%E2%99%A52N)r r� r6 rz rd r r� )rG ry rg r$ r r r �test_get_comment_urlm s z_T.test_get_comment_urlc C s | j jjdd�}| j jjjd|dd�}|j}| �|dk� tj � d| j |f � t � d�}d �d �|d < d |d < d|d< d|d< d|d< | j j||ddd� | j �|�}| �|d d �d �� | �|d d� | �|d d� | �|d d� | �| j jj| jdg� dS )z"update() with changing description�bashr� z test description for test bug.�testbug�r� r� ry r �(https://%s/bugs/%i) r� s bogus→rr �OneLiner� f() g() h(1)ru �lineone linetwo�ShortGoo�one two three four five six�DpkgTerminalLogs �VarLogDistupgradeBinGoo�NotMeT�r� r� N)r6 re r� rY r� rq r{ rd rL rM rN rf r r� r� r� r� rb r� �rG rh r� r{ rg r r r �test_update_description� s0 � �z_T.test_update_descriptionc C s | j jjdd�}| j jjjd|dd�}|j}| �|dk� tj � d| j |f � t � d�}d |d < d |d < d |d< d|d< d|d< | j j||ddd� | j �|�}| �d |� | �d|� | �|d d� | �|d d� | �|d d� | �| j jj| jdg� dS )z update() with appending commentr� r� z Pr0blem --- ProblemType: Bugr� r� r r� r� � bogus→r� r� ru r� r� r� r� � r� ZmeowFr� r r� N�r6 re r� rY r� rq r{ rd rL rM rN rf r r� r� r� r rb r� r� r r r �test_update_comment� s2 � �z_T.test_update_commentc C s | j jjdd�}| j jjjd|dd�}|j}| �|dk� tj � d| j |f � t � d�}d |d < d |d < d |d< d|d< d|d< | j j||dddddgd� | j �|�}| �d |� | �|d d � | �|d d� | �|d d� | �d|� | �| j jj| jg � dS )zupdate() with a key filterr� r� z test description for test bugr� r� r r� r� r� r� r� ru r� r� r� r� r� r� r� Tr )r� r� Nr� r� r r r �test_update_filter� s2 � � z_T.test_update_filterc C s" | � | j�| �� �| jd � dS )zget_distro_release()r& N)rb r6 r� ri rU rd r r r �test_get_distro_release� s �z_T.test_get_distro_releasec C s$ | � | j�| �� �| jd g� dS )zget_affected_packages()rs N)rb r6 r� ri rU rd r r r �test_get_affected_packages� s �z _T.test_get_affected_packagesc C s, | � | j�| �� �� | �| j�d�� dS )z is_reporter()r+ N)rd r6 r� ri � assertFalserd r r r �test_is_reporter� s z_T.test_is_reporterc C s, | � | j�| �� �� | �| j�d�� dS )z can_update()r+ N)rd r6 r� ri r� rd r r r �test_can_update� s z_T.test_can_updatec C s� | � | j�| �� �d� | � | j�| �� �d� | �� }| �� }| jdd�}| j�|�}| j�|||� | � | j�|�|� | j�|||� | � | j�|�|� | � | j�|�d� | j�||d� | � | j�|�d� | � | j�|�d� | j�| �� �}| �d|� | �d|� | �d|� | �d|� | �d |� | �d |� | j�t � � ||� | j�|||� | � | j�|�|� | j�t � � |d� | j�t � � |d� | j�||d� | j�t � � |d� | � | j�|�d� | j� ||� | � |� dS ) zduplicate handlingNT�rf r� r+ rv r Z ProcStatusrw r ) rb r6 r� ri r rr r� r r r r� r �_verify_marked_regression)rG Zsegv_idZ known_test_idZknown_test_id2rg r r r �test_duplicates� sF ��z_T.test_duplicatesc C s� | j �� }| �| �� |� | �| �� |� | j �| �� � | j �� }| �| �� |� | �||�t | �� g��� | �| j � | �� �d� | � | �� � | j �| �� � | j � | �� � | j �� }| �| �� |� | �||�t | �� g��� | �| j � | �� �d� | � | �� � | j �| �� � | j � | �� d� | j �� }| �| �� |� | �||�t | �� g��� | �| j � | �� �d� dS )z-processing status markings for signal crashesNzI don't like your� ) r6 r� ry ri r ro r rb �unionr$ r �_mark_needs_retracer )rG �unretraced_before�unretraced_afterr r r �test_marking_segv5 s< � � ��z_T.test_marking_segvc C s� | j jjjd| j j| j jdd�}tj�d�p.d}t tj�d�d|d��}|jjjd |j|jd d�}|� � }| � |j |� | � |j |� |�|j � |� � }| � |j |� | �||�t|j g��� | �| j �|j �d � d S ) �0processing status markings for a project CrashDB�foozubuntu distro retrace bug�r� r� r� ry r( ro �LP_CREDENTIALSrS �r. r) Zbarzproject retrace bugN)r6 rY r� rq r9 re r2 �environr4 r' r� ry r{ r r rb r� r$ r )rG Z distro_bugr) Z project_dbZ project_bugr� r� r r r �test_marking_projectX s6 � �� �z_T.test_marking_projectc C s� t j�d�pd}tt j�d�d|dd��}|�� }| j�� }| jjjjddg| jj d d �}t d |j � |�� }| j�� }| � ||� | � ||� t|j g��� d S ) r� r( ro r� �ubuntuZfakearch)r- r) r/ r� zneed-fakearch-retracez&ubuntu distro retrace bug for fakearchr� z4fake arch bug: https://staging.launchpad.net/bugs/%iN)r2 r� r4 r' r� r6 rY r� rq re �printr{ rb r� r$ )rG r) Z fakearch_dbZfakearch_unretraced_beforeZ systemarch_unretraced_beforer� Zfakearch_unretraced_afterZsystemarch_unretraced_afterr r r �test_marking_foreign_arch{ s, �� � �z _T.test_marking_foreign_archc C s� | j �� }| �| �� |� | �| �� |� | j �| �� | j� | j �� }| �| �� |� | �||� t | �� g��� | �| j � | �� �d� dS )z2processing status markings for interpreter crashesN) r6 r� ry ro r ri r rU rb r� r$ r )rG �unchecked_before�unchecked_afterr r r �test_marking_python� s �z_T.test_marking_pythonc C sp | j dd�}| j�|�}| j�||| � � � d|d< d|d< d|d< | j�||d � | j�|�}| �d |� d S ) z�updating an invalid crash This simulates a race condition where a crash being processed gets invalidated by marking it as a duplicate. Tr� r} ru r{ r r| r r~ r+ N)ri r6 r� r r� r )rG r{ rg r r r �test_update_traces_invalid� s z _T.test_update_traces_invalidr� c G s\ dt j_| �| �� � | j�| �� �}| �|d� | �| �� � | �| j�| �� �d� dS )z�get_fixed_version() for fixed bugs Other cases are already checked in test_marking_segv() (invalid bugs) and test_duplicates (duplicate bugs) for efficiency. z3.14N) r' r� Z return_value�_mark_report_fixedri r6 r rb �_mark_report_new)rG rx Z fixed_verr r r �test_get_fixed_version� s z_T.test_get_fixed_versionc C s( t j�d�pd}tt j�d�d|d��S )zCreate a CrashDB instancer( ro r� r� )r- r) )r2 r� r4 r' )r4 r) r r r rT � s ��z_T._get_instancec C s@ |j �d�}d|kr&|jj|d d�S |r6|jj| S | jS dS )z&Return the bug_target for this report.r. rs r� N)r< r4 re r� rY rb )rG �dbrk r. r r r rc � s z_T._get_bug_targetNc C sP |dkr d}| j �|�}tr(t�|�}n t�|�}|�� |�� }trN|�� pT|� � }|� � sbt �trn|�� pt|� � }|� � r�t �|� � dks�t �|d|j dd��dd� 7 }| j jjj||d �� ||�d |�� �d �} |D ]6}|� � r�t �| jd |�� d|j dd�|�� d d� q�|d �� D ]&} | j jj| } | �r"| j| d� �q"| jS )zLFile a bug report for a report. Return the bug ID. Nzsome descriptionr� z Tr� rr r� r� rq r� r� Fr� r, r )r6 rg rK r� r� r� rj r� r� r� r� r8 r� r� r� rY r� rq r r4 rt r� r� r r r{ ) rG rh rk r� r� rX r� �headerr� r� Z subscriberr� r r r re � sD � �z _T._file_bugc C s: | j jj| }| j j|jkr6|j| j jg |_|�� dS )z$Mark a report ID as needing retrace.N)r6 rY r� r9 r� r� r� r r r r� s z_T._mark_needs_retracec C s2 | j jj| }d|jkr.|jdg |_|�� dS )z,Mark a report ID as needing duplicate check.r� N)r6 rY r� r� r� r� r r r �_mark_needs_dupcheck$ s z_T._mark_needs_dupcheckc C sB | j jj| }t|j�}t|�dks(t�|d }d|_|�� dS )z Close a report ID as "fixed".r+ r r� N� r6 rY r� r� r� r r8 r� r� �rG r{ r� r% r_ r r r r� , s z_T._mark_report_fixedc C sB | j jj| }t|j�}t|�dks(t�|d }d|_|�� dS )z Reopen a report ID as "new".r+ r �NewNr� r� r r r r� 6 s z_T._mark_report_newc C s | j jj| }| �d|j� dS )z.Verify that report ID is marked as regression.r N)r6 rY r� ry r� r� r r r r� @ s z _T._verify_marked_regressionc C sv t j�d�pd}tt j�d�d|d��}| �|jd� t�d�}d|d < d |d < |�� |� � | �|� � d � | � ||�}| �|j d� | � ||�}| �|d k� tj�d| j|f � |�|�}d|d< d|d< d|d< |�||d� |�|�}| �|�|�d� |�||| �� � | �|�|�| �� � | �|�|�d� |�||d� | �|�|�d� | �|�|�d� dS )z7reporting crashes against a project instead of a distror( ro r� rS r� Nr� rj rk rl r rm r r� r} ru r{ r r| r r~ r� )r2 r� r4 r' rb r- r r� rV rW rt rc r re rd rL rM rN rf r� r� r r rr r� )rG r) r6 rg rh r{ r r r � test_projectF sB �� � z_T.test_projectc C sH | j �| �� �}| �|d d� | �|d d� | �|d �d�� dS )z*download() of uncommon description formatsr r� r* Zamd64r& zUbuntu N)r6 r� rr rb rd r" )rG rg r r r �test_download_robustnessv s z_T.test_download_robustnessc sD t j�d�pd}tt j�d�d|ddd��}d}|jj|jd j� d }z�t||d �D ]�}|d7 }t j � d| � |� t �� || �� � |jj| �� }|jd |jk}t� fdd�|jD ��}|dkr�| �|� | �|� qX| �|� | �|� qXW 5 t||| �D ](}t j � d | � |� t �� |d � �qX t j � d� d S )z,Escalating bugs with more than 10 duplicatesr( ro r� r� Z omgkittenszapport-hackers)r- r) r r r r iV' zR%i N� r+ z%i r c s g | ]}|j � k�qS r )r� )r! �s�r r r r( � s z&_T.test_escalation.<locals>.<listcomp>r � )r2 r� r4 r' rY r r< r �rangerL rM rN r r r� ri r� r� �anyr� r� rd )rG r) r� �countZ first_dupr� Zhas_escalation_tagZhas_escalation_subscriptionr r� r �test_escalation s8 �� z_T.test_escalationc C s� | � | �� � | j�� }| �| �� |� | jjj| �� }|jrNd|_|�� |j d }| jjj d |_ |�� |j | jjj d d� | j�| �� �}| j�| �� |� | j�� }| �| �� |� | �||�t| �� g��� | jjj| �� }| �|j d jd� | �|j d jd� | �|j d jd� | �|j d jd � | �|j d jd� | �|j d jd � | �| j�| �� �d � d S ) z9source package task fixup for marking interpreter crashesFr r� rR )r� r� r� r+ zcoreutils (Ubuntu)r� N)r� ro r6 r� ry rY r� r r� r� ra r� ZaddTaskrb r� r r rb r� r$ r[ r� r� r )rG r� r� r_ rg r� r r r � test_marking_python_task_mangle� s4 �z"_T.test_marking_python_task_manglert c C s& d}t �� }tj�� }z�t�� }t� t j |� t �|� t dd��}|� d� W 5 Q R X t�dddddg�d kstt�t j�d�s�t�tjd d d d d ddgtjd� t j�d�s�t�t�dg� tjdddgtjd�d ks�t�t j�|d�|d< t j�|d�f|d< ||d< |�� W 5 t �|� X |S )a Create a test executable which will die with a SIGSEGV, generate a core dump for it, create a problem report with those two arguments (ExecutablePath and CoreDump) and call add_gdb_info(). Return the apport.report.Report. Nzcrash.c�wzS int f(x) { int* p = 0; *p = x; return x+1; } int main() { return f(42); } Zgccz-gz-oZcrashr Zgdbz--batchz--ex�runzgenerate-core-file corez./crash)�stdoutZcore�syncZreadelfz-nrk r+ rs )r2 �getcwdr rk r� �chdirrA rB rC rD rE rF r rN � subprocess�callr8 rP �exists�PIPEZ check_callr� ra )r4 �signalZworkdirZorig_cwd�pr�fdr r r r^ � s< �� �� z_T._generate_sigsegv_report)F)F)N)rt ).r6 r7 r8 r` rx rY rX r: rf ri ro rr rz r� r� r� r� r� r� r� r� r� r� r� r� r� r� r� rP �objectr' r� r; rT rc re r� r� r� r� r� r� r� r� r� r^ r r r r rQ h sT # $"Q #!6## : 0 ")rQ )Nrp );r9 rA rC �os.pathr2 r] r� rL r� r? rE Zhttplib2r �ior rH rI Zurllib2r r r Zhttplibr Zurllibr r rK Zurllib.requestZ urllib.parseZ http.clientZlaunchpadlib.errorsr Zlaunchpadlib.launchpadr � ImportErrorZapport.crashdbr rP � expanduserr5 r r& r6 r' r= r< rB rh r6 Zunittestr� Z unittest.mockrP r] rn rp ZTestCaserQ �mainr r r r �<module> sh H F 7
Save File
Cancel