|
Server : Apache System : Linux iad1-shared-b8-43 6.6.49-grsec-jammy+ #10 SMP Thu Sep 12 23:23:08 UTC 2024 x86_64 User : dh_edsupp ( 6597262) PHP Version : 8.2.26 Disable Function : NONE Directory : /lib/python3/dist-packages/duplicity/__pycache__/ |
Upload File : |
o
�~�_;* � @ s� d Z ddlmZ e�� ddlmZ ddlZddlmZ ddlm Z ddlm
Z
ddlm Z dd lm Z ej
�� Zej
�� ZG d
d � d e�ZdS ) z_
Asynchronous job scheduler, for concurrent execution with minimalistic
dependency guarantees.
� )�standard_library)�objectN)�log)�require_threading)�interruptably_wait)� async_split)� with_lockc @ sP e Zd ZdZdd� Zdd� Zdd� Zdd � Zd
d � Zd d
� Z dd� Z
dd� Z dS )�AsyncSchedulera�
Easy-to-use scheduler of function calls to be executed
concurrently. A very simple dependency mechanism exists in the
form of barriers (see insert_barrier()).
Each instance has a concurrency level associated with it. A
concurrency of 0 implies that all tasks will be executed
synchronously when scheduled. A concurrency of 1 indicates that a
task will be executed asynchronously, but never concurrently with
other tasks. Both 0 and 1 guarantee strict ordering among all
tasks (i.e., they will be executed in the order scheduled).
At concurrency levels above 1, the tasks will end up being
executed in an order undetermined except insofar as is enforced by
calls to insert_barrier().
An AsynchScheduler should be created for any independent process;
the scheduler will assume that if any background job fails (raises
an exception), it makes further work moot.
c C s� t �d| jjtd�| f � |dks J d| jjf ��d| _d| _|| _d| _d| _ d| _
t � � | _
|dkr@td|f � dS dS )zr
Create an asynchronous scheduler that executes jobs with the
given level of concurrency.
�%s: %sz instantiating at concurrency %dr z!%s concurrency level must be >= 0FNzconcurrency > 0 (%d))r �Info� __class__�__name__�_�_AsyncScheduler__failed� _AsyncScheduler__failed_waiter� _AsyncScheduler__concurrency� _AsyncScheduler__worker_count� _AsyncScheduler__waiter_count�_AsyncScheduler__barrier� threading� Condition�_AsyncScheduler__cvr )�self� concurrency� r �:/usr/lib/python3/dist-packages/duplicity/asyncscheduler.py�__init__@ s ��
�zAsyncScheduler.__init__c sD t �d� jjtd�f � � jdkr � fdd�}t� j|� dS dS )aL
Proclaim that any tasks scheduled prior to the call to this
method MUST be executed prior to any tasks scheduled after the
call to this method.
The intended use case is that if task B depends on A, a
barrier must be inserted in between to guarantee that A
happens before B.
r
zinserting barrierr c s
d� _ d S �NT)r r �r r r �_insert_barrierf s
z6AsyncScheduler.insert_barrier.<locals>._insert_barrierN)r �Debugr r
r r r r )r r r r r �insert_barrierW s
�z AsyncScheduler.insert_barrierc C sn |dusJ �| j dkr!t�d| jjtd�f tjj� | �||�S t�d| jjtd�f tjj � | �
||�S )a�
Schedule the given task (callable, typically function) for
execution. Pass the given parameters to the function when
calling it. Returns a callable which can optionally be used
to wait for the task to complete, either by returning its
return value or by propagating any exception raised by said
task.
This method may block or return immediately, depending on the
configuration and state of the scheduler.
This method may also raise an exception in order to trigger
failures early, if the task (if run synchronously) or a previous
task has already failed.
NOTE: Pay particular attention to the scope in which this is
called. In particular, since it will execute concurrently in
the background, assuming fn is a closure, any variables used
most be properly bound in the closure. This is the reason for
the convenience feature of being able to give parameters to
the call, to avoid having to wrap the call itself in a
function in order to "fixate" variables in, for example, an
enclosing loop.
Nr r
z4running task synchronously (asynchronicity disabled)z*scheduling task for asynchronous execution) r r r r r
r �InfoCode�synchronous_upload_begin�"_AsyncScheduler__run_synchronously�asynchronous_upload_begin�#_AsyncScheduler__run_asynchronously)r �fn�paramsr r r �
schedule_taskk s
�� �� z AsyncScheduler.schedule_taskc s � fdd�}t � j|� dS )aD
Wait for the scheduler to become entirely empty (i.e., all
tasks having run to completion).
IMPORTANT: This is only useful with a single caller scheduling
tasks, such that no call to schedule_task() is currently in
progress or may happen subsequently to the call to wait().
c s t � j� fdd�� d S )Nc s � j dko � jdkS )Nr )r r r r r r �<lambda>� s z4AsyncScheduler.wait.<locals>._wait.<locals>.<lambda>)r r r r r r �_wait� s z"AsyncScheduler.wait.<locals>._waitN)r r )r r+ r r r �wait� s zAsyncScheduler.waitc s8 ||� � � fdd�}t �d| jjtd�f t jj� |S )Nc s � S �Nr r ��retr r �_waiter� s z3AsyncScheduler.__run_synchronously.<locals>._waiterr
ztask completed successfully)r r r r
r r" �synchronous_upload_done)r r' r( r0 r r. r �__run_synchronously� s ��z"AsyncScheduler.__run_synchronouslyc sJ t ��fdd��\}}�fdd�� � �fdd�}t�j|� ��|� |S )Nc s � �� S r- r r )r' r( r r r* � s z5AsyncScheduler.__run_asynchronously.<locals>.<lambda>c sD � j r t�d� jjtd�f tjj� � �� t d� jjf ��d S )Nr
zJa previously scheduled task has failed; propagating the result immediatelyz9%s: waiter should have raised an exception; this is a bug)
r r r r r
r r" �asynchronous_upload_doner �AssertionErrorr r r r �check_pending_failure� s ����zBAsyncScheduler.__run_asynchronously.<locals>.check_pending_failurec s� � � �j �jks �jr@�j dkr!�jsJ d��d�_�j�� n� jd7 _�j�� � jd8 _� � �j �jks �js � j d7 _ t�d�j j
t d��j f f � d S )Nr zbarrier should be in effectF� r
�active workers = %d) r r r r � notifyAllr r, r r r r
r r )r5 r r r � wait_for_and_register_launch� s
� �zIAsyncScheduler.__run_asynchronously.<locals>.wait_for_and_register_launch)r r r � _AsyncScheduler__start_worker)r r' r( �waiter�callerr9 r )r5 r' r( r r �__run_asynchronously� s
z#AsyncScheduler.__run_asynchronouslyc s � �fdd�}t �|d� dS )z%
Start a new worker.
c sD z�� � � W �fdd�} t�j| � d S �fdd�} t�j| � w )Nc s> � j d8 _ t�d� jjtd�� j f f � � j�� d S )Nr6 r
r7 )r r r r r
r r r8 r r r r �complete_worker� s
�zJAsyncScheduler.__start_worker.<locals>.trampoline.<locals>.complete_worker)� _AsyncScheduler__execute_callerr r )r> �r< r r r �
trampoline� s �z1AsyncScheduler.__start_worker.<locals>.trampoliner N)�thread�start_new_thread)r r< rA r r@ r �__start_worker� s z AsyncScheduler.__start_workerc sP |� \}�|s� �fdd�}t � j|� t�d� jjtd�| f tjj� d S )Nc s$ � j sd� _ �� _� j�� d S d S r )r r r r8 r �r r; r r �_signal_failed� s
�z7AsyncScheduler.__execute_caller.<locals>._signal_failedr
z!task execution done (success: %s)) r r r r r r
r r" r3 )r r< � succeededrF r rE r �__execute_caller� s
��z AsyncScheduler.__execute_callerN) r
�
__module__� __qualname__�__doc__r r! r) r, r$ r& r: r? r r r r r * s 2' r )rK �futurer �install_aliases�builtinsr � duplicityr �duplicity.dup_threadingr r r r �
dup_threading�
thread_modulerB �threading_moduler r r r r r �<module> s