|
Server : Apache System : Linux iad1-shared-b8-43 6.6.49-grsec-jammy+ #10 SMP Thu Sep 12 23:23:08 UTC 2024 x86_64 User : dh_edsupp ( 6597262) PHP Version : 8.2.26 Disable Function : NONE Directory : /lib/python3/dist-packages/duplicity/__pycache__/ |
Upload File : |
o
b�aQ6 � @ s� d Z ddlmZ ddlmZ ddlmZ ddlmZmZ ddlZ ddl
Z
ddl Z ddl Z ddl
Z
ddlmZ ddlmZ dd lmZ dadaG d
d � d e j�ZG d d
� d
e�Zdd� ZG dd� de j�ZdS )a�
Functions to compute progress of compress & upload files
The heuristics try to infer the ratio between the amount of data collected
by the deltas and the total size of the changing files. It also infers the
compression and encryption ration of the raw deltas before sending them to
the backend.
With the inferred ratios, the heuristics estimate the percentage of completion
and the time left to transfer all the (yet unknown) amount of data to send.
This is a forecast based on gathered evidence.
� )�absolute_import)�division)�object)�datetime� timedeltaN)�config)�log)�utilc s^ e Zd ZdZedd� �Zdd� Zg df� fdd� Zd d
� Zd d � Z d
d� Z
� fdd�Z � Z S )�Snapshotz�
A convenience class for storing snapshots in a space/timing efficient manner
Stores up to 10 consecutive progress snapshots, one for each volume
c
C s� t � } tjdurHtjjdkrHztdtjj d�}t�|�} |� � W | S t
yG } zt � d�
t�|��t jj� t � } W Y d}~| S d}~ww | S )z9
De-serializes cached data it if present
N� z %s/progress�rzBWarning, cannot read stored progress info from previous backup: {})r
r �restart� start_vol�open�archive_dir_path�name�pickle�load�close� Exceptionr �Warn�formatr �uexc� WarningCode� cannot_stat)�snapshot�
progressfd�e� r �4/usr/lib/python3/dist-packages/duplicity/progress.py�
unmarshall< s
����zSnapshot.unmarshallc C s* t dtjj d�}t�| |� |�� dS )z,
Serializes object to cache
s %s/progresszwb+N)r r r r r �dumpr )�selfr r r r �marshallO s zSnapshot.marshall�
c s t t| ��||� d| _d S �Nr )�superr
�__init__�last_vol)r" �iterable�maxlen�� __class__r r r' W s
zSnapshot.__init__c C s: t | �}|dkr
dS | tdt|| | j d |d �� S )Nr � r )�len�max�minr( )r" �volume�nitemsr r r � get_snapshot[ s &zSnapshot.get_snapshotc C s | � |� || _d S �N)�appendr( )r" r1 �
snapshot_datar r r �
push_snapshota s
zSnapshot.push_snapshotc C s | � � S r4 )�popleft�r" r r r � pop_snapshote s zSnapshot.pop_snapshotc s t t| ��� d| _d S r% )r& r
�clearr( r9 r+ r r r; h s
zSnapshot.clear)
�__name__�
__module__� __qualname__�__doc__� staticmethodr r# r' r3 r7 r: r; �
__classcell__r r r+ r r
6 s
r
c @ sL e Zd Zdd� Zdd� Zdd� Zdd� Zd d
� Zd d � Zd
d� Z dd� Z
dS )�ProgressTrackerc C s| d | _ d| _d | _d| _d| _d| _d| _d| _d| _d| _ d | _
d | _ t � | _
d| _t�� | _d| _d| _d| _d | _d S )Nr r- F)� total_stats�nsteps�
start_time�change_mean_ratio�change_r_estimation�progress_estimation�time_estimation�total_bytecount�last_total_bytecount�last_bytecount�stall_last_time� last_timer � elapsed_sum�speed�sys_collections�deque� transfers�is_full�current_estimation�prev_estimation� prev_datar9 r r r r' o s&
zProgressTracker.__init__c C s, | j dur| j �|| j� | j �� dS dS )z�
Snapshots the current progress status for each volume into the disk cache
If backup is interrupted, next restart will deserialize the data and try start
progress from the snapshot
N)rW r7 rH r# �r" r1 r r r �snapshot_progress� s
�z!ProgressTracker.snapshot_progressc C s
| j duS )z�
Returns true if the progress computation is on and duplicity has not
yet started the first dry-run pass to collect some information
N)rC r9 r r r �has_collected_evidence� s
z&ProgressTracker.has_collected_evidencec C sb t jr| �� s dS t�� }| jdu r|| _| jdur || j }nt� }|| _| jdu r.|| _|| j j t
ddt j �krRt �
d| j | j| j|| j j | jd� dS | jd7 _ ddlm} |jj|jj }| jj| jj }|dksz|jjdkr|dS | j}| jr�t| j�t|� | _ n�t| j�t|jj� }|| j }| j |t| j� 7 _ | j ||| j 7 _ t � t �!| j t| j� ��} t|�t|� | j d | d
| j | j d | | j | _ | j d
k�r t|�t|� | j d | d
| j | j d | | j | _ | j d
k�r | j t|� t|� | _ t
d t"| j#d
| j# | j d
��| _ | j$|7 _$d
}
| jdk�rGd
| j | j }
t%|
t| j$�&� � �| _| j|k �r\|| _ |�&� dk�rv| j'�(t| j| j) �t|�&� � � | j| _)t*| j'�dk�r�| j'�+� d | _| j'D ]
} d| d| j | _�q�t �
d| j | j| j|| j j | jd� dS )zV
Aproximative and evolving method of computing the progress of upload
N� � g Y@Tr r )�diffdirgq=
ףp�?g �?g ��Q� �?r- �G�z��?� g333333�?gffffff�?F),r �progressrZ r �nowrE rN r rM �secondsr/ �
progress_rater �TransferProgressrH rI rJ rP rD � duplicityr] �stats� NewFileSize�ChangedFileSizerC � RawDeltaSizerT �floatrU rF rG �math�sqrt�fabsr0 rV rO �int�
total_secondsrS r5 rK r. r8 ) r" � current_time�elapsedr] �changes�
total_changes�last_progress_estimation� change_ratio� change_delta� change_sigma�
projection�xr r r �log_upload_progress� s�
�
�� ��
���
�
�z#ProgressTracker.log_upload_progressc C sB t || j d�}| jt|�7 _|| _|dkr t�� | _dS dS )z�
Annotate the number of bytes that have been added/changed since last time
this function was called.
bytecount param will show the number of bytes since the start of the current
volume and for the current volume
r N)r/ rL rJ rn r ra rM )r" � bytecount�changingr r r �annotate_written_bytes s �z&ProgressTracker.annotate_written_bytesc C s || _ || _dS )z�
Stores the collected statistics from a first-pass dry-run, to use this
information later so as to estimate progress
N)rC rT )r" rf rT r r r � set_evidence( s
z ProgressTracker.set_evidencec C s0 t �� | _| j�|�| _tdt| jd��| _d S )Nr- r^ )r
r rW r3 rV r/ r0 rH rX r r r �set_start_volume0 s
z ProgressTracker.set_start_volumec C s t �� | j jS )zT
Elapsed seconds since the first call to log_upload_progress method
)r ra rE rb r9 r r r �total_elapsed_seconds5 s z%ProgressTracker.total_elapsed_secondsN) r< r= r> r' rY rZ rz r} r~ r r� r r r r rB m s
rB c C s& t durtdurt�| � dS dS dS )z�
Method to call tracker.annotate_written_bytes from outside
the class, and to offer the "function(long, long)" signature
which is handy to pass as callback
N)�progress_thread�trackerr} )r{ �
totalbytesr r r �report_transfer< s �r� c s( e Zd ZdZ� fdd�Zdd� Z� ZS )�LogProgressThreadz_
Background thread that reports progress to the log,
every --progress-rate seconds
c s" t t| ��� | �d� d| _d S )NTF)r& r� r' � setDaemon�finishedr9 r+ r r r' M s
zLogProgressThread.__init__c C sH t js t jr t�� r | js"t�� t�t j � | jr
d S d S d S d S d S r4 )
r �dry_runr` r� rZ r� rz �time�sleeprc r9 r r r �runR s
� �zLogProgressThread.run)r< r= r> r? r' r� rA r r r+ r r� H s r� )r? �
__future__r r �builtinsr r r � collectionsrQ rk r � threadingr� re r r r r� r� rR r
rB r� �Threadr� r r r r �<module> s( 7 P