o d8y@sPdZddlmZddlmZmZddlmZmZm Z ddl Z ddl m Z mZmZddlmZmZmZmZmZddlmZdd lmZdd lmZmZmZmZdd lm Z m!Z!m"Z"er`dd l#m$Z$dyddZ%dzddZ&d{d|ddZ'gdZ(gd Z)d}d$d%Z*d~d)d*Z+ +    ,    ddd:d;Z, <  ,  ddd=d>Z-dd?d@Z. <  ,    dddEdFZ/   dddGdHZ0 dddLdMZ1dddNdOZ2  P dddSdTZ3ddVdWZ4 +   dddYdZZ5 ddd\d]Z6dd`daZ7e7  dddcddZ8e7  dddedfZ9e7dddgdhZ:e7dddidjZ;e8e9dkZddsdtZ?ddwdxZ@dS)z$ Routines for filling missing data. ) annotations)partialwraps) TYPE_CHECKINGAnycastN)NaTalgoslib) ArrayLikeAxisAxisIntFnpt)import_optional_dependency)infer_dtype_from) is_array_likeis_numeric_v_string_likeis_object_dtypeneeds_i8_conversion)is_valid_na_for_dtypeisnana_value_for_dtype)Indexmasknpt.NDArray[np.bool_]lengthintcCs8t|rt||krtdt|d|||}|S)zJ Validate the size of the values passed to ExtensionArray.fillna. z'Length of 'value' does not match. Got (z ) expected )rlen ValueError)valuerrr!G/app/.heroku/python/lib/python3.10/site-packages/pandas/core/missing.pycheck_value_size1s  r#arrr returnc Cst|\}}tj||d}d}t|rd}t|}t|}||}tj|jtd}|D]1}t||r5q-|rItj|jtj d} |||k| |<n||k} t | tj sZ| j tdd} || O}q-| ri|t|O}|S)a  Return a masking array of same size/shape as arr with entries equaling any member of values_to_mask set to True Parameters ---------- arr : ArrayLike values_to_mask: list, tuple, or scalar Returns ------- np.ndarray[bool] )dtypeFT)r&Zna_value)rnparrayrrZzerosshapeboolrZbool_ isinstancendarrayZto_numpyany) r$Zvalues_to_maskr&Z potential_naZarr_maskZna_maskZnonnarxZnew_maskr!r!r" mask_missing@s,       r/Fmethod str | None allow_nearestr*cCsv|dvrdSt|tr|}|dkrd}n|dkrd}ddg}d}|r+|dd}||vr9td |d ||S) N)NZasfreqZffillpadZbfillbackfillzpad (ffill) or backfill (bfill)nearestz(pad (ffill), backfill (bfill) or nearestzInvalid fill method. Expecting z. Got )r+strlowerappendr)r0r2Z valid_methodsZ expectingr!r!r"clean_fill_methodys   r9)lineartimeindexvalues)r5zeroslinear quadraticcubic barycentrickroghspline polynomialfrom_derivativespiecewise_polynomialpchipakima cubicspliner6r<rcKsh|d}|dvr|durtdtt}||vr$td|d|d|dvr2|js2t|d|S) Norder)rDrEz7You must specify the order of the spline or polynomial.zmethod must be one of z. Got 'z ' instead.)rCrGrHz4 interpolation requires that the index be monotonic.)getr NP_METHODS SP_METHODSZis_monotonic_increasing)r0r<kwargsrKvalidr!r!r"clean_interp_methods rQhowis_valid int | NonecCs|dvsJt|dkrdS|jdkr|jdd}|dkr&|dd}n|dkr9t|d|ddd }||}|sAdS|S) aG Retrieves the index of the first valid value. Parameters ---------- values : ndarray or ExtensionArray how : {'first', 'last'} Use this parameter to change between the first or last valid index. is_valid: np.ndarray Mask to find na_values. Returns ------- int or None )firstlastrNaxisrUrV)rndimr-Zargmax)r=rRrSZidxposZ chk_notnar!r!r"find_valid_indexs    r]r3forwarddata np.ndarrayrZr Index | Nonelimitlimit_direction limit_area fill_value Any | NonecoercedowncastNonec Kszt|} Wn tyd} Ynw| dur)|durtdt|| |||ddS|dus/Jtd||||||||d| dS)z Wrapper to dispatch to either interpolate_2d or _interpolate_2d_with_fill. Notes ----- Alters 'data' in-place. Nz&Cannot pass both fill_value and method)r0rZrbrd)r_r<rZr0rbrcrdrer!)r9rinterpolate_2d_interpolate_2d_with_fill) r_r0rZr<rbrcrdrergrhrOmr!r!r"interpolate_array_2ds8    rmr:c  st|fit|jrt|jdddkr%t|js#tddgd} | vr.func)rsr`r%ri) rQrr&rrrr7r Zvalidate_limit_index_to_interp_indicesr'apply_along_axis) r_r<rZr0rbrcrdrerOZvalid_limit_directionsZvalid_limit_areasrxr!rwr"rks@   rkcCsb|j}t|jr |d}|dkr|}ttj|}|St|}|dvr/|jtjkr/t |}|S)zE Convert Index to ndarray of indices to pass to NumPy/SciPy. i8r:)r=r<) _valuesrr&viewrr'r,asarrayZobject_r Zmaybe_convert_objects)r<r0ZxarrZindsr!r!r"ryjs      ryrtrsrurKc Kst|} | } | s dS| rdStt| } t|d| d} | dur'd} tt| }t|d| d}|durr?r@rArErE)kindrerurDz;order needs to be specified and greater than 0; got order: k)rrrr'r~Zbarycentric_interpolateZkrogh_interpolate_from_derivativesgetattrr|ZastypeZpchip_interpolate_akima_interpolate_cubicspline_interpolateZinterp1drrZUnivariateSplineflagsZ writeablecopy)r.yZnew_xr0rerurKrOrrZ alt_methodsZinterp1d_methodsZterpZnew_yr!r!r"rsV         rderint | list[int] | None extrapolatec Cs4ddlm}|jj}|||dd||d}||S)a Convenience function for interpolate.BPoly.from_derivatives. Construct a piecewise polynomial in the Bernstein basis, compatible with the specified values and derivatives at breakpoints. Parameters ---------- xi : array-like sorted 1D array of x-coordinates yi : array-like or list of array-likes yi[i][j] is the j-th derivative known at xi[i] order: None or int or array-like of ints. Default: None. Specifies the degree of local polynomials. If not None, some derivatives are ignored. der : int or list How many derivatives to extract; None for all potentially nonzero derivatives (that is a number equal to the number of points), or a list of derivatives to extract. This number includes the function value as 0th derivative. extrapolate : bool, optional Whether to extrapolate to ouf-of-bounds points based on first and last intervals, or to return NaNs. Default: True. See Also -------- scipy.interpolate.BPoly.from_derivatives Returns ------- y : scalar or array-like The result, of length R or length M or M by R. rrr[rX)Zordersr)rrZBPolyrFreshape) xiyir.rKrrrr0rlr!r!r"r9s $rcCs(ddlm}|j|||d}|||dS)a[ Convenience function for akima interpolation. xi and yi are arrays of values used to approximate some function f, with ``yi = f(xi)``. See `Akima1DInterpolator` for details. Parameters ---------- xi : array-like A sorted list of x-coordinates, of length N. yi : array-like A 1-D array of real values. `yi`'s length along the interpolation axis must be equal to the length of `xi`. If N-D array, use axis parameter to select correct axis. x : scalar or array-like Of length M. der : int, optional How many derivatives to extract; None for all potentially nonzero derivatives (that is a number equal to the number of points), or a list of derivatives to extract. This number includes the function value as 0th derivative. axis : int, optional Axis in the yi array corresponding to the x-coordinate values. See Also -------- scipy.interpolate.Akima1DInterpolator Returns ------- y : scalar or array-like The result, of length R or length M or M by R, rrrY)nu)rrZAkima1DInterpolator)rrr.rrZrPr!r!r"rfs $ r not-a-knotbc_typestr | tuple[Any, Any]cCs(ddlm}|j|||||d}||S)aq Convenience function for cubic spline data interpolator. See `scipy.interpolate.CubicSpline` for details. Parameters ---------- xi : array-like, shape (n,) 1-d array containing values of the independent variable. Values must be real, finite and in strictly increasing order. yi : array-like Array containing values of the dependent variable. It can have arbitrary number of dimensions, but the length along ``axis`` (see below) must match the length of ``x``. Values must be finite. x : scalar or array-like, shape (m,) axis : int, optional Axis along which `y` is assumed to be varying. Meaning that for ``x[i]`` the corresponding values are ``np.take(y, i, axis=axis)``. Default is 0. bc_type : string or 2-tuple, optional Boundary condition type. Two additional equations, given by the boundary conditions, are required to determine all coefficients of polynomials on each segment [2]_. If `bc_type` is a string, then the specified condition will be applied at both ends of a spline. Available conditions are: * 'not-a-knot' (default): The first and second segment at a curve end are the same polynomial. It is a good default when there is no information on boundary conditions. * 'periodic': The interpolated functions is assumed to be periodic of period ``x[-1] - x[0]``. The first and last value of `y` must be identical: ``y[0] == y[-1]``. This boundary condition will result in ``y'[0] == y'[-1]`` and ``y''[0] == y''[-1]``. * 'clamped': The first derivative at curves ends are zero. Assuming a 1D `y`, ``bc_type=((1, 0.0), (1, 0.0))`` is the same condition. * 'natural': The second derivative at curve ends are zero. Assuming a 1D `y`, ``bc_type=((2, 0.0), (2, 0.0))`` is the same condition. If `bc_type` is a 2-tuple, the first and the second value will be applied at the curve start and end respectively. The tuple values can be one of the previously mentioned strings (except 'periodic') or a tuple `(order, deriv_values)` allowing to specify arbitrary derivatives at curve ends: * `order`: the derivative order, 1 or 2. * `deriv_value`: array-like containing derivative values, shape must be the same as `y`, excluding ``axis`` dimension. For example, if `y` is 1D, then `deriv_value` must be a scalar. If `y` is 3D with the shape (n0, n1, n2) and axis=2, then `deriv_value` must be 2D and have the shape (n0, n1). extrapolate : {bool, 'periodic', None}, optional If bool, determines whether to extrapolate to out-of-bounds points based on first and last intervals, or to return NaNs. If 'periodic', periodic extrapolation is used. If None (default), ``extrapolate`` is set to 'periodic' for ``bc_type='periodic'`` and to True otherwise. See Also -------- scipy.interpolate.CubicHermiteSpline Returns ------- y : scalar or array-like The result, of shape (m,) References ---------- .. [1] `Cubic Spline Interpolation `_ on Wikiversity. .. [2] Carl de Boor, "A Practical Guide to Splines", Springer-Verlag, 1978. rr)rZrr)rrZ CubicSpline)rrr.rZrrrrr!r!r"rs M rr=cCst|}|}|sTt|d|d}|durd}t|d|d}|dur't|}t|||d|dkr;d|||d <n|d krMd|d|<||d d<tj||<dSdS) a Apply interpolation and limit_area logic to values along a to-be-specified axis. Parameters ---------- values: np.ndarray Input array. method: str Interpolation method. Could be "bfill" or "pad" limit: int, optional Index limit on interpolation. limit_area: str Limit area for interpolation. Can be "inside" or "outside" Notes ----- Modifies values in-place. rUrNrrV)r0rbrpFrXrq)rrr]rrjr'r)r=r0rbrdrrSrUrVr!r!r"_interpolate_with_limit_areas(rr cCs|durttt|||d||dS|dkrddndd}|jdkr6|dkr,td|td |j}t |}||}|d krJt ||d dSt ||d dS) a Perform an actual interpolation of values, values will be make 2-d if needed fills inplace, returns the result. Parameters ---------- values: np.ndarray Input array. method: str, default "pad" Interpolation method. Could be "bfill" or "pad" axis: 0 or 1 Interpolation axis limit: int, optional Index limit on interpolation. limit_area: str, optional Limit area for interpolation. Can be "inside" or "outside" Notes ----- Modifies values in-place. N)r0rbrdrcSs|SNr!r.r!r!r"Isz interpolate_2d..cSs|jSr)Trr!r!r"rIsrXz0cannot interpolate on a ndim == 1 with axis != 0rXr3rb) r'rzrrr\AssertionErrorrtupler)r9_pad_2d _backfill_2d)r=r0rZrbrdZtransfZtvaluesr!r!r"rjs0    rjnpt.NDArray[np.bool_] | NonecCs |durt|}|tj}|Sr)rr}r'Zuint8)r=rr!r!r" _fillna_prep]s rrxrcs tdfdd }tt|S)z> Wrapper to handle datetime64 and timedelta64 dtypes. NcsPt|jr!|dur t|}|d||d\}}||j|fS|||dS)Nr{)rbr)rr&rr})r=rbrresultrxr!r"new_funcns z&_datetimelike_compat..new_funcNN)rrr)rxrr!rr"_datetimelike_compatis r(tuple[np.ndarray, npt.NDArray[np.bool_]]cC"t||}tj|||d||fSNr)rr Z pad_inplacer=rbrr!r!r"_pad_1d} rcCrr)rr Zbackfill_inplacerr!r!r" _backfill_1drrcC8t||}t|jrtj|||d||fS ||fSr)rr'rr)r Zpad_2d_inplacerr!r!r"r  rcCrr)rr'rr)r Zbackfill_2d_inplacerr!r!r"rrrr3r4rXr\cCs&t|}|dkr t|Sttd|S)NrXr)r9 _fill_methodsrr)r0r\r!r!r" get_fill_funcsrcCs t|ddS)NT)r2)r9)r0r!r!r"clean_reindex_fill_methods rrcst|t}t}fdd}|dur'|dkr"tt|d}n|||}|durN|dkr1|St||ddd|}tdt|}|dkrN|S||@S)ak Get indexers of values that won't be filled because they exceed the limits. Parameters ---------- invalid : np.ndarray[bool] fw_limit : int or None forward limit to index bw_limit : int or None backward limit to index Returns ------- set of indexers Notes ----- This is equivalent to the more readable, but slower .. code-block:: python def _interp_limit(invalid, fw_limit, bw_limit): for x in np.where(invalid)[0]: if invalid[max(0, x - fw_limit):x + bw_limit + 1].all(): yield x cs`t|}t||dd}tt|d|tt|d|ddkdB}|S)NrXr)min_rolling_windowrrr'whereZcumsum)rrbZwindowedidxNr!r"inners "z_interp_limit..innerNrr[rX)rrr'rlistr~)rZfw_limitZbw_limitZf_idxZb_idxrZ b_idx_invr!rr"rs   rawindowcCsJ|jdd|jd|d|f}|j|jdf}tjjj|||dS)z [True, True, False, True, False], 2 -> [ [True, True], [True, False], [False, True], [True, False], ] Nr[rX)r)strides)r)rr'r Z stride_tricksZ as_strided)rrr)rr!r!r"rs$ r)rrrr)r$r r%r)F)r0r1r2r*)r0r6r<rr%r6)rRr6rSrr%rT) r3rNNr^NNFN)r_r`r0r6rZr r<rarbrTrcr6rdr1rerfrgr*rhr1r%ri)r:Nr^NN)r_r`r<rrZr r0r6rbrTrcr6rdr1rerfr%ri)r<rr0r6r%r`)r:Nr^NNFN)rtr`rsr`r0r1rbrTrcr6rdr1rerfrur*rKrTr%ri)NFN)rur*)NrF)rrrr*)rr)rrrZr )rrN)rZr rr) r=r`r0r6rbrTrdr1r%ri)r3rNN) r=r`r0r6rZr rbrTrdr1r%rir)rrr%r)rxrr%rr)r=r`rbrTrrr%r)r=r`rr)rrr)r\r)r%r1)rr)rrrrr%r)A__doc__ __future__r functoolsrrtypingrrrnumpyr'Z pandas._libsrr r Zpandas._typingr r r rrZpandas.compat._optionalrZpandas.core.dtypes.castrZpandas.core.dtypes.commonrrrrZpandas.core.dtypes.missingrrrZpandasrr#r/r9rMrNrQr]rmrkryrvrrrrrrjrrrrrrrrrrrr!r!r!r"s       9  + 9 R q N -/ V1 H       ?