Metrics

Submodules

Calculations module

In: scintillometry.metrics.calculations.py

Calculates metrics from datasets.

class scintillometry.metrics.calculations.MetricsFlux

Bases: object

Calculate metrics for fluxes.

constants

Inherited atmospheric constants.

Type:

AtmosConstants

plotting

Inherited methods for plotting figures.

Type:

FigurePlotter

derivation

Inherited methods for deriving parameters and fluxes from scintillometer.

Type:

DeriveScintillometer

construction

Inherited methods for constructing vertical profiles.

Type:

ProfileConstructor

iteration

Inherited methods for MOST iterative method.

Type:

IterationMost

append_vertical_variables(data)

Derives vertical measurements and appends them to input.

For several days or more of data, the returned dictionary may be quite large.

Parameters:

data (dict) –

Must at least contain the keys “vertical” and “weather”, where “vertical” is a dictionary of vertical measurements for temperature and humidity:

data = {
    "weather": weather_data,
    "timestamp": bls_time,
    "vertical": {
        "temperature": pd.DataFrame,
        "humidity": pd.DataFrame,
        }
    }

Returns:

If the input dictionary contained a dictionary under the key “vertical” measurements, the dictionary under “vertical” is updated with vertical data for water vapour pressure, air pressure, mixing ratio, virtual temperature, mean sea-level pressure, and potential temperature. Otherwise, the dictionary is returned unmodified.

Return type:

dict

calculate_switch_time(datasets, method='sun', switch_time=None, location='')

Calculates and plots local time of switch in stability.

Optionally uses vertical measurements in a dictionary under datasets[“vertical”] to plot potential temperature profiles.

Parameters:

• datasets (dict) – Parsed and localised dataframes, containing data to construct a potential temperature profile, or eddy covariance data, or global irradiance.

• method (str) – Method to calculate switch time. Default “sun”.

• switch_time (Union[str, pd.Timestamp]) – Local time of switch between stability conditions. Overrides <method>. Default None.

• location (str) – Location of data collection. Default empty string.

Returns:

Local time of switch between stability conditions.

Return type:

pd.Timestamp

compare_lapse_rates(air_temperature, saturated, unsaturated)

Compares parcel temperatures to find instability.

Parameters:

• air_temperature (pd.DataFrame) – Vertical measurements, temperature, \(T\) [K].

• saturated (pd.DataFrame) – Vertical measurements, saturated parcel temperature, \(T_{sat}\) [K].

• unsaturated (pd.DataFrame) – Vertical measurements, unsaturated parcel temperature, \(T_{unsat}\) [K].

Returns:

Boolean series of absolute and conditional instability for heights \(z\).

Return type:

tuple[pd.Series[bool], pd.Series[bool]]

construct_flux_dataframe(interpolated_data, z_eff, beam_wavelength=880, beam_error=20)

Compute sensible heat flux for free convection.

Computes sensible heat flux for free convection, plots a comparison between the computed flux and the flux recorded by the scintillometer, and saves the figure to disk.

Warning: this will overwrite existing values for \(C_{T}^{2}\) in <interpolated_data>.

Parameters:

• interpolated_data (pd.DataFrame) – Dataframe containing parsed and localised weather and scintillometer data with matching temporal resolution.

• z_eff (np.floating) – Effective path height, z_eff| [m].

• beam_wavelength (int) – Transmitter beam wavelength, nm. Default 880 nm.

• beam_error (int) – Transmitter beam error, nm. Default 20 nm.

Keyword Arguments:

• beam_wavelength (int) – Transmitter beam wavelength, nm. Default 880 nm.

• beam_error (int) – Transmitter beam error, nm. Default 20 nm.

Returns:

Interpolated dataframe with additional column for sensible heat flux under free convection, and derived values for \(C_{T}^{2}\) [K \(^{2}\cdot\) m \(^{-2/3}\)].

Return type:

pd.DataFrame

get_boundary_height(grad_potential, time_index, max_height=2000)

Estimate height of boundary layer from potential temperature.

Estimates the height of the boundary layer by calculating the elbow point of the gradient potential temperature using the Kneedle algorithm, i.e. where the gradient potential temperature starts to weaken. It is not a substitute for visual inspection.

Parameters:

• grad_potential (pd.DataFrame) – Vertical measurements, gradient potential temperature, \(\Delta \theta/\Delta z\) [Km ^-1].

• time_index (pd.Timestamp) – Local time at which to estimate boundary layer height.

• max_height (int) – Cutoff height for estimating boundary layer height. Default 2000.

Returns:

Estimated boundary layer height, \(z_{BL}\) [m].

Return type:

int

get_elbow_point(series, min_index=None, max_index=None)

Calculate elbow point using Kneedle algorithm.

Only supports convex curves. Noisier curves may have several elbow points, in which case the function selects the smallest acceptable index.

Parameters:

• series (pd.Series) – Numeric data following a convex curve.

• min_index (Any) – Indices below this threshold are discarded. Default None.

• max_index (Any) – Indices above this threshold are discarded. Default None.

Returns:

Integer index of elbow point. Returns None if no elbow point is found.

Return type:

int

get_nearest_time_index(data, time_stamp)

Get timestamp of dataframe index nearest to input timestamp.

Parameters:

• data (pd.DataFrame) – Dataframe with DatetimeIndex.

• time_stamp (pd.Timestamp) – Time of day.

Returns:

Closest index to time stamp.

Return type:

pd.Timestamp

get_regression(x_data, y_data, intercept=True)

Performs regression on labelled data.

Parameters:

• x_data (pd.Series) – Labelled explanatory data.

• y_data (pd.Series) – Labelled response data.

• intercept (bool) – If True, calculate intercept (e.g. data is

• True. (not centred). Default) –

Returns:

Contains the fitted estimator for regression data, the coefficient of determination math:R^{2}, and predicted values for a fitted regression line.

Return type:

dict

get_switch_time(data, method='sun', local_time=None, ri_crit=0.25)

Gets local time of switch between stability conditions.

To override automatic detection, pass one of the following to the <method> argument:

• eddy: eddy covariance (NotImplemented)

• sun: global irradiance (i.e. sunrise)

• static: static stability from potential temperature profile

• bulk: bulk Richardson number

• lapse: temperature lapse rate

• brunt: Brunt-Väisälä frequency (NotImplemented)

To manually set the regime switch time, pass a localised timestamp or string to <local_time>. This overrides all other methods.

Parameters:

• data (dict) – Parsed and localised dataframes, containing data to construct a potential temperature profile, or eddy covariance data, or global irradiance.

• method (str) – Method to calculate switch time. Default “sun”.

• local_time (Union[str, pd.Timestamp]) – Local time of switch between stability conditions. Overrides <method>. Default None.

• ri_crit (float) – Critical bulk Richardson number for CBL. Only used if method = “bulk”. For street canyons consider values between 0.5 - 1.0[2]. Default 0.25[1].

Returns:

Local time of switch between stability conditions.

Return type:

pd.Timestamp

Raises:

UnboundLocalError – No data to calculate switch time. Set <local_time> manually with –switch-time.

get_switch_time_vertical(data, method='static', ri_crit=0.25)

Gets local time of switch between stability conditions.

Pass one of the following to the <method> argument:

• eddy: eddy covariance (NotImplemented)

• static: static stability from potential temperature profile

• bulk: bulk Richardson number

• lapse: temperature lapse rate

• brunt: Brunt-Väisälä frequency (NotImplemented)

Parameters:

• data (dict) – Parsed and localised dataframes, containing vertical measurements to construct a potential temperature profile or calculate lapse rates.

• method (str) – Method to calculate switch time. Default “static”.

• ri_crit (float) – Critical bulk Richardson number for CBL. Only used if method = “bulk”. For street canyons consider values between 0.5 - 1.0[2]. Default 0.25[1].

Returns:

Local time of switch between stability conditions.

Return type:

pd.Timestamp

Raises:

NotImplementedError – Switch time algorithm not implemented for <method>.

iterate_fluxes(z_parameters, datasets, most_id='an1988', algorithm='sun', switch_time=None, location='')

Compute sensible heat fluxes with MOST through iteration.

Trades speed from vectorisation for more accurate convergence.

Parameters:

• z_parameters (dict[str, tuple[float, float]) – Tuples of effective and mean path height \(z_{eff}\) and \(\bar{z}\) [m], with stability conditions as keys.

• datasets (dict) – Contains parsed, tz-aware dataframes, with at least \(C_{T}^{2}\), wind speed, air density, and temperature.

• most_id (str) – MOST coefficients for unstable and stable conditions. Default “an1988”.

• algorithm (str) – Method to calculate switch time. Default “sun”.

• switch_time (Union[str, pd.Timestamp]) – Local time of switch between stability conditions. Overrides <method>. Default None.

• location (str) – Location of data collection. Default empty string.

Returns:

Interpolated data with additional columns for Obukhov length \(L_{Ob}\), sensible heat flux \(H\), friction velocity \(u^{*}\), and temperature scale \(\theta^{*}\).

Return type:

pd.DataFrame

match_time_at_threshold(series, threshold, lessthan=True, min_time=None)

Gets time index of first value in series exceeding threshold.

Parameters:

• series (pd.Series) – Time series of numeric variable.

• threshold (float) – Threshold value.

• lessthan (bool) – If True, finds first value less than threshold. Otherwise, returns first value greater than threshold. Default True.

• min_time (pd.Timestamp) – Time indices below this threshold are discarded. Default None.

Returns:

Local time of the first value in the series that is less than the given threshold.

Return type:

pd.Timestamp

plot_derived_metrics(derived_data, time_id, regime=None, location='')

Plot and save derived fluxes.

Parameters:

• derived_data (pd.DataFrame) – Interpolated tz-aware dataframe with column for sensible heat flux under free convection.

• time_id (pd.Timestamp) – Start time of scintillometer data collection.

• regime (str) – Stability condition. Default None.

• location (str) – Location of data collection. Default empty

• string. –

Returns:

Time series comparing sensible heat fluxes under free convection to on-board software.

Return type:

list[tuple[plt.Figure, plt.Axes]]

plot_iterated_metrics(iterated_data, time_stamp, location='')

Plots and saves iterated SHF, comparison to free convection.

Parameters:

• iterated_data (pd.DataFrame) – TZ-aware with columns for sensible heat fluxes calculated for free convection \(H_{free}\), and by MOST \(H\).

• time_stamp (pd.Timestamp) – Local time of data collection.

• location (str) – Location of data collection. Default empty string.

Returns:

Time series of sensible heat flux calculated through MOST, and a comparison to sensible heat flux under free convection.

Return type:

list[tuple[plt.Figure, plt.Axes]]

plot_lapse_rates(vertical_data, dry_adiabat, local_time, bl_height=None, location='')

Plots comparison of lapse rates and boundary layer height.

The figures are saved to disk.

Parameters:

• vertical_data (dict) – Vertical measurements for lapse rates and temperatures.

• dry_adiabat (float) – Dry adiabatic lapse rate.

• local_time (pd.Timestamp) – Local time of switch between stability conditions.

• bl_height (float) – Boundary layer height, [m]. Default None.

• location (str) – Location of data collection. Default empty string.

Returns:

Vertical profiles of lapse rates on a single axis, and vertical profiles of parcel temperatures on a single axis. If a boundary layer height is provided, vertical lines denoting its height are added to the figures.

Return type:

list[tuple[plt.Figure, plt.Axes]]

plot_switch_time_stability(data, local_time, location='', bl_height=None)

Plot and save profiles of potential temperature and gradient.

Parameters:

• data (dict) – Contains dataframes for vertical profiles of potential temperature and optionally the gradient of potential temperature.

• local_time (pd.Timestamp) – Local time of switch between stability conditions.

• location (str) – Location of data collection. Default empty string.

• bl_height (int) – Boundary layer height. Default None.

Returns:

Vertical profile of potential temperature. If the gradient potential temperature is also provided, the two vertical profiles are placed side-by-side in separate subplots.

Return type:

list[tuple[plt.Figure, plt.Axes]]

class scintillometry.metrics.calculations.MetricsTopography

Bases: object

Calculate metrics for topographical data.

transect

Inherited methods for calculating path heights.

Type:

TransectParameters

get_path_height_parameters(transect, regime=None)

Get effective and mean path heights of transect.

Computes effective and mean path heights of transect under stable and unstable conditions, and with no height dependency. Prints the effective and mean path height for the user-selected stability conditions.

Select the stability conditions using --r, --regime <str>.

Parameters:

• transect (pd.DataFrame) – Parsed path transect data.

• regime (str) – Target stability condition. Default None.

Returns:

Tuples of effective and mean path height \(z_{eff}\) and \(\bar{z}\) [m], with stability conditions as keys.

Return type:

dict[str, tuple[np.floating, np.floating]]

class scintillometry.metrics.calculations.MetricsWorkflow

Bases: MetricsFlux, MetricsTopography

Standard workflow for metrics.

calculate_standard_metrics(data, regime=None, most_name='an1988', method='sun', switch_time=None, location='', beam_wavelength=880, beam_error=20, **kwargs)

Calculates and plots metrics from wrangled data.

This wrapper function:

• Calculates effective path heights for all stability conditions.

• Derives \(C_{T}^{2}\) and sensible heat flux for free convection.

• Estimates the time when stability conditions change.

• Calculates sensible heat flux using MOST.

• Plots time series comparing sensible heat flux for free convection \(H_{free}\) to on-board software, time series of sensible heat flux calculated with MOST \(H\), and a comparison to sensible heat flux for free convection.

• Saves plots to disk.

If this function is imported as a package, mock user arguments with an argparse.Namespace object.

Parameters:

• data (dict) –

  Contains BLS, weather, and transect dataframes, an interpolated dataframe at 60s resolution containing merged BLS and weather data, a pd.Timestamp object of the scintillometer’s recorded start time, and optionally vertical measurements:

  data = {
      "bls": bls_data,
      "weather": weather_data,
      "transect": transect_data,
      "interpolated": interpolated_data,
      "timestamp": bls_time,
      "vertical": {
          "temperature": pd.DataFrame,
          "humidity": pd.DataFrame,
          }
      }
  

• regime (str) – Target stability condition. Default None.

• most_name (str) – MOST coefficients for unstable and stable conditions. Default “an1988”.

• method (str) – Method to calculate switch time. Default “sun”.

• switch_time (Union[str, pd.Timestamp]) – Local time of switch between stability conditions. Overrides <method>. Default None.

• location (str) – Location of data collection. Default empty string.

• beam_wavelength (int) – Transmitter beam wavelength, nm. Default 880 nm.

• beam_error (int) – Transmitter beam error, nm. Default 20 nm.

Returns:

Input dictionary with additional keys “derivation”, “iteration” for derived and iterated data, respectively.

Return type:

dict

compare_eddy(own_data, ext_data, source='innflux', location='')

Compares data to an external source of eddy covariance data.

Plots and saves time series comparing Obukhov lengths and sensible heat fluxes between an input dataframe and external eddy covariance measurements.

If this function is imported as a package, mock user arguments with an argparse.Namespace object.

Parameters:

• own_data (pd.DataFrame) – Labelled data.

• ext_data (pd.DataFrame) – Eddy covariance data from an external source.

• source (str) – Data source of vertical measurements. Currently supports processed innFLUX data. Default “innflux”.

• location (str) – Location of data collection. Default empty string.

Returns:

Time series comparing Obukhov length and sensible heat flux to innFLUX measurements.

Return type:

list[tuple[plt.Figure, plt.Axes]]

Raises:

NotImplementedError – <source> measurements are not supported. Use “innflux”.

compare_innflux(own_data, innflux_data, location='')

Compares SHF and Obukhov lengths to innFLUX measurements.

This wrapper function:

• Plots time series comparing Obukhov lengths and sensible heat fluxes between an input dataframe and innFLUX measurements.

• Saves plots to disk.

If this function is imported as a package, mock user arguments with an argparse.Namespace object.

Parameters:

• own_data (pd.DataFrame) – Labelled data for SHF and Obukhov length.

• innflux_data (pd.DataFrame) – Eddy covariance data from innFLUX.

• location (str) – Location of data collection. Default empty string.

Returns:

Time series comparing Obukhov length and sensible heat flux to innFLUX measurements.

Return type:

list[tuple[plt.Figure, plt.Axes]]

References

[1] (1,2)

A. Jeričević, B Grisogono (2006). The Critical Bulk Richardson Number in Urban Areas: Verification and Application in a Numerical Weather Prediction Model. Tellus A: Dynamic Meteorology and Oceanography, Vol. 58:1, [19-27]. DOI: https://doi.org/10.1111/j.1600-0870.2006.00153.x

[2] (1,2)

Y. Zhao, L.W. Chew, A. Kubilay, J. Carmeliet (2020). Isothermal and Non-Isothermal Flow in Street Canyons: a Review From Theoretical, Experimental and Numerical Perspectives. Building and Environment, Vol. 184 [107163] . DOI:https://doi.org/10.1016/j.buildenv.2020.107163

Module contents