Modules

Filter Option 4

class filter_option4.FilterOption4[source]

Main class of filter_option4.py

filter4(param)[source]

Main method of filter_option4.py

Parameters:param (string) – “NmF2”/”hmF2”
exclusion_outliers(dict_IS, maxValue, minValue)[source]

This method returns a dictionary of those values of dict_input, which don’t exceed the minValue-maxValue bound

Parameters:
  • dict_IS (dict) – dictionary of ionosonde data, format dict[“station”][“YYYY-MM-DD hh:mm”] = float
  • maxValue (float) – upper bound
  • minValue (float) – lower bound
Returns:

dictionary of those values of dict_input, which don’t exceed the minValue-maxValue bound, format dict[“station”][“YYYY-MM-DD hh:mm”] = float

Return type:

dict

statistics_daily(dict_IS, statistics)[source]

This method returns a dictionary with statistical values in interval of 15 minutes

Parameters:
  • dict_IS (dict) – dictionary of ionosonde data, format dict[“station”][“YYYY-MM-DD hh:mm”] = float
  • statistics (string) – “std” (standard deviation)/”median”
Returns:

dictionary with statistical values for one day, format dict[“station”][minute_of_day] = float

Return type:

dict

calc_STD_factor_Kp(dict_std, param)[source]

This method calculates a STD factor for all days between the dates specified in config.py and all times in dict_std STD factor is chosen from config.py depending on hourly Kp values and daily F10.7 values read with reader_ionodata.py

Parameters:
  • dict_std (dict) – dictionary of std values (return of statistics_daily(dict_IS, “std”))
  • param (string) – “NmF2”/”hmF2”
Returns:

dictionary with std factors, format dict[“station”][“YYYY-MM-DD hh:mm”] = float

Return type:

dict

calc_bounds(dict_median, dict_std, dict_factor, lowerupper)[source]

This method calculates the bounds from median, std and std factors for all days specified in config.py

Parameters:
  • dict_median (dict) – dictionary of median values (return of statistics_daily(dict_IS, “median”))
  • dict_std (dict) – dictionary of std values (return of statistics_daily(dict_IS, “std”))
  • dict_factor (dict) – dictionary of std factor values (return of calc_STD_factor_Kp(dict_std, param))
  • lowerupper (string) – “lower”/”upper”
Returns:

dictionary with lower or upper bounds, format dict[“station”][“YYYY-MM-DD hh:mm”] = float

Return type:

dict

interpolate_datetime_to_minute(dict_input)[source]

This method returns a dictionary of SciPy interpolation objects for each station

Parameters:dict_input (dict) – dictionary, format dict[“station”][“YYYY-MM-DD hh:mm”] = float
Returns:dictionary, format dict[“station”] = SciPy interpolation function with minute_of_year as input parameter
Return type:dict
datetime_to_moy(value_datetime)[source]

This method converts datetime string (“2014-12-31 23:59”) to minute of year

Parameters:value_datetime (string) – datetime string, format “YYYY-MM-DD hh:mm”
Returns:minute_of_year
Return type:int
output_outliers(data, dict_interp_lower, dict_interp_upper, param)[source]

This method creates a file containing all outliers

Parameters:
  • data (dict) – dictionary of ionosonde data, format dict[“station”][“YYYY-MM-DD hh:mm”] = float
  • dict_interp_lower (dict) – dictionary of lower bounds, format dict[“station”] = SciPy interpolation function
  • dict_interp_upper (dict) – dictionary of upper bounds, format dict[“station”] = SciPy interpolation function
  • param (string) – “NmF2”/”hmF2”
plot_data_median_bounds(dict_data, dict_median, dict_interp_lower, dict_interp_upper, param)[source]

This method creates png plots of hmF2/NmF2 values, median, lower and upper bounds

Parameters:
  • dict_data (dict) – dictionary of ionosonde data, format dict[“station”][“YYYY-MM-DD hh:mm”] = float
  • dict_median (dict) – dictionary of ionosonde data, format dict[“station”][“YYYY-MM-DD hh:mm”] = float
  • dict_interp_lower (dict) – dictionary of lower bounds, format dict[“station”] = SciPy interpolation function
  • dict_interp_upper (dict) – dictionary of upper bounds, format dict[“station”] = SciPy interpolation function
  • param (string) – “NmF2”/”hmF2”

Filter Scikit

class filter_scikit.FilterScikit[source]

Main class of filter_scikit.py

exclusion_outliers(dict_IS, maxValue, minValue)[source]

This method returns a dictionary of those values of dict_input, which don’t exceed the minValue-maxValue bound

Parameters:
  • dict_IS (dict) – dictionary of ionosonde data, format dict[“station”][“YYYY-MM-DD hh:mm”] = float
  • maxValue (float) – upper bound
  • minValue (float) – lower bound
Returns:

dictionary of those values of dict_input, which don’t exceed the minValue-maxValue bound, format dict[“station”][“YYYY-MM-DD hh:mm”] = float

Return type:

dict

fit_dict(dict_IS)[source]

This method reshapes dictionaries to work with calc_filter_mcd_simple() and calc_filter_mcd_correlation()

Parameters:dict_IS (dict) – dictionary of ionosonde data, format dict[“station”][“YYYY-MM-DD hh:mm”] = float
Returns:dictionary of ionosonde data, format dict[“station”][minute_of_day (interval of 15 minutes)] = [“YYYY-MM-DD hh:mm”, float]
Return type:dict
calc_filter_mcd_simple(dict_fit)[source]

This method filters ionosonde data with sklearn.covariance, using only one ionosonde parameter

Parameters:dict_fit (dict) – dictionary of ionosonde data, format dict[“station”][minute_of_day (interval of 15 minutes)] = [“YYYY-MM-DD hh:mm”, float] (output of fit_dict())
Returns:dictionary containing ionosonde data and boolean value (“Is this value an outlier?”), format dict[“station”][“YYYY-MM-DD hh:mm”] = [float, bool_outlier]
Return type:dict
calc_filter_mcd_correlation(NmF2_fit, hmF2_fit)[source]

This method filters ionosonde data with sklearn.covariance, using both ionosonde parameters (NmF2 and hmF2)

Parameters:
  • NmF2_fit (dict) – dictionary of NmF2 data, format dict[“station”][minute_of_day (interval of 15 minutes)] = [“YYYY-MM-DD hh:mm”, float] (output of fit_dict())
  • hmF2_fit (dict) – dictionary of hmF2 data, format dict[“station”][minute_of_day (interval of 15 minutes)] = [“YYYY-MM-DD hh:mm”, float] (output of fit_dict())
Returns:

dictionary containing NmF2 and hmF2 and boolean value (“Is this value an outlier?”), format dict[“station”][“YYYY-MM-DD hh:mm”] = [float, float, bool_outlier]

Return type:

dict

intersect(data_NmF2, data_hmF2, log_message)[source]

This method returns the intersection of two objects and logs those elements, which are not in the intersection, meaning that they are no longer used

Parameters:
  • data_NmF2 (iterable) – iterable object containing NmF2 data
  • data_hmF2 (iterable) – iterable object containing hmF2 data
  • log_message (string) – additional text to output in the log file
Returns:

set of the intersection of data_NmF2 and data_hmF2

Return type:

set

reduce_dict_param(dict_filter_NmF2_hmF2, param)[source]

This method extract NmF2/hmF2 values from the dictionary returned with calc_filter_mcd_correlation()

Parameters:
  • dict_filter_NmF2_hmF2 (dict) – dictionary containing NmF2, hmF2 and boolean values (return of calc_filter_mcd_correlation())
  • param (string) – “NmF2”/”hmF2”
Returns:

dictionary containing NmF2 or hmF2 values and boolean values, format dict[“station”][“YYYY-MM-DD hh:mm”] = [float, bool_outlier]

Return type:

dict

output_outliers(dict_filter, param)[source]

This method creates a file containing all outliers

Parameters:
plot_data_outliers(dict_filter, param)[source]

This method creates png plots of hmF2/NmF2 values and outliers

Parameters:

Reader IonoData

class reader_ionodata.Reader_IonoData[source]

Main class of reader_ionodata.py

read_iono_data(param)[source]

This method creates a dictionary of the files in config.path_input and returns it

The files should be in the format used with the DownloadClient:

# URSIC Station Name
# Lat:0.0 Lon:0.0
YYYY-MM-DD hh:mm value
Parameters:param (string) – ionospheric parameter
Returns:dictionary of ionosonde data, format dict[“station”][“YYYY-MM-DD hh:mm”] = float
Return type:dict
convert_fof2_nmf2()[source]

This method reads foF2 files (using read_iono_data()), converts all values to NmF2 * 10^-12 and returns a dictionary of NmF2 data

Returns:dictionary of NmF2 data, format dict[“station”][“YYYY-MM-DD hh:mm”] = float
Return type:dict
read_F10p7()[source]

This method returns a dictionary of F10.7 values read from a file with the format:

YYYY-MM-DD hh:mm value
Returns:dictionary of F10.7 data, format dict[“YYYY-MM-DD”] = float
Return type:dict
read_Kp()[source]

This method returns a dictionary of Kp values read from a file with the format:

YYYY-MM-DD hh:mm value
Returns:dictionary of Kp data, format dict[“YYYY-MM-DD hh:mm”] = float
Return type:dict
read_Kp_interp()[source]

This method reads a Kp file and returns a SciPy interpolation function: interp(hour_of_year) = interpolated Kp value

Returns:SciPy interpolation function to interpolate Kp values
Return type:SciPy interpolation function

Table Of Contents

Previous topic

Starting IonoDataFilter

This Page