Chapter 2 Data structures and main functions
2.1 Meteorological variables
Package meteoland assists in the estimation of the following daily variables over lanscapes (units in parentheses):
DOY: Day of the year ([1-366]).
MeanTemperature: Mean daily temperature (in degrees Celsius).
MinTemperature: Minimum daily temperature (in degrees Celsius).
MaxTemperature: Maximum daily temperature (in degrees Celsius).
Precipitation: Daily precipitation (in mm of water).
MeanRelativeHumidity: Mean daily relative humidity (in percent).
MinRelativeHumidity: Minimum daily relative humidity (in percent).
MaxRelativeHumidity: Maximum daily relative humidity (in percent).
Radiation: Incoming radiation (in MJ/m2).
WindSpeed: Wind speed (in m/s).
WindDirection: Wind direction (in degrees from North).
PET: Potential evapo-transpiration (in mm of water).
Since specific humidity is often specified instead of relative humidity, the package also allows reading and using this variable, although it will not be written in files:
SpecificHumidity: Specific humidity (in kg/kg).
2.2 Spatial classes
The package deals with three kinds of spatial structures: individual points, a set of pixels from a spatial grid and full (i.e. complete) grids. The package includes six S4 spatial classes, which are defined as children of classes in package sp. Spatial structures are used to represent both topographical and meteorological data. These are described in the following subsections.
Three classes are defined to represent the variation of topographic features (i.e., elevation, slope and aspect) over space:
SpatialPointsDataFrameand represents the topographic features of a set of points in a landscape.
## Class "SpatialPointsTopography" [package "meteoland"] ## ## Slots: ## ## Name: data coords.nrs coords bbox proj4string ## Class: data.frame numeric matrix matrix CRS ## ## Extends: ## Class "SpatialPointsDataFrame", directly ## Class "SpatialPoints", by class "SpatialPointsDataFrame", distance 2 ## Class "Spatial", by class "SpatialPointsDataFrame", distance 3 ## Class "SpatialVector", by class "SpatialPointsDataFrame", distance 3
SpatialGridDataFrameand represents the continuous variation of topographic features over a full spatial grid.
## Class "SpatialGridTopography" [package "meteoland"] ## ## Slots: ## ## Name: data grid bbox proj4string ## Class: data.frame GridTopology matrix CRS ## ## Extends: ## Class "SpatialGridDataFrame", directly ## Class "SpatialGrid", by class "SpatialGridDataFrame", distance 2 ## Class "Spatial", by class "SpatialGridDataFrame", distance 3
SpatialPixelsDataFrameand represents the continuous variation of topographic features over a set if cells in a grid.
## Class "SpatialPixelsTopography" [package "meteoland"] ## ## Slots: ## ## Name: data coords.nrs grid grid.index coords bbox proj4string ## Class: data.frame numeric GridTopology integer matrix matrix CRS ## ## Extends: ## Class "SpatialPixelsDataFrame", directly ## Class "SpatialPixels", by class "SpatialPixelsDataFrame", distance 2 ## Class "SpatialPointsDataFrame", by class "SpatialPixelsDataFrame", distance 2 ## Class "SpatialPoints", by class "SpatialPixelsDataFrame", distance 3 ## Class "Spatial", by class "SpatialPixelsDataFrame", distance 4 ## Class "SpatialVector", by class "SpatialPixelsDataFrame", distance 4
Although the three classes have the same slots as their parent S4 classes, data frames in
SpatialPixelsTopography objects have only three variables:
elevation (in meters),
slope (in degrees from the horizontal plane) and
aspect (in degrees from North).
Analogously to topography classes, three spatial classes are used to represent the variation of daily meteorology over space:
SpatialPointsand represents daily meteorology series for a set of points in a landscape.
## Class "SpatialPointsMeteorology" [package "meteoland"] ## ## Slots: ## ## Name: dates data coords bbox proj4string ## Class: Date vector matrix matrix CRS ## ## Extends: ## Class "SpatialPoints", directly ## Class "Spatial", by class "SpatialPoints", distance 2 ## Class "SpatialVector", by class "SpatialPoints", distance 2
SpatialGridand represents the continuous variation of daily meteorology across a grid of cells.
## Class "SpatialGridMeteorology" [package "meteoland"] ## ## Slots: ## ## Name: dates data grid bbox proj4string ## Class: Date vector GridTopology matrix CRS ## ## Extends: ## Class "SpatialGrid", directly ## Class "Spatial", by class "SpatialGrid", distance 2
SpatialPixelsand represents the variation of daily meteorology for a set of pixels (cells) of a spatial grid.
## Class "SpatialPixelsMeteorology" [package "meteoland"] ## ## Slots: ## ## Name: dates data grid grid.index coords bbox proj4string ## Class: Date vector GridTopology integer matrix matrix CRS ## ## Extends: ## Class "SpatialPixels", directly ## Class "SpatialPoints", by class "SpatialPixels", distance 2 ## Class "Spatial", by class "SpatialPixels", distance 3 ## Class "SpatialVector", by class "SpatialPixels", distance 3
In addition to their corresponding inherited slots, classes
SpatialPixelsMeteorology have two additional slots:
dates (a vector of days specifying a time period), and
data (a vector of data frames with the meteorological data). Although the three classes have a
data slot containing data frames, meteorological data is in different form in each class. In objects of
SpatialPointsMeteorology, there is one data frame for each point where variables are in columns and dates are in rows. In objects of
SpatialPixelsMeteorology, each data frame describes the meteorology over a complete grid, or a subset of cells, for a single day. In these cases, the data frame has grid cells in rows and variables in columns.
2.3 Reading and writing meteorological data
2.3.1 Point meteorology (ascii/rds files)
Objects of class
SpatialPointsMeteorology are stored in the disk using one data file for each of their spatial points. Files can be stored in ascii (i.e. text) format or rds (i.e. R data of a single object compressed) format. Package meteoland provides four input/output functions for point meteorology:
readmeteorologypoint()reads the meteorological data stored in one ascii/rds data file and returns a data frame.
writemeteorologypoint()writes the meteorological data of a single point as an ascii/rds file in the file system.
readmeteorologypointfiles()reads several ascii/rds files and returns an object of class
writemeteorologypointfiles()writes several ascii/rds files in the disk, one per spatial point. Metadata (i.e. the spatial coordinates of each point and the corresponding file path) is stored in an additional file.
2.3.2 Point/gridded meteorology (netCDF)
NetCDF is a set of libraries to write machine-independent data formats that support the creation and sharing of array-oriented scientific data. NetCDF is very useful useful to store spatio-temporal datasets, as it allows storing the grid topology, the time period being described and the spatial projection as well as the actual data all in the same file. Moreover, NetCDF can be read and written lazily, without having to load all the file in memory.
Package meteoland allows objects of classes
SpatialGridMeteorology to be stored in the disk as NetCDF thanks to the functions provided by ncdf4 R package, which provides an interface to Unidata’s netCDF library (version 4 or earlier). We try to conform to CF specifications in files written by meteoland, but improvements can always be made.
The following functions are available for input/output of point meteorology:
writemeteorologypoints()writes meteorological point data on a NetCDF.
readmeteorologypoints()reads point meteorological data stored in one or several NetCDFs and returns an object of class
SpatialPointsMeteorology. If several files are read, the function tries to merge the result.
And the following functions are available for input/output of meteorology on pixels or full grids:
readmeteorologypixels()read the meteorological data stored in one or several NetCDF file and return an object of class
SpatialPixelsMeteorology, respectively. If several files are read, the function tries to merge the result. Rotated grids should not be read using this functions.
readmeteorologygridpoints()allows reading grid pixels as points. It returns an object of class
SpatialPointsMeteorology. This feature becomes useful to read data from rotated grids.
writemeteorologypixels()write the meteorological data of the full grid or the subset of grid cells, for a NetCDF. The same functions allow adding and replacing data content in a previously existing NetCDF.
writemeteorologygridpixel()allows writing/replacing data for specific pixels in an existing NetCDF.
2.4 Visualizing input topography and meteorological data
Although very simple, the package provides two kinds of functions to visualize the temporal and spatial variation of meteorology:
spplot()has been redefined from package sp to draw maps of specific weather variables corresponding to specific dates. The function can be used on objects of class
meteoplot()allows the temporal series of specific variables on specific spatial points to be plotted. The function can read the data from the disk or from objects of class
spplot() also accepts objects of classes
SpatialPixelsTopography, so that topography can also be easily displayed.
2.5 Subsetting, merging and reshaping data
A number of functions are available to manipulate data structures and to make them available for other R packages.
2.5.1 Subsetting and reshaping topography data
Following the design of package sp, package meteoland provides several functions to manipulate topographic data structures (see fig. 2.1).
- It is possible to coerce objects between from topography classes to the corresponding class in package sp using functions
as(). This includes coercing to spatial objects with or without data columns. Additionally, an object of
SpatialPointsTopographycan be coerced into a
SpatialPixelsTopographyusing a call like
- It is possible to coerce objects to the general-purpose class
sfof package sf using
- Subsetting (i.e.
[operator) is allowed on all three spatial topographic classes. The valid arguments of this function call depend on the kind of structure.
2.5.2 Subsetting and reshaping meteorology data
Analogously, package meteoland provides several functions to manipulate spatio-temporal data structures (see fig. 2.2).
- It is possible to coerce objects between spatial classes using functions
- Subsetting (i.e.
[operator) is allowed on all six spatial classes (topographic or meteorological). The valid arguments of this function call depend on the kind of structure.
- The package provides functions to extract meteorological data (these allow reading data in memory or from the disk):
extractgridindex()extracts the meteorology of a particular pixel index from a grid and returns a data frame with dates in rows and variables in columns.
extractgridpoints()extract the meteorology of point locations from a grid and returns an object of
SpatialPointsMeteorology, similarly to the
extractvars()extract the meteorology of a set of dates (or variables, respectively) from a
SpatialPixelsMeteorology) and returns a
SpatialPixelsDataFrame) for each date (resp. variable) in a named list. This kind of conversion is also useful to export meteoland outputs to package raster, which provides wrappers for the abovementioned sp classes.
- Finally, the package provides some functions to reshape meteorological data structures into more general structures of spatio-temporal data, so that they can be more easily processed, combined and stored using other R packages. In particular, objects of classes
SpatialPixelsMeteorologycan be reshaped into objects of packages stars and spacetime by using calls like
2.5.3 Merging meteorology data
mergepoints() take any number of objects of the same spatial class and return a merged object. The functions pool over dates and variables, but the objects to merge should have the same spatial structures (points, grids, reference systems, …). This is specially useful to merge data corresponding to different periods.
2.6 Summarizing meteorological data
2.6.1 Temporal summaries
The package provides functions to generate temporal summaries of meteorological data. These accept meteorology objects as input and return their corresponding spatial dataframe structure with the summary statistics in columns:
summarypoints()summarizes the meteorology of spatial points. It accepts objects of
SpatialPointsMeteorologyas input and returns an object of
SpatialPointsDataFramewith point summaries for the requested variable. Temporal summaries can be calculated for different periods and using different summarizing functions (e.g. mean, sum, minimum, maximum, etc.).
summarypixels()summarize the meteorology of full grids or of subset of grid cells, respectively. They accept objects of
SpatialPixelsMeteorology, respectively, as input and return an object of
SpatialPixelsDataFrame, respectively, with temporal summaries for the requested variable over the range of dates indicated.
summaryinterpolationdata()works similarly to
summarypoints(), but takes an object of class
All temporal summary functions except
summaryinterpolationdata() accept a data file, or a metadata file as input. This allows, producing summaries lazily, without loading complete data structures in memory. Of course, this mode of producing summaries may be slower than operating on objects already loaded in memory.
2.6.2 Spatial summaries
The package provides one function to produce spatial summaries. Function
averagearea() averages the coordinates and meteorological values of any spatial meteorology object, returning an object of class
SpatialPointMeteorology with a single point representing the average.
2.7 Meteorology estimation functions
2.7.1 Spatial interpolation
Package meteoland provides three functions for interpolating meteorological data (i.e., one for each data structure):
interpolationpoints()interpolates weather for a set of locations given in
SpatialPointsTopographyand returns an object of class
interpolationpixels()interpolates weather for pixels in a grid specified in
SpatialPixelsTopographyand returns an object of class
interpolationgrid()interpolates weather for a whole grid specified in
SpatialGridTopographyand returns an object of class
Both functions require an object of class
MeteorologyInterpolationData, which contains the X-Y coordinates, the meteorological data and topography of a set of weather stations as well as weather interpolation parameters.
## Class "MeteorologyInterpolationData" [package "meteoland"] ## ## Slots: ## ## Name: coords elevation slope aspect ## Class: matrix numeric numeric numeric ## ## Name: MinTemperature MaxTemperature SmoothedPrecipitation Precipitation ## Class: matrix matrix matrix matrix ## ## Name: SmoothedTemperatureRange RelativeHumidity Radiation WindSpeed ## Class: matrix matrix ANY ANY ## ## Name: WindDirection WindFields WFIndex WFFactor ## Class: ANY ANY ANY ANY ## ## Name: params dates bbox proj4string ## Class: list Date matrix CRS ## ## Extends: ## Class "MeteorologyProcedureData", directly ## Class "Spatial", by class "MeteorologyProcedureData", distance 2
When calling functions
interpolationgrid(), the user may require interpolation outputs to be written into the file system, instead of being stored in memory. For example, if
interpolationpoints() is called with
export = TRUE and a location directory, the function will write the data frame produced for each point into an ascii text file or a rds file. Metadata files will also be written, so that results can later be loaded in memory. Alternatively,
interpolationpoints() can be forced to write results in a netCDF by specifying
export = TRUE and
exportFile = <filename>. Similarly, if
interpolationgrid() are called specifying
exportFile = <filename>, the functions will create and add data to a netCDF. These options becomes important when dealing with very large spatial structures.
interpolation.cv() are included in meteoland to calibrate interpolation parameters and evaluate predictive performance of interpolation routines before using them. Details of interpolation routines are described in chapter 3.
2.7.2 Statistical correction
One function is available for statistical correction of meteorological data series (i.e., one function for each data structure). Function
correctionpoints() performs statistical correction of weather data series on a set of locations and it returns an object of class
SpatialPointsMeteorology containing corrected weather predictions. Statistical correction requires an object of class
MeteorologyUncorrectedData, which contains the X-Y coordinates and the coarse-scale meteorological data to be corrected, which includes a reference (historic) period and projected (e.g. future) period:
## Class "MeteorologyUncorrectedData" [package "meteoland"] ## ## Slots: ## ## Name: coords reference_data projection_data params dates bbox ## Class: matrix ANY ANY list Date matrix ## ## Name: proj4string ## Class: CRS ## ## Extends: ## Class "MeteorologyProcedureData", directly ## Class "Spatial", by class "MeteorologyProcedureData", distance 2
The reference (historical) period is compared with observed meteorological data of the same period, and the routine uses this information to correct the projected (e.g. future) period. Therefore, apart from the
MeteorologyUncorrectedData object, the correction function requires accurate meteorological data (for a set of spatial points or a grid). Normally, these data will be the result of spatial interpolation.
As before, when calling functions
correctionpoints(), the user may require the outputs to be written into the file system, instead of being returned in memory. The options are the same as described for interpolation.
correctionpoints.errors() was included in the package to evaluate the errors of the less accurate and more accurate series. Comparisons can be made before and after applying statistical corrections. In the latter case, cross-validation is also available. Details of correction routines are described in chapter 5.
2.7.3 Weather generation
weathergeneration() can be used to generate synthetic weather series for a range of inputs, including a (non-spatial) data frame (corresponding to weather series of a single location) as well as objects of classes
SpatialGridDataFrame. The output of the function is of the same class as the input object. Weather generation algorithms are described in chapter 7.