[gradsusr] GrADS query: creating a control file to handle multiple netcdf files when each netcdf file has a uniquely different scale_factor and add_offset

[Vizy, Edward K]

Dear gradusers,

Had a query that I was looking for help with.  I've been downloading upper-level pressure ERA5 data from Copernicus Climate Change Service (C3S) in NETCDF format.  Since the data volume is too large, I've had to break the downloads of the hourly data into monthly increments. Thus, I have files:

ecmwf.era5.anal.hourly.pl.u.201407.nc   ------ > containing July 2014 1-hourly output
ecmwf.era5.anal.hourly.pl.u.201408.nc   ------ > containing August 2014 1-hourly output
ecmwf.era5.anal.hourly.pl.u.201409.nc   ------ > containing September 2014 1-hourly output

Of course wanting to open this data up in GrADS using a single GrADS control file to evaluate time series, I generated the GrADS control file shown below:
_____________________________________________

dset ^./ecmwf.era5.anal.hourly.pl.u.%y4%m2.nc
options yrev zrev template
title
undef -9.99e+33
unpack scale_factor add_offset
dtype netcdf
xdef 1440 linear 0 0.25
ydef 721 linear -90 0.25
zdef 19 levels 1000 975 950 925 900 875 850 825 800 775 750 700 650 600 500 400 300 200 100
tdef 2208 linear 00Z01JUL2014 60mn
vars 1
u=>u  19  t,z,y,x  U component of wind (m/s)
endvars
_____________________________________________

At first glance this control file appeared to have worked.  However, once I started quality controlling by comparing with the directly opened NETCDF files there were differences in the output for the second and third months.  After some digging around, I realized that the C3S service from ERA5 does not implement standard values for the scale_factor and add_offset when processed into NETCDF form.  For example, using ncdump to examine the NETCDF files individually, I find:

NETCDF File:  ecmwf.era5.anal.hourly.pl.u.201407.nc
        short u(time, level, latitude, longitude) ;
                u:scale_factor = 0.00256569367738034 ;
                u:add_offset = 25.6669300437863 ;
                u:_FillValue = -32767s ;
                u:missing_value = -32767s ;
                u:units = "m s**-1" ;
                u:long_name = "U component of wind" ;
                u:standard_name = "eastward_wind" ;

NETCDF File: ecmwf.era5.anal.hourly.pl.u.201408.nc
        short u(time, level, latitude, longitude) ;
                u:scale_factor = 0.00344970129404031 ;
                u:add_offset = 30.5973199491577 ;
                u:_FillValue = -32767s ;
                u:missing_value = -32767s ;
                u:units = "m s**-1" ;
                u:long_name = "U component of wind" ;
                u:standard_name = "eastward_wind" ;

NETCDF File: ecmwf.era5.anal.hourly.pl.u.201409.nc
        short u(time, level, latitude, longitude) ;
                u:scale_factor = 0.002590087296414 ;
                u:add_offset = 21.8817387087932 ;
                u:_FillValue = -32767s ;
                u:missing_value = -32767s ;
                u:units = "m s**-1" ;
                u:long_name = "U component of wind" ;
                u:standard_name = "eastward_wind" ;

Essentially, when I open my GrADS control file up, it is not reading these values for scale_factor and add_offset from each of the NETCDF files when time is changed into another month, only the first data file at the initial time (i.e., from file ecmwf.era5.anal.hourly.pl.u.201407.nc).

Does anyone know of a way to account for this change with additional setting/manipulation in the GrADS control file, or will I need to reprocess the data once downloaded from C3S to standardize it to a single convention?  BTW… I’m using GrADS Version 2.1.a2.

Thanks for your assistance.

Best regards,
Ned Vizy

______________________________________
Dr. Edward "Ned" Vizy
Research Scientist Associate V
Department of Geological Sciences
Jackson School of Geosciences
The University of Texas at Austin
Austin, TX 78712
______________________________________

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gradsusr.org/pipermail/gradsusr/attachments/20180607/de3b6162/attachment-0001.html>