[gradsusr] [EXTERNAL] New subsetting of Merra-2 data

Thu Mar 26 15:54:07 EDT 2020

Antoine –
I tried one of your modified URLs and it worked for me. What version of wget are you using? We recommend 1.18 or later.
--Jennifer

--
Jennifer Miletta Adams
ADNET Systems, Inc.
NASA/GSFC, Code 610.2
Building 32, Room S159
(301) 614-6070

From: gradsusr <gradsusr-bounces at gradsusr.org> on behalf of Antoine Molin <amolin at plenr.fr>
Reply-To: GrADS Users Forum <gradsusr at gradsusr.org>
Date: Thursday, March 26, 2020 at 12:12 PM
To: 'GrADS Users Forum' <gradsusr at gradsusr.org>
Subject: Re: [gradsusr] [EXTERNAL] New subsetting of Merra-2 data

Hi Jennifer,

Ok, i changed also ..F01%..  in ..F02%..  but still an error, see enclosed file…   not easy !

Best regards.

Antoine Molin
Etudes de vent PlenR
Tel : 09 67 06 77 23

De : gradsusr [mailto:gradsusr-bounces at gradsusr.org] De la part de Adams, Jennifer M. (GSFC-610.2)[ADNET SYSTEMS INC]
Envoyé : jeudi 26 mars 2020 15:55
À : GrADS Users Forum
Objet : Re: [gradsusr] [EXTERNAL] New subsetting of Merra-2 data

Hi, Antoine –
Oops, my mistake -- there are several places in the URL where you need to change the date:

https://goldsmr4.gesdisc.eosdis.nasa.gov/daac-bin/OTF/HTTP_services.cgi?FILENAME=%2Fdata%2FMERRA2%2FM2T1NXSLV.5.12.4%2F2020%2F01%2FMERRA2_400.tavg1_2d_slv_Nx.20200101.nc4&FORMAT=bmM0Lw&BBOX=42.2%2C-5.5%2C51.2%2C8.2&TIME=2020-01-01T00%3A30%3A00%2F2020-01-01T23%3A30%3A59&LABEL=MERRA2_400.tavg1_2d_slv_Nx.20200101.SUB.nc&SHORTNAME=M2T1NXSLV&SERVICE=SUBSET_MERRA2&VERSION=1.02&DATASET_VERSION=5.12.4&VARIABLES=SLP%2CT10M%2CU50M%2CV50M

https://goldsmr4.gesdisc.eosdis.nasa.gov/daac-bin/OTF/HTTP_services.cgi?FILENAME=%2Fdata%2FMERRA2%2FM2T1NXSLV.5.12.4%2F2020%2F02%2FMERRA2_400.tavg1_2d_slv_Nx.20200201.nc4&FORMAT=bmM0Lw&BBOX=42.2%2C-5.5%2C51.2%2C8.2&TIME=2020-02-01T00%3A30%3A00%2F2020-02-01T23%3A30%3A59&LABEL=MERRA2_400.tavg1_2d_slv_Nx.20200201.SUB.nc&SHORTNAME=M2T1NXSLV&SERVICE=SUBSET_MERRA2&VERSION=1.02&DATASET_VERSION=5.12.4&VARIABLES=SLP%2CT10M%2CU50M%2CV50M

--
Jennifer Miletta Adams
ADNET Systems, Inc.
NASA/GSFC, Code 610.2
Building 32, Room S159
(301) 614-6070

From: gradsusr <gradsusr-bounces at gradsusr.org> on behalf of Antoine Molin <amolin at plenr.fr>
Reply-To: GrADS Users Forum <gradsusr at gradsusr.org>
Date: Thursday, March 26, 2020 at 10:23 AM
To: 'GrADS Users Forum' <gradsusr at gradsusr.org>
Subject: Re: [gradsusr] [EXTERNAL] New subsetting of Merra-2 data

Hi Jennifer,

Good idea, but i got the same error with wget, using the changed january to february enclosed file.

Wget problem.. ?

Antoine Molin
Etudes de vent PlenR
Tel : 09 67 06 77 23

De : gradsusr [mailto:gradsusr-bounces at gradsusr.org] De la part de Adams, Jennifer M. (GSFC-610.2)[ADNET SYSTEMS INC]
Envoyé : jeudi 26 mars 2020 14:25
À : GrADS Users Forum
Objet : Re: [gradsusr] [EXTERNAL] New subsetting of Merra-2 data

Hi, Antoine --
I have opened a new ticket with our Help Desk to resolve your issue. I will find out what changed so that your requests are going to opendap (again). I have also asked about the failure of your wget command. One thing you could try in the meanwhile is to edit the file from January (the one with the HTTP_services URLs) and just change the dates (202001 --> 202002) so you get the February data instead.
–Jennifer

--
Jennifer Miletta Adams
ADNET Systems, Inc.
NASA/GSFC, Code 610.2
Building 32, Room S159
(301) 614-6070

From: gradsusr <gradsusr-bounces at gradsusr.org> on behalf of Antoine Molin <amolin at plenr.fr>
Reply-To: GrADS Users Forum <gradsusr at gradsusr.org>
Date: Thursday, March 26, 2020 at 7:47 AM
To: 'GrADS Users Forum' <gradsusr at gradsusr.org>
Cc: "jjvinat at plenr.fr" <jjvinat at plenr.fr>
Subject: Re: [gradsusr] [EXTERNAL] New subsetting of Merra-2 data

Hi Jennifer,

Well, your method (last email below) doesn’t work anymore, and it seems that the Nasa changed again its site…

Using this method, i got for the february data the enclosed file « subset_M2T1NXSLV_5.12.4_20200326_101407.txt »,  different from the file i had for the january data (see subset_jan2020.txt) ; and i can’t get the data with wget anymore (see « Merra-2_wget-error.docx »).
The same with the enclosed file « subset_M2T1NXSLV_5.12.4_20200326_110522.txt » , where i didn’t check the « Use selected region to crop files » in « Refine region ».

Do you see what could be done ?

Thanks a lot, and best regards.

Antoine Molin
Etudes de vent PlenR
Tel : 09 67 06 77 23

De : gradsusr [mailto:gradsusr-bounces at gradsusr.org] De la part de Adams, Jennifer M. (GSFC-610.2)[ADNET SYSTEMS INC]
Envoyé : mardi 21 janvier 2020 20:17
À : GrADS Users Forum
Cc : gsfc-help-disc at lists.nasa.gov
Objet : Re: [gradsusr] [EXTERNAL] New subsetting of Merra-2 data

Hi, Antoine –
The GES DISC help desk has opened a ticket to help you resolve the issue your having using wget to download a list of URLs.

The problem of the new *.nc4.nc4 files being sufficiently different from the old *.SUB.nc files so that templating breaks … that is a different problem. I have an idea that may fix it, but it will require you to get a new list of URLs for the more recent MERRA-2 data subsets.

From the MERRA-2 dataset landing page, https://disc.gsfc.nasa.gov/datasets/M2T1NXSLV_5.12.4/summary?keywords=M2T1NXSLV.5.12.4, click on the “Subset / Get Data” link at the bottom of the Data Access options in the box on the right hand side of the page. That will pop open a dialog box for you to enter the constraints of your subset. Use the “Refine Date Range” accordion to specify the desired date range. Use the “Spatial Subset” and “Variables” accordions to choose your lat/lon domain and variable list. Then open the “Time of Day” accordion and select all 24 hours, from 00:30 through 23:30. Leave the “Grid” accordion with the default value of ‘None’, and change the “File Format” to ‘netCDF’.

The URLs that you get when you click on the green “Get Data” button should have “servername/daac-bin/OTF/HTTP_services” in the URL instead of “servername/opendap/”. Adding the Time of Day constraint to your subset and selecting all times so you get every time step is what redirects your request away from opendap to HTTP_services and should give you files that (a) have the familiar *SUB.nc naming convention, and (b) a structure that is compatible with the older subsets you have already downloaded so that templating will work properly.

Your URLs look something like this:

https://goldsmr4.gesdisc.eosdis.nasa.gov/daac-bin/OTF/HTTP_services.cgi?FILENAME=%2Fdata%2FMERRA2%2FM2T1NXSLV.5.12.4%2F2019%2F12%2FMERRA2_400.tavg1_2d_slv_Nx.20191231.nc4&FORMAT=bmM0Lw&BBOX=42.2%2C-5.5%2C51.2%2C8.2&TIME=2019-12-31T00%3A30%3A00%2F2019-12-31T23%3A30%3A59&LABEL=MERRA2_400.tavg1_2d_slv_Nx.20191231.SUB.nc&SHORTNAME=M2T1NXSLV&SERVICE=SUBSET_MERRA2&VERSION=1.02&DATASET_VERSION=5.12.4&VARIABLES=SLP%2CT10M%2CU50M%2CV50M

…instead of this:

https://goldsmr4.gesdisc.eosdis.nasa.gov/opendap/MERRA2/M2T1NXSLV.5.12.4/2019/12/MERRA2_400.tavg1_2d_slv_Nx.20191231.nc4.nc4?SLP[0:23][264:282][279:301],T10M[0:23][264:282][279:301],U50M[0:23][264:282][279:301],V50M[0:23][264:282][279:301],time,lat[264:282],lon[279:301]

I hope this explanation is clear and helps to resolve the incompatibility issues you’re having with the newer MERRA-2 subsets. Please let us know how it goes.
--Jennifer

--
Jennifer Miletta Adams
ADNET Systems, Inc.
NASA/GSFC, Code 610.2
Building 32, Room S159
(301) 614-6070

From: gradsusr <gradsusr-bounces at gradsusr.org> on behalf of Antoine Molin <amolin at plenr.fr>
Reply-To: GrADS Users Forum <gradsusr at gradsusr.org>
Date: Tuesday, January 21, 2020 at 11:02 AM
To: "gradsusr at gradsusr.org" <gradsusr at gradsusr.org>
Subject: [EXTERNAL] [gradsusr] New subsetting of Merra-2 data

Hi,

Nasa changed the way to get subsets of Merra-2 data, and that is a problem for me….

  *   First, i can’t succeed in getting one month of daily files with wget, under Windows, after downloading the links of these daily files…

  *   Then, these new files are *.nc4.nc4 files ; i was used to *.SUB.nc4 files, and wanted to mix these 2 kinds of  files ; but for a same subset, the file sizes are different !
  i got same names (using rename *.nc4  *. «  »   and   rename *.nc4  *.SUB.nc4), and can read these new files using xdfopen  (warning about the cachesize, but it is working) ;   but it is not possible to read mix files (using template in .ctl) with xdfopen .

Has anybody a solution ?

Thanks !

Antoine Molin
Etudes de vent PlenR
Tel : 09 67 06 77 23

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gradsusr.org/pipermail/gradsusr/attachments/20200326/2f4f1a21/attachment-0001.html>