The eSWua RESTful Web service is based on the PHP-CRUD-API project (https://github.com/mevdschee/php-crud-api).
DATA FROMAT: The data are exposed in the JSON interchange format.
DATA POLICY: All the data available through the different services are distributed under the Creative Commons Attribution 4.0 International (CC BY 4.0). Please refer to the "collection info and data policy" page for more information about the eSWua data collections. Always remeber to cite the data with the provided DOI.
DATA DOWNLOAD LIMIT: The downloads are limited to a certain number of records at once. The maximum timespan for a single query depends on the specific data-colletion and on the number of the fields/parameters included. When a query requires more than 1 minutes the web-service will stop the execution and the request will fail; to avoid this kind of issue we suggest you to not exceed 12 hours timespan for each single request.
OTHER USEFUL INFORMATION:
Use the tools available on the website to check the data availaibilty, the network/instruments description and for other relevant informations.
Upper atmosphere physics and radiopropagation Working Group; Cesaroni, C.; De Franceschi, G.; Marcocci, C.; Pica, E.; Romano, V.; Spogli, L. Electronic Space Weather Upper Atmosphere Database (eSWua) - GNSS Scintillation Data, Version 1.0. Istituto Nazionale di Geofisica e Vulcanologia (INGV) August 1, 2020. https://doi.org/10.13127/ESWUA/GNSS.
Please note that the collection consists of data and instruments which could be property of the INGV as well as of other Institutions which share and distribute their data through the eSWua system; in the latter case we kindly ask you to also acknowledge the Institution owner of the data. The list of the instruments and related owning Institutions is available
here
http://ws-eswua.rm.ingv.it/scintillation.php/records/wsstation
Each station has its own endpoint to access the data stored into the eSWUa database. The following table lists all the stations/instruments and web-service endpoints. The last column reports an example of web-service call to access the data.
|
The following table lists all the parameter (FIELD NAME) that could be retrieved for each pertinent instrument (RELATED TO INSTRUMENT):
|
CITATION FOR THE RAW IONOGRAMS AND AUTOSCALED PARAMETERS:
Upper atmosphere physics and radiopropagation Working Group; Marcocci, C.; Pezzopane, M.; Pica, E.; Romano, V.; Sabbagh, D.; Scotto, C.; Zuccheretti, E. Electronic Space Weather Upper Atmosphere Database (eSWua) - HF Data, Version 1.0. Istituto Nazionale di Geofisica e Vulcanologia (INGV) August 1, 2020. https://doi.org/10.13127/ESWUA/HF.
Please note that the collection consists of data and instruments which could be property of the INGV as well as of other Institutions which share and distribute their data through the eSWua system; in the latter case we kindly ask you to also acknowledge the Institution owner of the data. The list of the instruments and related owning Institutions is available
here.
CITATION FOR MANUALLY VALIDATED DATA:
Upper atmosphere physics and radiopropagation Working Group, Cossari, A., Fontana, G., Marcocci, C., Pau, S., Pezzopane, M., Pica, E., & Zuccheretti, E. (2020). Electronic Space Weather upper atmosphere database (eSWua) - HF validated data (1.0). Istituto Nazionale di Geofisica e Vulcanologia (INGV). https://doi.org/10.13127/ESWUA/HFVALIDATED.
Each ionosonde has its own endpoints (1 for each "level of elaboration") to access the data stored into the eSWua database.
The following table lists all the Ionosondes and web-service endpoints. The last column reports an example of web-service call to access the data. The endpoint "STATION_CODE"_auto contain the autoscaled value and the reconstructed traces, the endpoint "STATION_CODE"_rdf contain the raw ionograms, the endpoint "STATION_CODE"_rev contain the manually validated parameters:
The following table lists all the HF product (map) web-service endpoints (1 for each products) with an example of web-service call to access the data.
|
The following table lists all the parameter (FIELD NAME) that could be retrieved for each instrument (RELATED TO INSTRUMENT/ ELABORATION LEVEL).
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Upper atmosphere physics and radiopropagation Working Group; Cesaroni, C.; Marcocci, C.; Pica, E.; Spogli, L. Electronic Space Weather Upper Atmosphere Database (eSWua) - Total Electron Content (TEC) Data, Version 1.0. Istituto Nazionale di Geofisica e Vulcanologia (INGV) August 1, 2020. https://doi.org/10.13127/ESWUA/TEC.
The following table lists all the GNSS TEC data web-service endpoints (1 for each products) with an example of web-service call to access the data.
|
The following table lists all the parameter (FIELD NAME) that could be retrieved for each endpoint (RELATED TO DATA).
|
The following instructions contains examples referred to the GNSS SCINTILLATION stations endpoint; the same rules apply to the other data collections (HF SYSTEM and GNSS TEC) by changing the web service endpoint and the requested fields.
ACCESS TO SINGLE/MULTIPLE FIELD OF A TABLE
By default all the fields of an endpoint are selected. With the include string you can select specific fields, separeted by a comma. Multiple fields should be comma separated. An asterisk (*) may be used as a wildcard to indicate all fields. Similar to include you can use the exclude string to remove certain fields.
Example1: http://ws-eswua.rm.ingv.it/scintillation.php/records/wsstation?include=code
returns the field code (station code) for all the records of the station list endpoint.Example2: http://ws-eswua.rm.ingv.it/scintillation.php/records/wsstation?include=lat,lon
returns the lat (latitude) and lon (longitude) field for all the records of the station list endpoint.
ORDERING THE RESULTS
With the order string you can sort the results. By default the sort is in ascending order, but by specifying desc this can be reversed:
Example1: http://ws-eswua.rm.ingv.it/scintillation.php/records/wsstation?include=code,id&order=code
the station list (id and code field) will be order by codeExample2: http://ws-eswua.rm.ingv.it/scintillation.php/records/wsstation?include=code,id&order=id,desc
the station list (id and code field) will be order (descending) by id
APPLY FILTERS AND MULTIPLE CONDITIONS TO THE REQUESTS
Filters provide advanced search functionality. The supported filters are the following:
You can negate all filters by prepending a "n" character (i.e. "eq" becomes "neq").
Example1: http://ws-eswua.rm.ingv.it/scintillation.php/records/wsstation?filter=code,eq,dmc0p
returns only the station where the code is equal to dmc0pExample2: http://ws-eswua.rm.ingv.it/scintillation.php/records/wsdmc0p?filter=dt,bt,2019-10-16 02:45:00,2019-10-16 03:00:00&order=dt
It returns all the fields for the endpoint of the station dmc0p for the time period between 2019-10-16 02:45:00 and 2019-10-16 03:00:00 ordered by time.Example3: http://ws-eswua.rm.ingv.it/scintillation.php/records/wslyb0p?filter=dt,bt,2019-10-16 02:45:00,2019-10-16 03:00:00&order=dt&include=svid
It returns only the field svid for the endpoint of the station lyb0p for the time period between 2019-10-16 02:45:00 and 2019-10-16 03:00:00 ordered by time
If you want to apply multiple filters and conditions as an AND operator you need to use the & concatenator.
Example: http://ws-eswua.rm.ingv.it/scintillation.php/records/wsstation?filter=area,eq,ANTARCTIC&filter=code,sw,dmc
returns the stations located in the ANTARCTIC area wich names starts with the string dmc
If you want to apply the same filter multiple times as an OR operator you need to enumerate the filters.
Example: http://ws-eswua.rm.ingv.it/scintillation.php/records/wsstation?filter1=area,eq,ANTARCTIC&filter2=area,eq,ARCTIC
returns the stations located in the ANTARCTIC or the ARCTIC area
IMPORTANT NOTE
The FIELD NAME dt is the TIMESTAMP of each record and has the following format (for all the data collection): YY-MM-DD hh-mm-ss.
Note that the blank space in the dt field between DD and hh must to be substituted with a %20 in the web-service call.
Example: if the chosen dt is: 2019-10-16 02:45:00 the string must be included as 2019-10-16%2002:45:00. (note that some code editor software automatically generate the substitution during "get" requests)
You can use the eSWua download tool to understand how the web-service http calls should be arranged.
By choosing the data-collection and applying the several filters you will be redirected to a page where (beside the direct download of the data)
both the html string for the web-service request and a pre-formatted code in Python language will be generated.
MATLAB Example
%% Matlab code for retrieve all the fields/parameters from the last temporal record recorded in the database
clc
close all
clear all
%retrieve the scintillation stations/instruments list
stations=''.http://ws-eswua.rm.ingv.it/scintillation.php/records.'/wsstation';
station_list=cell2mat(struct2cell(webread(stations))); %%Matlab structure with the stations/instruments list
%% retrieve the complete set of parameters in the last available time instant
station='lyb0p'; %change the station code for retrieve the parameters from other stations
%identification of the last available record
time_limit=sprintf(''.http://ws-eswua.rm.ingv.it/scintillation.php/records.'/ws%s?include=dt&order=dt,desc&size=1',station);
time=webread(time_limit); %last available temporal instant
%retireve the parameters
parameter_link=sprintf(''.http://ws-eswua.rm.ingv.it/scintillation.php/records.'/ws%s?filter=dt,eq,%s&order=dt',station,time.records.dt);
parameter_list=webread(parameter_link);
parameter=cell2mat(struct2cell(parameter_list)); %Matlab structure with all the parameters for the chosen instrument
Python Example
### Python code for retrieve all the fields/parameters from the last temporal record recorded in the database
import json
import urllib.request
#retrieve the scintillation stations/instruments list
url=''.http://ws-eswua.rm.ingv.it/scintillation.php/records.'/wsstation'
webURL=urllib.request.urlopen(url)
station_list=json.loads(webURL.read()) # station_list is a dictionary with the stations/instruments list
#Retrieve the complete set of parameters in the last available time instant ####
station='lyb0p' #change the station code for retrieve the parameters from other stations
#identification of the last available record
url = ''.http://ws-eswua.rm.ingv.it/scintillation.php/records.'/ws{}?include=dt&order=dt,desc&size=1'.format(station)
webURL=urllib.request.urlopen(url)
data=json.loads(webURL.read())
time=data["records"][0]["dt"] #last available record
#retireve the parameters
time=time.replace(" ","%20") #blank spaces needs substitution with "%20"
url=''.http://ws-eswua.rm.ingv.it/scintillation.php/records.'/ws{}?filter=dt,eq,{}&order=time'.format(station,time)
webURL=urllib.request.urlopen(url)
parameter=json.loads(webURL.read()) # parameter is a dictionary with the stations/instruments list
print(parameter) #print of the parameters
The following errors may be reported:
Error / HTTPresponse code / Message:
1000 / 404 Not found / Route not found
1001 / 404 Not found / Table not found
1002 / 422 Unprocessable entity / Argument count mismatch
1003 / 404 Not found / Record not found
1004 / 403 Forbidden / Origin is forbidden
1005 / 404 Not found / Column not found
1006 / 409 Conflict / Table already exists
1007 / 409 Conflict / Column already exists
1008 / 422 Unprocessable entity / Cannot read HTTP message
1009 / 409 Conflict / Duplicate key exception
1010 / 409 Conflict / Data integrity violation
1011 / 401 Unauthorized / Authentication required
1012 / 403 Forbidden / Authentication failed
1013 / 422 Unprocessable entity / Input validation failed
1014 / 403 Forbidden / Operation forbidden
1015 / 405 Method not allowed / Operation not supported
1016 / 403 Forbidden / Temporary or permanently blocked
1017 / 403 Forbidden / Bad or missing XSRF token
1018 / 403 Forbidden / Only AJAX requests allowed
1019 / 403 Forbidden / Pagination Forbidden
9999 / 500 Internal server error / Unknown error
For any problem,information or request contact us at eswua@ingv.it