This is a place to share information about open source weather and climate data sets, as well as tips and resources on how to use them
This was kindly shared by an OU tutor for our club members on the Maths and Stats forums:
Something that might be of interest to this club is the Copernicus Climate Data Store
This is an enormous data repository of observations, reanalysis and projections for global weather and climate. There is so much information, it is a challenge to find one’s way around, and beware that any dataset you download may be huge. There are interactive representations of key data to explore. With some programming experience, you can produce plots and process information yourself online. And it is free to use. (A student of mine used it last year.)
Note: In September 2024 the Copernicus Climate Data Store will be migrating infrastructure to https://cds-beta.climate.copernicus.eu, and this will require an ECMWF account for access. This page has more information on ECMWF public access: https://www.ecmwf.int/en/forecasts/accessing-forecasts
The Copernicus Data Space Ecosystem is another Copernicus dataset that has really helpful user guide videos in its dedicated YouTube channel.
https://user.eumetsat.int/dashboard
There are a lot of resources to support anyone wishing to get started - this EUMETSAT guide lists them all:
https://user.eumetsat.int/resources/user-guides/getting-started-using-data
The resources listed there include this YouTube playlist where a series of videos covers
How to access satellite data from EUMETSAT and the missions it operates as part of the Copernicus programme
Basics of working with different types of data, including ocean colour, SST and altimetry
NOAA are the US scientific and regulatory agency charged with forecasting weather, monitoring oceanic and atmospheric conditions, charting the seas, conducting deep-sea exploration, and managing fishing and protection of marine mammals and endangered species
The National Centers for Environmental Information manages all of the databases for NOAA’s oceanic, atmospheric, and geophysical data:
https://www.ncei.noaa.gov/access
The measurements of the Global Greenhouse Gas Reference Network represent the longest on-going calibrated record of global GHG atmospheric levels. The GGGRN background air measurements are used to track year-to-year changes and long-term trends in GHG levels. Documenting GHG global trends reliably is important to quantify their impact on the earth’s radiative budget and climate. The measurements serve as a comparison with measurements made by many other international laboratories, and with regional studies. They are widely used in studies inferring space-time patterns of emissions and removals of greenhouse gases that are optimally consistent with the atmospheric observations.
Esri StoryMaps present complex scientific or safety information in easy-to-understand terms. They combine maps, narrative text, images, and multimedia with history, to transform data and products into interactive, visual narratives
StoryMap topics include Mapping NOAA Data
Overview of ECMWF Datasets:
https://www.ecmwf.int/en/forecasts/datasets/open-data
Information on the subset of ECMWF real-time forecast data that are made available to the public free of charge:
https://www.ecmwf.int/en/forecasts/datasets/open-data
ECMWF YouTube Training Webinars: https://www.youtube.com/@ecmwf
Tropical Climate Research Lab (video): How to download ECMWF weather and climate data for free
MEDIN is a partnership of UK organisations committed to improving access to and stewardship of UK marine data.
The MEDIN portal describes over 18,000 marine datasets, owned or managed by more than 600 UK organisations: https://portal.medin.org.uk/portal/start.php
MEDIN free online training workshops: https://medin.org.uk/data-standards/medin-workshops
The National River Flow Archive (NRFA), based at the UK Centre for Ecology & Hydrology, is the UK’s official record of river flow data. We collate, curate and provide access to data from over 1,600 gauging stations all around the UK. Our data are used in many ways; for example, to assess water availability and to assess the impacts of climate change over time. Some of our records go back to the nineteenth century, and many go back to the 1960s, allowing us to identify long-term trends in river flows, and to assess flood and drought events in relation to other similar events in the past.
NRFA data guide (5 min video): https://youtu.be/EfFFGufCkYg?si=PwulJ9JCbzjaial-
Suggestion kindly shared by club member:
A useful, free for non commercial use, dataset is provided by https://open-meteo.com. Recent past history at 1km and 11km sourced from well known datasets. All they ask is that use is fair and reasonable. Historic requests can take up to 5 days to be made available. Access is via api, and they give some CURL examples for how to programatically access data.
https://www.kaggle.com/datasets
A subsidiary of Google, it is an online community of data scientists and machine learning engineers. Kaggle allows users to find datasets they want to use in building AI models, publish datasets, work with other data scientists and machine learning engineers, and enter competitions to solve data science challenges.
Getting Started on Kaggle (YouTube Playlist): https://www.youtube.com/playlist?list=PLqFaTIg4myu8gbDh6oBl7XRYNBlthpDEW