Frequently Asked Questions Frequently Asked Questions

If you don't find the answer to your question here, please use the feedback form to let us know.

 

 

Finding Data

With many data products and field sites to choose from, exploring NEON data may feel a bit overwhelming. The Data Portal is in active development, and you may see major changes to web pages over the next few months. With that in mind, here are a few ways to get started:

Option 1: Type in what you are looking for in the search bar on the homepage of the data portal. You can search by keyword, data product name, year, site, or domain. A dynamic dropdown list will appear as you type. Select an option, and the data product(s) related to your choice will pre-populate in the Data Products page (all sites and months available). If you don’t see what you’re looking for, read on.

Option 2: Go to the interactive Data Product Catalog and explore our data products. This resource provides detailed information about each data product, along with links to related documentation (such as protocols and science designs), links to other related data products, and change logs. Each data product page also provides a link to start the data download process. All of the information provided in each data product’s catalog page is also delivered in README files with each download.

Option 3: Go to the Data Products page, and in the left side bar, select a date range, sites, and data products or themes of interest. This will populate a list of data products that are available and meet your criteria. This page will be replaced in coming months with an updated version of Option 4!

Option 4: Explore the beta browse data page. This page gives you a shopping cart experience, blending the capability of the Data Product Catalog with searching and filtering capabilities. For instrumented time-series data products, this page also provides a dynamic time-series charting tool, so you can visualize the data over time, selecting data by site, senor, and variable.

A horizontal bar will appear for each data product selected, followed by 12 square boxes. Each box represents one month as indicated by the time axis along the top. If data are available for all of the selected sites for a given month, the box will be solid green. If data exist for the data product but not for all sites that were selected, the box will be filled with green diagonal lines. If the box is clear, then no data are available for the data product at any of the selected sites for the month that the box represents. A green triangle to the right or left of the 12 boxes indicates available data in another part of the total available time range. Click on this arrow to move the time frame.

NEON is reprocessing early releases of data in order to standardize output formats and documentation. Reprocessing will occur on all data released to the portal before 2017. During reprocessing of each product, there may be brief interruptions in data availability.

NEON is currently in construction; data that was collected prior during the construction period will become available incrementally through 2019. Please check back frequently! See the Information for Researchers page for more general information, or Field Sites for more specific construction status updates.

 

 

Downloading Data

Once you have selected data of interest in the Data Products page, click on the Configure Dataset button. A set of informational links, including basic information about the data product and the number of sites that have the data you requested, will be displayed. A menu of data package configuration options will also be displayed. Hover your mouse over each underlined phrase to get a tooltip with more information. Select your options and click the Download Dataset button.

At this time, only one data product may be downloaded within a single data package, with the exception of the Bundled data products - eddy covariance. However, data from the entire possible date range and several sites for a specific data product can be downloaded in a single data package. Be careful to check the size of the data file you have requested - they can be rather large and take a long time to download.

There currently is no limit except for your patience. However, you may wish to check the size of your requested package before hitting the download button, as a package of many gigabytes could take a long time to download.

 

 

How Data Are Organized

NEON measures a diverse suite of biological, physical, chemical and ecological characteristics at field sites across the continent. NEON data are sent to headquarters after site construction is complete and data collection begins. The Observatory processes these measurements to derive standard, quality-assured data products that support greater understanding of complex ecological processes at local, regional and continental scales. Available NEON data, supporting metadata, science designs, data collection documentation and data processing documentation are accessible through the NEON Data Portal. For more information, please visit the Data Processing webpage on the NEON web portal.

A data package is a zip file containing a collection of data and metadata files. This zipped data package is dynamically generated when you submit a query consisting of a data product, one or more sites, and a date range to the data portal.

Observational and instrumented data products (with the exception of eddy covariance data) are divided into many small files in the Comma Separated Values (CSV) format. Each file contains data for a single data product at one site over one month, and at an additional level of granularity.

For observational data products, the level of granularity is a type of data collection activity. For example, in the ground beetle trapping data, this includes individual tables for field data, sorting, initial identification, and later expert identification (if needed). A file containing metadata about data validation is also included.

For instrumented data products (except for eddy covariance), the level of granularity is the vertical and/or horizontal position of the sensor collecting the data. A single field site often has multiple sensors of the same type (for example, soil mosture sensors along an array), each at a different location.

Eddy covariance data are delivered in the Hierarchical Data Format (HDF5) as a 'bundle' of many data products that are not delivered individually. Similarly to other instrumented data products, each data file contains data for a single site and month. To learn more about this product, click on the links in this paragraph.

The Airborne Observation Platorm (AOP) only flies over each site once in a year. AOP data files are organized by data product and site (sometimes two sites if they are close to one another), and year of collection. The data portal and API allow for a more granular approach to downloading data files because files may be very large.

 

 

How Data Are Documented

First, check out the Document Library. This is a rich resource of many types of documents, including overarching science designs, site characterization reports, spatial data, protocols (both from NEON and external labs that NEON contracts work with), data processing documentation (also known as Algorithm Theoretical Basis Documents, or ATBDs), and User Guides for observational data products. If you use any of these documents, please cite them as you would with any other publication.

Data packages may contain some of these documents, specific to the data product downloaded. Each data package also may contain a readme file, a machine-readable form of the readme file using the Ecological Metadata Language (EML) format, and a file that describes all of the variables available for the data product. Observational data products include a file that includes the validation rules used when ingesting the data.

Yes, we update our documentation as frequently as needed. In the Document Library, you will find a few subfolders called "Obsolete Protocols" and "Past Versions". Most of our documents use a lettering system for versioning, starting with "DRAFT", then vA, vB, etc. We also update readme files and EML files on an as-needed basis. Newer data packages may have updates to their readme and EML files that are not present in older data packages.

EML is a widely used, community supported XML schema that supports rich documentation of data related to ecological research, particularly including environmental, ecological, and earth science data. It is supported by The Knowledge Network for Biodiversity (KNB); more information may be found here. The EML available at this time is NEON’s first implementation of the schema and will continue to be improved during NEON’s construction period. EML is valuable for batch-processing or integrating many data packages. KNB provides a stand-alone software package, Morpho , for generating EML-documented data packages. The eml package for reading and writing EML is available from  rOpenSci, a community-driven organization that develops and provides free and open-source tools.  NEON’s Work With Data set of tutorials includes  Time Series 01: Why Metadata Are Important: How to Work with Metadata in Text & EML Format as well as numerous tutorials about R, python, and data.

Some scientific characters do not display properly in web browsers. When you download them to your computer and view them with a good PDF viewer, these characters will be rendered correctly.

 

 

How To Work With Downloaded Data

There are a number of resources available for training and self-learning about the different types of data that NEON provides. Check out the Resources link in the data portal's navigation menu!

You might also be interested in asking the NEON Science Community Forum.

You can also ask specific questions through the data portal Feedback page - your question will be routed to the science team relevant to the data product you select from a drop down list.

For any of our observational or instrumented data products, we provide R and python tools to help you combine the files into tables you can easily use in your analyses. Check out our GitHub repository, NEON-utilities to learn more.

 

 

Using and Citing Data

Yes, please see our open NEON Data Usage and Citation Policy. This policy is subject to revision as needed.

We currently only provide unique identifiers for our Prototype Data. We are investigating how to provide unique identifiers, such as Digital Object Identifiers (ODIs), for our standard data products.

 

 

About Accessing Data Programmatically Via The API

If you are unfamiliar with an API, think of it as a 'middleperson' that provides a communication path for a software application to obtain information from a digital data source. APIs are becoming a very common means of sharing digital information. Many of the apps that you use on your computer or mobile device to produce maps, charts, reports, and other useful forms of information pull data from multiple sources using APIs. In the ecological and environmental sciences, many researchers use APIs to programmatically query and obtain data for their analyses.

Currently, the NEON API will return information about NEON data products and research locations through the /products and /sites or /locations endpoints, currently. Results are returned in a common machine-readable format, JSON (JavaScript Object Notation). Many software packages, like R, have add-ons that help translate JSON into other formats. The /data endpoint will return individual data files.

Instructions on how to construct a URL, as well as a tool to build URLs to your specifications, are on NEON's API page. There are some examples on our GitHub repository, neon-data-api, and rOpenSci is developing an R package, nneo. NEON is also planning to build tutorials and examples and host these from the NEON Work With Data webpage, as well as host future workshops.

Absolutely! We provide information, code examples, and an issue-tracking tool via NEON’s GitHub account, in the neon-data-api repository. We appreciate feedback and examples of how you are using the API.

We would be thrilled to learn about what you build using the API. Let us know how to link to your online example and we may choose to link to it from our GitHub repository.

 

 

NEON Data Portal Accounts

No. While we highly recommend that you download data while logged in, you can use the Data Portal and download NEON data without an account.

We will never give or sell your email address or other personal information to anyone. We have an obligation to record and report user demographic information and general data portal activity to NEON's sole funder, the National Science Foundation. We may use your anonymized account information to help us develop data portal usage metrics and to improve the quality of data portal services for all users. See Privacy Policy.

It's free and easy! Just click on the Sign In link in the header area then click on the Create Account link at the bottom of that page. Fill out the form and click the Create Account button.

When you have signed in, a My Account link will appear next to the Sign In link in the header area. Click on the link and you will be taken to the My Account screen where you can update your personal profile.