New version of neonUtilities (v1.3.6)

neonUtilities 1.3.6 is now available on CRAN! Updating is recommended. Changes in this version include:

  • Fixes stackByTable() bugs that caused errors when stacking large numbers of site by month files
  • sensor_positions files in new format stack correctly
  • zipsByURI() is compatible with fastq files
  • stackByTable() stacks lab-specific files as well as site-specific files

If you have any issues or questions about this release, please visit our issues log on GitHub.

Rodent-borne Pathogen Data Product Scope Change

NEON recently obtained approval to begin testing small mammal blood and ear tissue samples for tick-borne pathogens rather than hantaviruses. This change will be reflected in the data product beginning in the 2020 field season.

The details of this adjustment in scope are the result of extensive dialogue with NEON users and advisory groups stemming from user requests to improve the utility of this data product. This will shift the focus to a set of emerging pathogens that are more abundant and widespread than hantaviruses, and that link the tick vector and small mammal datasets. This change will also improve the public health relevance, capacity to detect continental- and decadal-scale patterns, and overall data utility. Blood samples collected in excess of our capacity to test will continue to be archived at the NEON Biorepository and can be requested by individual researchers.

The 2020 sampling year will consist of a pilot project in which ear and blood samples from 5 sites will be tested for a range of tick-borne pathogens. The list of pathogens is likely to include: Borrelia burgdorferi, Borrelia miyamotoi, Borrelia mayonii, Anaplasma phagocytophilum, Rickettsia ricketsii, Rickettsia parkeri, Francisella tularensis, and Babesia microti. The list of sites included in the pilot study will depend in part on the number of samples available given fluctuating field sampling efforts due to the pandemic; however the preference is to test samples from HARV, SCBI, TREE, TALL and SJER. Pending results from this pilot study, full implementation of tick-borne disease testing in small mammals is anticipated beginning in the 2021 field season. Moving forward, (from the 2020 field season onwards) all rodent pathogen testing will be for tick-borne diseases, and no additional hantavirus tests will be conducted by NEON.

Please use the data portal contact form for further inquiries and include the term “INC0039430“ in the body of your request

Timestamp mismatch between the Picarro G2131 and L2130 instruments

We have recently discovered multiple instances where timestamps became out of sync between the Picarro G2131 and L2130 instruments and computers, and what was assigned to the data points upon transmission to NEON Headquarters. This has resulted in offsets ranging from 20 seconds to 12 minutes in data products DP1.00036 Atmospheric CO2 isotopes and DP1.00037 Atmospheric H2O isotopes. Many sites and time periods have been affected, we will update with specific instances by the end of July 2020.

There are two root causes:

  1. The instrument becomes out of sync with the location controller at the site. This issue has been resolved and is not expected to impact data going forward.
  2. The instrument becomes out of sync with its computer. When detected, this can be resolved by a hard restart of the instrument. We are working with the manufacturer on further troubleshooting and to find a permanent resolution.

When this problem occurs, it often leads to high variance and offset mean values in reference gas measurements because the averaging window in data processing covers two span gasses.

We are raising the science review flag for the affected time periods and investigating the possibility of data correction. Please use the data portal contact form for further inquiries; select DP1.00036 or DP1.00037 in the Data Product dropdown list and include the term “PRB0040670“ in the body of your request

LI-840 Infrared Gas Analyzer downtime resulting in data flags and gaps

The LI-840 Infrared Gas Analyzer requires more maintenance and has an observed failure rate higher than other NEON sensors measuring atmospheric carbon dioxide and water vapor. This has led to multiple data gaps and periods of flagged data.

Many sites and time periods have been affected, we will update this post with specific instances within the next month. Increased monitoring has reduced the detection time of these issues, but downtime has been magnified by the suspension of field work due to COVID-19.

NaN values and raised quality flags occur in these data products, which are contained in the bundled eddy-covariance file (DP4.00200.001):

  • DP1.00100 H2O concentration - storage
  • DP1.00099 CO2 concentration - storage
  • DP2.00008 CO2 concentration rate of change
  • DP2.00009 H2O concentration rate of change
  • DP3.00009 CO2 concentration rate of change profile
  • DP3.00010 H2O concentration rate of change profile
  • DP4.00067 Carbon dioxide flux
  • DP4.00137 Latent heat flux

Please use the data portal contact form for further inquiries; select DP4.00200.001 or any of the constituent data products listed above in the Data Product dropdown list and include the term “PRB0040480“ in the body of your request

Update 4: Microbial Community Composition Data Products Being Reprocessed

This is an update to the following past news posts:

In order to be more agile in identifying and correcting quality issues with the microbial community composition data products, we are developing an in-house data processing pipeline to generate the microbial community composition data products. Testing of the pipeline is currently underway and will be followed by parameter optimization. A firm timeline for completing testing and optimization is unclear: this news item will be updated periodically as development progresses.

This issue will be updated again August 17, 2020. Please use the data portal contact form for further inquiries and include the term “INC0033894“ in the body of your request.

Resolved: Faulty algorithm in Windspeed and direction above water on-buoy (DP1.20059.001)

Update to: Faulty algorithm in Windspeed and direction above water on-buoy (DP1.20059.001).

Buoy wind speed and direction data have been reprocessed with the wind direction-averaging algorithm fix through 2019 and instrument data reprocessing is presently continuing into 2020 as of July 1, 2020. However, an investigation on the reprocessed wind direction data identified another issue of inconsistent 2D wind sensor install orientations, causing unreliable buoy wind direction data. This issue will be addressed in further detail in a separate data portal news announcement. Buoy wind speed data remains unaffected. We recommend data users continue to use the 2D wind speed and direction (DP1.00001.001) at lake sites for reliable wind direction data.

Update: Notice of incorrect soil CO2 concentration data (DP1.00095.001)

Update: to Notice of incorrect soil CO2 concentration data (DP1.00095.001).

A fix has been developed for the issue of missing atmospheric pressure corrections for the soil CO2 concentration data product (DP1.00095.001). This will result in the pressure correction being performed if the necessary inputs are available or flagging the data if the pressure correction inputs are unavailable. Reprocessing previously collected data has begun with the expectation that the corrected data will start to be published by early July. In the meantime, all previously published soil CO2 concentration data has been flagged on the NEON data portal. An update will be provided in early July 2020.

Please use the data portal contact form for further inquiries; please select DP1.00095.001 in the Data Product dropdown list and include the term "PRB0040660" in the body of your request.

Spurious trace precipitation in Primary Precipitation data product (DP1.00006.001)

A review of the precipitation data product (part of to DP1.00006.001) identified that spurious trace precipitation is recorded frequently for all primary precipitation sensors (weighing gauges). While actual precipitation events are typically distinguishable from spurious events by visual inspection of the time series, the spurious events can add significantly to precipitation totals when aggregated over long intervals. Sums of spurious precipitation range from < 0.05 mm/day to > 0.5mm/day, with higher sums generally corresponding to sites and time periods of greater evaporative demand.  

We are working on additional quality control algorithms in order to detect and remove spurious trace events in the primary precipitation data product. An updated algorithm is expected sometime in 2021. An update to this issue will be provided by March, 2021. In the meantime, data remain available and notice of this issue has been added to the to data product description. Users are advised to manually inspect and clean the dataset of spurious trace precipitation events.

For further inquiries, please use the data portal contact form; please select DP1.00006.001 in the Data Product dropdown list and include the term "PRB0040676" in the body of your request.

Instrumented data reprocessed

The full timeseries of most instrumented data products (exceptions below) were reprocessed with the most recent algorithms, quality control thresholds, and/or other metadata to improve overall data coverage and quality. Specific algorithm updates and data corrections are noted in other news items and in the issue log for each data product.  

Publication of reprocessed data will begin Wednesday, June 17, 2020 and will likely complete in early July, 2020. Reprocessed data can be identified by the publication date embedded in the file name of each downloaded data file. Dates on or after June 17, 2020 indicate reprocessed data.  

Publication of June 2020 data may be delayed until mid to late July while this process completes.  

Products not included in the current reprocessing effort include surface atmosphere exchange data (DP4.00200.001: Bundled data products - eddy covariance) and stream discharge products (DP4.00130.001: Stream discharge, and DP4.00133.001: Stream discharge rating curve). These products are reprocessed on separate schedules.

Please use the data portal contact form for further inquiries; please include the term “CHG0032696“ in the body of your request.

Resolved: Correction of 2D wind direction data

Resolution to Upcoming corrections to 2D wind direction data.

Previously, we identified an issue with the 2D anemometers on the NEON towers that were producing 2D wind data that did not correlate with 2D wind data collected at NEON meteorological stations nearby. The data have been reprocessed, and, the corrected data will be available via the data portal in July 2020. Please use the data portal contact form for further inquiries and include the term “INC0033824“ in the body of your request.

Faulty algorithm in Windspeed and direction above water on-buoy (DP1.20059.001)

NEON recently discovered an error in the algorithm that creates the  Wind speed and direction above water on-buoy (DP1.20059.001) data product. As a result, all published buoy wind direction values averaged at both 2-minute and 30-minute intervals have been faulty to present day. Buoy wind speed was not affected by this algorithm error, so this data product has not been removed from the portal. All past and present-day buoy wind speed and direction data from the corrected algorithm will be republished following a large data republication effort. An update will be provided by July 1. The nearby, on-shore aquatic met station  2D wind speed and direction (DP1.00001.001) at buoy sites remains unaffected by the buoy’s wind direction algorithm and may serve as a suitable alternative in the meanwhile.

For further inquiries, please use the data portal contact form; please select DP1.20059.001 in the Data Product dropdown list and include the term “INC0038276“ in the body of your request.

API Rate Limiting Issue

Large data downloads using neonUtilities may fail to download all available data due to the new API rate limit. If your data download returns messages in the form "No files found for site SITE and month YYYY-MM", try downloading again, specifying only the sites and months that weren't found on the first attempt.

We are working to release a new version of neonUtilities that handles this situation smoothly. In the meantime, downloading small amounts of data and/or downloading using an API token will reduce the problem.

Resolution: Incorrect Geolocation in 2019 Toolik AOP Spectrometer Data

Resolution to Notice of Incorrect Geolocation in 2019 Toolik AOP Spectrometer Data.

Previously, we identified an issue with the 2019 Toolik airborne (AOP) spectrometer (NIS) data products. The data were originally processed with an incorrect GPS timing offset and, as a result, were not correctly geolocated. The data have been reprocessed, and, as of 30 April 2020, the corrected data are available via the data portal. Please use the data portal contact form for further inquiries and include the term “INC0036345“ in the body ofyour request.

Update 3: Microbial Community Composition Data Products Being Reprocessed.

This is an update to the following past news posts:

We have determined that significant modifications to the bioinformatics pipeline are needed to ensure the highest quality microbial community composition data products. To that end, we are performing optimization of analytical parameters in the pipeline. There is no change to data availability for the microbial community composition data products.

This issue will be updated again June 15, 2020. Please use the data portal contact form for further inquiries and include the term “INC0033894“ in the body of your request.

Notice of incorrect soil CO2 concentration data (DP1.00095.001)

We have discovered that the atmospheric pressure correction for the soil CO2 concentration data product (DP1.00095.001) is not being performed, causing incorrect values to be reported. This issue affects all TIS sites and, while the exact start date is not currently known with certainty, it started prior to 4 Jan 2019 and likely affects all previously published soil CO2 concentration data. The magnitude of the impact is variable among sites (less pronounced for sites close to sea level and more substantial at high elevation sites), but also varies, albeit less markedly, with local conditions (e.g., temperature and CO2 concentration). As an example, omitting the pressure correction at atmospheric pressures of 100, 90, 80, and 70 kPa (roughly corresponding to elevations of 100, 1000, 1900, and 3000 m, respectively) resulted in an underestimate of true soil CO2 concentration by 2%, 15%, 28%, and 41%, respectively. These values were calculated assuming 10 °C temperature, 75% relative humidity, 20.95% oxygen concentration, and an uncorrected CO2 sensor reading of 1000 ppm. Work has begun to correct this issue with the expectation of having it resolved by approximately July 2020. An update will be provided by the end of May 2020. Please use the data portal contact form for further inquiries; please select DP1.00095.001 in the Data Product dropdown list and include the term “PRB0040660“ in the body of your request.

Resolved: Timestamp mis-match discovered for sensor data products at WLOU and SYCA for part of 2019

This is the final update to Timestamp mis-match discovered for sensor data products at WLOU and SYCA for part of 2019.

Current Status: Data has been re-streamed, re-processed, and re-published for both SYCA and WLOU with the correct timestamps. Data is now available for download on the Portal and API. Please use the data portal contact form for further inquiries; please include the phrase “INC0034443“ and “INC0035951“ in the body of your request.

Update #1: Timestamp mis-match discovered for sensor data products at WLOU and SYCA for part of 2019

This is an update to Timestamp mis-match discovered for sensor data products at WLOU and SYCA for part of 2019.

Current Status: Data has been re-streamed for both SYCA and WLOU with the correct timestamp. The re-processing is in a testing phase with a plan to have the data re-processed for re-publication by the end of April. An update will be posted when the data is re-published on the portal or by April 30th, 2020. Please use the data portal contact form for further inquiries; please include the phrase “INC0034443“ and “INC0035951“ in the body of your request.

Timestamp mis-match discovered for sensor data products at WLOU and SYCA for part of 2019

We have identified that the timestamp of the on-site server was set to Mountain Time instead of Coordinated Universal Time (UTC) resulting in all sensor data streaming with the wrong timestamp for SYCA and WLOU from 6/4/2019 - 12/5/2019 and 7/23/2019 - 2/17/2020, respectively. All impacted data have been removed from the NEON data portal and will be re-streamed, re-processed, and re-published. We will provide an update iby March 20, 2020 with information about when data will be on the portal. An automated script is being put in place to prevent this issue in the future. Impacted data products include:

  • AIS data products:
    • NEON.DP1.20100: Elevation of groundwater
    • NEON.DP1.20015: Specific Conductivity in groundwater
    • NEON.DP1.20217: Temperature of groundwater
    • NEON.DP1.20016: Elevation of surface water
    • NEON.DP1.20033: Nitrate in surface water
    • NEON.DP1.20042: PAR at water surface
    • NEON.DP1.20053: Temperature (PRT) in surface water
    • NEON.DP1.20288: Water quality
  • Shared AIS/TIS data products:
    • NEON.DP1.00001: 2-D windspeed and direction
    • NEON.DP1.00002: Single aspirated air temperature
    • NEON.DP1.00004: Barometric pressure
    • NEON.DP1.00024: PAR
    • NEON.DP1.00098: Relative humidity
    • NEON.DP1.00023: Shortwave and longwave radiation (net radiometer)
    • NEON.DP1.00006: Precipitation
    • NEON.DP1.00038: Stable isotope concentrations in precipitation
    • NEON.DP1.00013: Wet deposition chemical analysis

Please use the data portal contact form for further inquiries; please include the phrase “INC0034443“ and “INC0035951“ in the body of your request.

Update 2: Microbial Community Composition Data Products Being Reprocessed

This is an update to the following past news posts:

Data quality analysis for 90% of the existing fungal ITS sequencing data sets is complete. In addition, draft quality criteria and data flagging rules are under review by members of the external user community. We expect to finalize the quality criteria and flagging rules by February 29, 2020 and begin flagging sequence data immediately thereafter, beginning with fungal ITS datasets. No changes to data availability have been made.

Data sets will be re-posted incrementally as quality flagging is completed. This issue will be updated again March 30, 2020. Please use the data portal contact form for further inquiries and include the term “INC0033894“ in the body of your request.

Orthorectification errors identified for camera images for 2018_YELL_1 and 2019_JERC_5

We have identified issues with the camera image and mosaic data for 2018_YELL_1 and 2019_JERC_5. The products involved are:

  • L1
    • DP1.30010.001: High-resolution orthorectified camera imagery
  • L3
    • DP3.30010.001: High-resolution orthorectified camera imagery mosaic

In both cases, the orthorectified images for a single flight line (about 40 images for each site) are off by about 200 meters due to an error in the raw timing data. These errors are not correctable, but only a small portion of the total site coverage is affected. To correct for these issues, we deleted the misaligned images and recomputed the mosaics which will result in no loss of coverage due to spatial overlap. Updated imagery will be re-posted as soon as correction is completed. This issue will be updated again March 19, 2020. Please use the data portal contact form for further inquiries and include the term “INC0036413“ in the body of your request.

— 20 Items per Page
Showing 1 - 20 of 94 results.