Protocol Change for Beetle Data Products

The original ground beetle sampling design accommodated 40 pitfall traps deployed within 10 plot locations (4 traps per plot; 10 plots per site) from 2013-01-01 to 2017-12-31. Due to budget limitations, beginning in 2018 the sampling design was altered to eliminate one trap cup from each plot, resulting in reduced sampling per bout (3 traps per plot; 10 plots per site). The change in sampling effort reduced the number of samples generated during a bout; requiring a reduced budget for analytical costs to analyze samples (for species composition). All NEON terrestrial sites sample 10 plot locations within the landscape. Before 2018, up to 40 samples could be expected per sampling bout; one sample per pitfall trap. After 2018, up to 30 samples could be expected per sampling bout. 

This change to 30 traps per site per bout affects the following data products at all terrestrial sites from collections occurring after 2018-01-01.

  • Ground beetles sampled from pitfall traps (DP1.10022.001)
    • Impact of this change on data values: Up to 40 field sampling records will be available for affected sites during this time frame. Current sampling regime yields up to 30 field sampling records.
  • Ground beetle sequences DNA barcode (DP1.10020.001)
    • Impact of this change on data values: Because there were more samples (and thus species) collected from the field during this time frame, a greater diversity of individuals were available for DNA sequencing during this time. This change did not impact the quantity of individuals sent for DNA barcoding.

The reduced quantity of sampling (starting in 2018), is under evaluation and consideration by the Ground Beetle Technical Working Group. Pending their evaluation, alternate design options may be implemented to produce data that adequately address the needs of the suite of beetle data products. Please use the data portal contact form for further inquiries; select DP1.10022.001 in the Data Product dropdown list.

Protocol Change for Mosquito Data Products

The original mosquito sampling design accommodated two nights of trapping and one day of trapping during each bout (approx. 40 hours of sampling). Due to budget limitations, the sampling design was altered beginning in 2018 to eliminate one night of trapping from each bout resulting in a reduction of the duration of sampling per bout to approximately 24 hours per plot. The change in sampling duration reduced the number of samples generated during a bout; allowing for a reduction in the budget to analyze samples for species composition and pathogen status. All NEON terrestrial sites sample 10 plot locations within the landscape. Before 2018, up to 30 samples could be expected per sampling bout; one sample per mosquito trap during the daytime interval and one sample per mosquito trap during each nighttime interval. After 2018, up to 20 samples can be expected per sampling bout; one sample per mosquito trap during the daytime interval and one sample per mosquito trap during the nighttime interval.

This change from two nights to one night of sampling per bout affects the following data products at all terrestrial sites from collections occurring after 2018-01-01.

  • Mosquitoes sampled from CO2 traps (DP1.10043.001)
    • Up to 30 field sampling records will be available for affected sites during this time frame. Current sampling regime yields up to 20 field sampling records.
  • Mosquito-borne pathogen status (DP1.10041.001)
    • The amount of pathogen data produced during this time frame is greater than generated after 2018, because more samples collected from the field were available for pathogen testing.
  • Mosquito sequences DNA barcode (DP1.10038.001)
    • Impact of this change on data values: Because there were more samples (and thus species) collected from the field during this time frame, a greater diversity of individuals were available for DNA sequencing during this time. This change did not impact the quantity of individuals sent for DNA barcoding.

The reduced quantity of sampling (starting in 2018), is currently being reviewed by the Mosquito Technical Working Group. Pending their evaluation, alternate design options may be implemented to produce data that adequately address the needs of the suite of mosquito data products. Please use the data portal contact form for further inquiries; select DP1.10043.001 in the Data Product dropdown list.

Update 5: Microbial Community Composition Data Products Being Reprocessed

This is an update to the following past news posts:

NEON has completed development of an in-house data processing pipeline to generate the microbial community composition data products and has completed optimization of the parameters to be used in the data processing pipeline. Data reprocessing is expected to begin in mid-October. The timeline for completion is not yet determined.

This issue will be updated again December 1, 2020. Please use the data portal contact form for further inquiries and include the term “INC0033894“ in the body of your request.

Resolution: Bundling terrestrial biogeochemistry data products

Update to Bundling terrestrial biogeochemistry data products.

On 8/13/20, we announced some impending changes to OS biogeochemistry data products. As of today, those changes are complete. In order to improve data product usability, we have repackaged several data products to deliver field data and downstream analyses for chemistry and stable isotopes all in one download. For example, end users no longer need to download 3 different data products to access root biomass standing stocks, root chemistry, and root stable isotopes, but instead are able to get all of these data from downloading a single product.

  • There are 7 bundled data products in total:
    • DP1.10067.001: Root biomass and chemistry, periodic
    • DP1.10026.001: Plant foliar traits
    • DP1.10033.001: Litterfall and fine woody debris production and chemistry
    • DP1.10086.001: Soil physical and chemical properties, periodic
    • DP1.10047.001: Soil physical and chemical properties, distributed initial characterization
    • DP1.00096.001: Soil physical and chemical properties, Megapit
    • DP1.10066.001: Root biomass and chemistry, Megapit

While the stand-alone chemistry and stable isotope data products can still be searched on the data portal, they are no longer available for separate download and instead can only be accessed via these bundled products. As such, end users may need to update code routines for pulling data from the portal, but we hope this improves the overall user experience for NEON terrestrial biogeochemistry data products.

Please use the data portal contact form for further inquiries.

Plant presence and percent cover (DP1.10058.001) data ingest error

A subset of data in the Plant presence and percent cover data product ( DP1.10058.001) did not process correctly through the data pipeline, and were missing from published data. This issue affects all sites and all time periods.

The protocol for plant presence and percent cover involves surveys of subplots within NEON plots. If a subplot is surveyed and found to contain no additional species, beyond those already found in another subplot within the same plot, a record is entered with data field additionalSpecies=N, and no additional data are recorded for that subplot. These are the records that were missing.

We estimate up to 2% of plots surveyed each year contain at least one subplot where no additional species were found. Also note that in 2018, the 10 and 100 meter subplots were only surveyed in half of the plots, and beginning in 2019 the 10 and 100 meter subplots are surveyed at each site every other year, so in these cases certain subplot records are not expected.

As of September 30, 2020, the missing records have been restored. Please use the data portal contact form for further inquiries; please select DP1.10058.001 in the Data Product dropdown list and include the term “INC0038517“ in the body of your request

New version of neonUtilities (v1.3.7)

neonUtilities 1.3.7 is now available on CRAN! Updates in this version are primarily to keep current with updates in package dependencies.

Changes in this version include:

  • Fixes stackByTable() bugs related to data.table package updates
  • Download by time interval now uses timeIndex= rather than avg= in zipsByProduct()

If you have any issues or questions about this release, please visit our issues log on GitHub.

Resolved: Notice of incorrect soil CO2 concentration data (DP1.00095.001)

This is a resolution to the following past news posts:

A fix was developed for the issue of missing atmospheric pressure corrections for the soil CO2 concentration data product (DP1.00095.001). All data have now been republished and are available on the NEON Data Portal.

Sample IDs updated in sediment data products

Sample IDs have been updated in Sediment data products. They had been incorrect, with station IDs that did not match the named location. Approximately 200 sample IDs were affected, across most sites and through several years. The sample IDs were updated across all data tables.

The affected products include:

  • Sediment Chemical Properties (DP1.20194.001)
  • Sediment Physical Properties (DP1.20197.001)

The sample IDs are primarily used for linking across the various tables, and that linkage has always been preserved; when they were incorrect, they were still consistent across the tables. Therefore the only issue for the data user was if they were extracting the station ID from the character string of the sample ID rather than from the named location field. All data downloaded after March 2020 have the correct sample IDs which match the named locations.

Please use the data portal contact form for further inquiries please select DP1.20194.001 or DP1.20197.001 in the Data Product dropdown list.

Water Chemistry external lab data transformed from wide to long format

All external lab data has been transformed from wide format to long format for the water chemistry data products. The wide format table (externalLabData_pub) has been deprecated, and replaced with this table: externalLabDataByAnalyte_pub.

The affected products include:

  • Chemical properties of Surface Water (DP1.20093.001)
  • Chemical Properties of Groundwater (DP1.20092.001)

All previous data have been transferred to the new table as of April 2020. There should be no further updates.

Please use the data portal contact form for further inquiries please select DP1.20093.001 or DP1.20092.001 in the Data Product dropdown list.

Quality Assurance updates and fixes to 2017 AOP data

The sites flown in 2017 did not have a uniform QAQC document available for the L3 tiles – a data quality measure implemented at the start of the 2018 flight season. This quality check revealed that the L3 Albedo tiles did not include a 'No Data' value and that several sites had a flight line in which the weather metadata was missing from the HDF5 file, which prevented weather information from being propagated to downstream QA processes. The sites missing with weather metadata missing from a flight line were BART, NOGP, WOOD, and JORN. The weather data was updated using the ASO-provided flight logs and the Albedo tiles had the appropriate 'No Data' value assigned. The L3 mosaiced tiles were recreated for the sites that were affected by missing weather data in case the previously omitted line was flown in better weather conditions. Once these corrections had been made, the QAQC document was generated for each site flown in the 2017 flight season. As of August 19, 2020, these data have been corrected and available for download on the Data Portal and through NEON's API.

Please use the data portal contact form for further inquiries and include the term “INC0037086“ in the body of your request

NEON aquatic dissolved oxygen sensors percent saturation normalized to sea level pressure

NEON dissolved oxygen (DO) sensors are calibrated using local atmospheric pressure.  However, the DO % saturation currently reported in the water quality data product (DP1.20288.001) have been normalized to sea level pressure (760 mmHg) by the YSI EXO2 multiparameter sonde.  NEON has begun the process of reconfiguring the multisondes to also report DO % saturation relative to local atmospheric pressure, which will be published as a new parameter in the water quality data product.  In the interim, NEON has developed the localPressureDO R-package that uses barometric pressure from the streamside meteorological station to post-correct historical data and can be accessed in the NEONscience/NEON-water-quality GitHub repo. The DO concentration in mg/L data stream is not impacted by the % saturation reporting settings, and is correct. 

Please use the data portal contact form for further inquires: please select DP1.20288.001 in the Data Product dropdown list and include the term “INC0040000“ in the body of your request

Bundling terrestrial biogeochemistry data products

In order to improve usability of the Observational System terrestrial biogeochemistry data products, we are repackaging several data products to deliver all of the relevant data for sample sets in one download. For example, end users will no longer need to download DP1.10067.001, DP1.10102.001 and DP1.10099.001 to access root biomass standing stocks, root chemistry, and root stable isotopes (respectively), but will instead be able to get all of these data from downloading just DP1.10067.001. The same will be true for leaf litter, canopy foliage, and soil samples. Once all plant and soil data products have been updated, we will update this news item. End users may need to update code routines for pulling data from the portal, but we hope this improves the overall user experience for NEON terrestrial biogeochemistry data products.

Please use the data portal contact form for further inquiries.

New version of neonUtilities (v1.3.6)

neonUtilities 1.3.6 is now available on CRAN! Updating is recommended. Changes in this version include:

  • Fixes stackByTable() bugs that caused errors when stacking large numbers of site by month files
  • sensor_positions files in new format stack correctly
  • zipsByURI() is compatible with fastq files
  • stackByTable() stacks lab-specific files as well as site-specific files

If you have any issues or questions about this release, please visit our issues log on GitHub.

Rodent-borne Pathogen Data Product Scope Change

NEON recently obtained approval to begin testing small mammal blood and ear tissue samples for tick-borne pathogens rather than hantaviruses. This change will be reflected in the data product beginning in the 2020 field season.

The details of this adjustment in scope are the result of extensive dialogue with NEON users and advisory groups stemming from user requests to improve the utility of this data product. This will shift the focus to a set of emerging pathogens that are more abundant and widespread than hantaviruses, and that link the tick vector and small mammal datasets. This change will also improve the public health relevance, capacity to detect continental- and decadal-scale patterns, and overall data utility. Blood samples collected in excess of our capacity to test will continue to be archived at the NEON Biorepository and can be requested by individual researchers.

The 2020 sampling year will consist of a pilot project in which ear and blood samples from 5 sites will be tested for a range of tick-borne pathogens. The list of pathogens is likely to include: Borrelia burgdorferi, Borrelia miyamotoi, Borrelia mayonii, Anaplasma phagocytophilum, Rickettsia ricketsii, Rickettsia parkeri, Francisella tularensis, and Babesia microti. The list of sites included in the pilot study will depend in part on the number of samples available given fluctuating field sampling efforts due to the pandemic; however the preference is to test samples from HARV, SCBI, TREE, TALL and SJER. Pending results from this pilot study, full implementation of tick-borne disease testing in small mammals is anticipated beginning in the 2021 field season. Moving forward, (from the 2020 field season onwards) all rodent pathogen testing will be for tick-borne diseases, and no additional hantavirus tests will be conducted by NEON.

Please use the data portal contact form for further inquiries and include the term “INC0039430“ in the body of your request

Timestamp mismatch between the Picarro G2131 and L2130 instruments

We have recently discovered multiple instances where timestamps became out of sync between the Picarro G2131 and L2130 instruments and computers, and what was assigned to the data points upon transmission to NEON Headquarters. This has resulted in offsets ranging from 20 seconds to 12 minutes in data products DP1.00036 Atmospheric CO2 isotopes and DP1.00037 Atmospheric H2O isotopes. Many sites and time periods have been affected, we will update with specific instances by the end of July 2020.

There are two root causes:

  1. The instrument becomes out of sync with the location controller at the site. This issue has been resolved and is not expected to impact data going forward.
  2. The instrument becomes out of sync with its computer. When detected, this can be resolved by a hard restart of the instrument. We are working with the manufacturer on further troubleshooting and to find a permanent resolution.

When this problem occurs, it often leads to high variance and offset mean values in reference gas measurements because the averaging window in data processing covers two span gasses.

We are raising the science review flag for the affected time periods and investigating the possibility of data correction. Please use the data portal contact form for further inquiries; select DP1.00036 or DP1.00037 in the Data Product dropdown list and include the term “PRB0040670“ in the body of your request

LI-840 Infrared Gas Analyzer downtime resulting in data flags and gaps

The LI-840 Infrared Gas Analyzer requires more maintenance and has an observed failure rate higher than other NEON sensors measuring atmospheric carbon dioxide and water vapor. This has led to multiple data gaps and periods of flagged data.

Many sites and time periods have been affected, we will update this post with specific instances within the next month. Increased monitoring has reduced the detection time of these issues, but downtime has been magnified by the suspension of field work due to COVID-19.

NaN values and raised quality flags occur in these data products, which are contained in the bundled eddy-covariance file (DP4.00200.001):

  • DP1.00100 H2O concentration - storage
  • DP1.00099 CO2 concentration - storage
  • DP2.00008 CO2 concentration rate of change
  • DP2.00009 H2O concentration rate of change
  • DP3.00009 CO2 concentration rate of change profile
  • DP3.00010 H2O concentration rate of change profile
  • DP4.00067 Carbon dioxide flux
  • DP4.00137 Latent heat flux

Please use the data portal contact form for further inquiries; select DP4.00200.001 or any of the constituent data products listed above in the Data Product dropdown list and include the term “PRB0040480“ in the body of your request

Update 4: Microbial Community Composition Data Products Being Reprocessed

This is an update to the following past news posts:

In order to be more agile in identifying and correcting quality issues with the microbial community composition data products, we are developing an in-house data processing pipeline to generate the microbial community composition data products. Testing of the pipeline is currently underway and will be followed by parameter optimization. A firm timeline for completing testing and optimization is unclear: this news item will be updated periodically as development progresses.

This issue will be updated again August 17, 2020. Please use the data portal contact form for further inquiries and include the term “INC0033894“ in the body of your request.

Resolved: Faulty algorithm in Windspeed and direction above water on-buoy (DP1.20059.001)

Update to: Faulty algorithm in Windspeed and direction above water on-buoy (DP1.20059.001).

Buoy wind speed and direction data have been reprocessed with the wind direction-averaging algorithm fix through 2019 and instrument data reprocessing is presently continuing into 2020 as of July 1, 2020. However, an investigation on the reprocessed wind direction data identified another issue of inconsistent 2D wind sensor install orientations, causing unreliable buoy wind direction data. This issue will be addressed in further detail in a separate data portal news announcement. Buoy wind speed data remains unaffected. We recommend data users continue to use the 2D wind speed and direction (DP1.00001.001) at lake sites for reliable wind direction data.

Update: Notice of incorrect soil CO2 concentration data (DP1.00095.001)

Update: to Notice of incorrect soil CO2 concentration data (DP1.00095.001).

A fix has been developed for the issue of missing atmospheric pressure corrections for the soil CO2 concentration data product (DP1.00095.001). This will result in the pressure correction being performed if the necessary inputs are available or flagging the data if the pressure correction inputs are unavailable. Reprocessing previously collected data has begun with the expectation that the corrected data will start to be published by early July. In the meantime, all previously published soil CO2 concentration data has been flagged on the NEON data portal. An update will be provided in early July 2020.

Please use the data portal contact form for further inquiries; please select DP1.00095.001 in the Data Product dropdown list and include the term "PRB0040660" in the body of your request.

Spurious trace precipitation in Primary Precipitation data product (DP1.00006.001)

A review of the precipitation data product (part of to DP1.00006.001) identified that spurious trace precipitation is recorded frequently for all primary precipitation sensors (weighing gauges). While actual precipitation events are typically distinguishable from spurious events by visual inspection of the time series, the spurious events can add significantly to precipitation totals when aggregated over long intervals. Sums of spurious precipitation range from < 0.05 mm/day to > 0.5mm/day, with higher sums generally corresponding to sites and time periods of greater evaporative demand.  

We are working on additional quality control algorithms in order to detect and remove spurious trace events in the primary precipitation data product. An updated algorithm is expected sometime in 2021. An update to this issue will be provided by March, 2021. In the meantime, data remain available and notice of this issue has been added to the to data product description. Users are advised to manually inspect and clean the dataset of spurious trace precipitation events.

For further inquiries, please use the data portal contact form; please select DP1.00006.001 in the Data Product dropdown list and include the term "PRB0040676" in the body of your request.