NCO Homepage

Known Problems through 2021 (version 4.9.9)

Generic Run-time Problems:

netCDF CDF5 corruption: netCDF libraries 4.4.0+ supports the CDF5 binary format. Unfortunately the CDF5 implementation is buggy for large (> 4 GiB) variables in library versions 4.4.0–4.6.0. We identified this netCDF CDF5 bug on 20170906. Writing CDF5 files with large variables is buggy unless there is only one such “large” variable and it is the last to be defined. If the file is an input dataset (i.e., NCO reads it) written by PnetCDF then the input data are fine (because PnetCDF writes CDF5 through a different mechanism than serial programs like NCO’s writer). And if the CDF5 dataset was originally written by any netCDF version 4.6.1 or greater, then it is unaffected by this bug. However, a CDF5 input file with large variables written by any serial netCDF writer (like NCO) employing netCDF library 4.4.0–4.6.0, is likely corrupt and variables were silently truncated when writing it. Output files (that NCO wrote) with large variables will definitely be corrupt if NCO was linked to netCDF library version 4.4.0–4.6.0 (so upgrade to netCDF 4.6.1+ ASAP). Here are two potential workarounds for data affected by this bug: 1. Re-write (using any netCDF version) original input files in netCDF4 format instead of CDF5, then process these as normal and write netCDF4 output (instead of CDF5); 2. Re-compile NCO with netCDF library 4.6.1+ or later and use it to convert non-corrupt datasets to netCDF4 format, then process the data. For more information on this nasty bug, see here and here. UPDATE: Unidata released netCDF 4.5.0 on 20171020. Unfortunately the patch to fix the CDF5 bug was not included. UPDATE: Unidata released netCDF 4.6.0 on 20180124. Unfortunately the patch to fix the CDF5 bug was not included. UPDATE: Unidata released netCDF 4.6.1 on 20180319. netCDF version 4.6.1 includes the patch that fixes the CDF5 bug. Yay!

netCDF4 Strided Hyperslab bug: Multiple users complain that access to strided hyperslabs of netCDF4 datasets is orders of magnitude slower than expected. This occurs with NCO and also with related software like NCL. The cause appears to be that all recent versions of netCDF up to 4.3.3 access strided hyperslabs via an algorithm (in nc_get_vars()) that becomes unwieldy and error-prone for large datasets. We developed and implemented a transparent workaround (that avoids the troublesome algorithm) for the most common case which is where strides are taken in only one dimension, e.g., -d time,0,,12. With the workaround introduced in NCO 4.4.6, strided access to netCDF4 datasets now completes in nearly the same amount of time as non-strided. This workaround works transparently with any version of netCDF. We are not yet sure that we have fully and correctly diagnosed the cause nor that our workaround is always effective. Comments welcome. Updates will be posted right here.

Operator-specific Run-time Problems:

Hyperslabbing masked and/or weighted variables bug: Versions 4.2.4—4.6.7 of ncwa incorrectly handle masking and weighting of hyperslabbed variables. Performing these actions simultaneously causes subtle arithmetic errors (the worst kind!), unless the hyperslab happens to start with the first element of the variable. In other words, results from commands of the form ncwa -d ... -m ... and ncwa -d ... -w ... are suspect. The workaround is to downgrade to NCO 4.2.3. The solution is to upgrade to NCO 4.6.8.

Minimization/maximization of packed variables bug: Versions 4.3.y—4.4.5 of ncwa incorrectly handled packed variables during these operations. The two workarounds are to unpack first or to perform the statistics in single precision with the --flt option. The solution is to upgrade to NCO 4.4.6.

Promoting packed records with missing values bug: Versions 4.3.X—4.4.5 of ncra could produce (wildly) inaccurate statistics when promoting (e.g., to double- from single-precision) variables that are packed and that have missing values. The two workarounds are to unpack first or to perform the statistics in single precision with the --flt option. The solution is to upgrade to NCO 4.4.6.

Chunking while hyperslabbing bug: Versions 4.3.X—4.4.4 of most operators could send incorrect chunking requests to the netCDF library, resulting in failures. This occurred only while simultaneously hyperslabbing. The solution is to upgrade to NCO 4.4.5.

ncwa mask condition bug: All versions through 4.4.3 of ncwa could return incorrect mask values for negative numbers. Thanks to Keith Lindsay for report, and Henry Butowsky for the fix. Prior to this fix, the ncwa lexer would drop the negative sign, if any, from the comparators appearing in the mask condition, e.g., ncwa --mask_condition "lat < -20" was parsed as "lat < 20" not "lat < -20". Hence, users of ncwa --mask_condition (or of -B) should upgrade. NB: The -m -M -T form of ncwa masking is/was not buggy. Thus the workaround is to use the -m -M -T form of ncwa masking, while the long-term solution is to upgrade to NCO 4.4.4+.

ncra, ncea, and ncrcat file close bug: Versions 4.3.9—4.4.0 of ncra, ncea, and ncrcat failed to correctly close and optionally remove input files. This could cause NCO to exceed system limits on the maximum number of open files when hundreds-to-thousands of input files were specified per NCO invocation. The exact failure point is OS-dependent (NCO commands on Mac OS X 10.9 would fail when processing more than 256 files at a time). This is embarassing because NCO has always been designed to work with arbitrary numbers of input files and we want power users to be comfortable running it on hundreds of thousands of input files. The workaround is to avoid versions 4.3.9—4.4.0, while the long-term solution is to upgrade to NCO 4.4.1+.

ncra MRO missing value bug: Versions 4.3.6—4.3.9 of ncra could treat missing values incorrectly during double-precision arithmetic. A symptom was that missing values could be replaced by strange numbers like, well, infinity or zero. This mainly affects ncra in MRO (multi-record output) mode, and the symptoms should be noticeable. The workaround is to run the affected versions of ncra using the --flt switch, so that single-precision floating point numbers are not promoted prior to arithmetic. The solution is to upgrade to NCO 4.4.0+.

ncwa hyperslabbing while averaging bug: Versions 4.3.3—4.3.5 of ncwa could return incorrect answers when user-specified hyperslabs were simultaneously extracted. In such cases, hyperslab limits were not consistently applied. This could produce incorrect answers that look correct. This bug only affected hyperslabbed statistics (those produced by simultaneously invoking -a and -d switches); “global averages” were unaffected. We urge all ncwa users to upgrade to NCO 4.3.6+.

ncpdq unpacking bug with auxiliary coordinates: Versions 4.3.2–4.3.3 of ncpdq did not correctly store unpacked variables. These versions unpacked (when specified with -U or -P upk) the values, but inadvertently stored their original packing attributes with the unpacked values. This would lead further operators to assume that the values were still packed. Hence consecutive operations could lead to incorrect values. Fixed in version 4.3.4. All ncpdq users are encouraged to upgrade. NB: this bug did not affect the other arithmetic operators which unpack data prior to arithmetic.

Platform-specific Run-time Problems: