Skip to content

Fix display of nested/vlen compound and variable-length string array datasets - #489

Closed
brtnfld wants to merge 9 commits into
HDFGroup:masterfrom
brtnfld:fix-nested-vlen-array-display
Closed

brtnfld wants to merge 9 commits into
HDFGroup:masterfrom
brtnfld:fix-nested-vlen-array-display

Conversation

@brtnfld

@brtnfld brtnfld commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Summary

This resolves the customer-reported issue that #474 set out to fix (HDFView failing to display compound datasets containing nested/variable-length members), and additionally fixes a case #474 did not handle: arrays of variable-length strings, including arrays of arrays of them.

This branch is built by taking #474's (nested_seq_cmpd) legitimate compound/vlen display fixes and replacing its H5Dread_VLStrings/H5Dwrite_VLStrings workaround for array-of-varstr with a fix at the actual root cause, plus fixing the underlying table-view indexing bug that workaround was papering over. If this is merged, #474 can be closed - the commits from that branch are included here (with original authorship preserved) as the first 8 commits, so this is not a duplicate/independent implementation.

Root cause of the array-of-varstr bug

H5Datatype.allocateArray()'s ARRAY branch allocated a flat, narrowly-typed buffer (e.g. String[]) by delegating to the base type's own allocation. But per the JNI's documented buffer data model (translate_rbuf()/translate_wbuf() in HDF5's h5util.c), an ARRAY reads as an ArrayList of its elements whenever the base is itself variable-length - a flat buffer can't hold that. The resulting type mismatch throws ArrayStoreException inside the JNI's SetObjectArrayElement call, and translate_rbuf's own exception handling (CHECK_JNI_EXCEPTION(ENVONLY, JNI_TRUE)) silently discards it, leaving the buffer empty - no error surfaces anywhere.

I confirmed this empirically with a standalone array-of-varstr reproducer built against current HDF5 develop: the generic H5DreadVL/H5DwriteVL path returns empty strings on read and throws h5validate_wbuf: expected a java.util.ArrayList element on write, with the unmodified buffer-allocation code. Once the caller passes the documented Object[] shape instead, the same JNI call reads and writes correctly - no JNI changes needed.

Changes on top of #474's commits (net a code reduction)

  • H5Datatype: new containsVlenOrVarStr(), mirroring the JNI's own recursive h5str_detect_vlen()/h5str_detect_vlen_str() (recurses through ARRAY bases and COMPOUND members to any depth). allocateArray's ARRAY branch now allocates Object[] instead of a flat buffer when the base needs it.
  • H5ScalarDS: read/write dispatch now uses the same recursive check instead of the one-level-deep isArray() && base.isVLEN() test, which missed ARRAY of ARRAY of variable-length string. Fix display for nested/vlen cmpds #474's H5Dread_VLStrings/H5Dwrite_VLStrings special-casing for arrays is reverted back to plain isVarStr() - not needed once the buffer shape is correct.
  • DataProviderFactory (ArrayDataProvider): the table view's flattened-offset indexing assumed one contiguous buffer spanning all rows, which cannot describe a List-per-row layout once nesting is involved (row 0 was pulling in row 1's data, row 1 went out of bounds, both showing *ERROR*). Added a container-based retrieval path that addresses nested List elements directly by index. The existing flat-buffer path is untouched and still used for plain numeric arrays.

Also two smaller fixes from review comments on #474 that apply independently of the array fix:

  • DefaultBaseTableView: showUnsafeWriteNotice()'s one-dialog-per-second throttle is unnecessary - Tools.showInformation() opens a modal dialog, so the triggering event isn't re-entered until the user dismisses it.
  • DefaultCompoundDSTableView: the cell-selection listener's early return keyed off unsafeForWrite was unrelated to write safety - it existed to dodge a baseIndexMap/expanded-column mismatch. Extracted the listener body into refineCellValuePreview() and replaced the flag check with a targeted try/catch around the actual failure mode.

Verification

Built HDF5 develop from source (Java bindings enabled) and tested against it end to end:

  • Direct H5ScalarDS/H5CompoundDS probes: array-of-varstr, array-of-array-of-varstr, and compound-of-vlen-compound (the original reported bug) all read and write correctly.
  • Actual HDFView table view (screenshots, clicked cells for full values): all three render correctly - e.g. the nested-array case shows [[r0e0a, r0e0b, r0e0c], [r0e1a, r0e1b, r0e1c]] exactly matching what was written, and the original compound-of-vlen file shows id/tags/nested columns with correct values - no more *ERROR* anywhere.
  • mvn test -pl object: 163/163 pass, no regressions.

mattjala and others added 9 commits September 11, 2026 09:54
(cherry picked from commit 575268f)
(cherry picked from commit e64602f)
(cherry picked from commit cd9c0ff)
(cherry picked from commit 5fbc86b)
(cherry picked from commit 8471a41)
(cherry picked from commit 6cd3296)
(cherry picked from commit 20b1c71)
…instead of H5Dread_VLStrings

Root cause: H5Datatype.allocateArray()'s ARRAY branch allocated a flat, narrowly-typed
buffer (e.g. String[]) by delegating to the base type's own allocation. But per the
JNI's documented buffer data model (translate_rbuf()/translate_wbuf() in h5util.c), an
ARRAY reads as an ArrayList of its elements when the base is itself variable-length -
a flat buffer can't hold that. The resulting type mismatch threw ArrayStoreException
inside the JNI, which translate_rbuf's own exception handling silently discards,
leaving the buffer empty. This reproduces on current HDF5 develop, unrelated to the
JNI fixes in #6413 - confirmed empirically with a standalone array-of-varstr
reproducer against both code paths.

Fixes, net a code reduction:
 - H5Datatype: new containsVlenOrVarStr() (mirrors h5str_detect_vlen()/
   h5str_detect_vlen_str() recursion through ARRAY bases and COMPOUND members).
   allocateArray's ARRAY branch now allocates Object[] instead of a flat buffer when
   the base needs it.
 - H5ScalarDS: read/write dispatch uses the same recursive check instead of a
   one-level-deep isArray()+base.isVLEN() test, which missed ARRAY of ARRAY of
   variable-length string. The H5Dread_VLStrings/H5Dwrite_VLStrings special-casing
   this PR added for arrays is no longer needed and is reverted back to plain
   isVarStr() - the underlying H5DreadVL/H5DwriteVL path is correct once given the
   right buffer shape.
 - DataProviderFactory (ArrayDataProvider): the table view's flattened-offset
   indexing assumed one contiguous buffer across all rows, which cannot describe a
   List-per-row layout once nesting is involved. Added a container-based retrieval
   path that addresses nested List elements directly by index; the existing
   flat-buffer path is untouched and still used for plain numeric arrays.

Also addresses two review comments from this PR:
 - DefaultBaseTableView: showUnsafeWriteNotice()'s one-dialog-per-second throttle is
   unnecessary - Tools.showInformation() opens a modal dialog, so the triggering
   event isn't re-entered until the user dismisses it.
 - DefaultCompoundDSTableView: the cell-selection listener's early return keyed off
   unsafeForWrite has nothing to do with write safety - it exists to dodge a
   baseIndexMap/expanded-column mismatch. Extracted the listener body into
   refineCellValuePreview() and replaced the flag check with a targeted try/catch
   around the actual failure mode, independent of whether the datatype is writable.

Verified: array-of-varstr, array-of-array-of-varstr, and compound-of-vlen-compound
all read/write correctly via direct H5ScalarDS/H5CompoundDS probes and render
correctly in the HDFView table view, against a from-source build of current HDF5
develop. All 163 existing object-module tests pass.
{
Object[] tempArray = new Object[(int)arraySize];

for (int i = 0; i < arraySize; i++) {
for (int j = 0; j < arrayDims.length; j++) {
log.trace("allocateArray(): Array dims[{}]={}", j, arrayDims[j]);

asize *= arrayDims[j];
catch (Exception ex) {
log.debug("CompoundDSCellSelectionListener: buildIndexMaps", ex);
}
baseIndexMap = maps[DataFactoryUtils.COL_TO_BASE_CLASS_MAP_INDEX];
if (val == null) {
cellValueField.setText("Null");
return;
if (val != null && ((String)val).compareTo("NULL") != 0) {
}
}
else if (valIsObjRef) {
if (val != null && ((String)val).compareTo("NULL") != 0) {
* only. For top-level CompoundData, this should be the entire width of the
* dataset. For nested CompoundData, nCols will be a subset of these columns.
*/
int nCols = (int)dataFormat.getWidth() * baseIndexMap.size();
* dataset. For nested CompoundData, nCols will be a subset of these columns.
*/
int nCols = (int)dataFormat.getWidth() * baseIndexMap.size();
int nRows = (int)dataFormat.getHeight();
cellLabel.setText(String.valueOf(rowIndex) + ", " + fieldName + colIndex + " = ");
String strVal = null;
if (valIsRegRef) {
boolean displayValues = ViewProperties.showRegRefValues();
@brtnfld

brtnfld commented Sep 11, 2026

Copy link
Copy Markdown
Contributor Author

Closing in favor of contributing directly to #474 (see mattjala#2) rather than opening a parallel PR that duplicates the same commits already proposed there. #474 remains the canonical PR for this fix; my array-of-varstr fix and the remaining review-comment fixes will land as additional commits on that branch.

@brtnfld brtnfld closed this Sep 11, 2026
@github-project-automation github-project-automation Bot moved this from To be triaged to Done in HDFView - TRIAGE & TRACK Sep 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants