Conversation
(cherry picked from commit 575268f)
(cherry picked from commit e64602f)
(cherry picked from commit cd9c0ff)
(cherry picked from commit 5fbc86b)
(cherry picked from commit 8471a41)
(cherry picked from commit f757785)
(cherry picked from commit 6cd3296)
(cherry picked from commit 20b1c71)
…instead of H5Dread_VLStrings Root cause: H5Datatype.allocateArray()'s ARRAY branch allocated a flat, narrowly-typed buffer (e.g. String[]) by delegating to the base type's own allocation. But per the JNI's documented buffer data model (translate_rbuf()/translate_wbuf() in h5util.c), an ARRAY reads as an ArrayList of its elements when the base is itself variable-length - a flat buffer can't hold that. The resulting type mismatch threw ArrayStoreException inside the JNI, which translate_rbuf's own exception handling silently discards, leaving the buffer empty. This reproduces on current HDF5 develop, unrelated to the JNI fixes in #6413 - confirmed empirically with a standalone array-of-varstr reproducer against both code paths. Fixes, net a code reduction: - H5Datatype: new containsVlenOrVarStr() (mirrors h5str_detect_vlen()/ h5str_detect_vlen_str() recursion through ARRAY bases and COMPOUND members). allocateArray's ARRAY branch now allocates Object[] instead of a flat buffer when the base needs it. - H5ScalarDS: read/write dispatch uses the same recursive check instead of a one-level-deep isArray()+base.isVLEN() test, which missed ARRAY of ARRAY of variable-length string. The H5Dread_VLStrings/H5Dwrite_VLStrings special-casing this PR added for arrays is no longer needed and is reverted back to plain isVarStr() - the underlying H5DreadVL/H5DwriteVL path is correct once given the right buffer shape. - DataProviderFactory (ArrayDataProvider): the table view's flattened-offset indexing assumed one contiguous buffer across all rows, which cannot describe a List-per-row layout once nesting is involved. Added a container-based retrieval path that addresses nested List elements directly by index; the existing flat-buffer path is untouched and still used for plain numeric arrays. Also addresses two review comments from this PR: - DefaultBaseTableView: showUnsafeWriteNotice()'s one-dialog-per-second throttle is unnecessary - Tools.showInformation() opens a modal dialog, so the triggering event isn't re-entered until the user dismisses it. - DefaultCompoundDSTableView: the cell-selection listener's early return keyed off unsafeForWrite has nothing to do with write safety - it exists to dodge a baseIndexMap/expanded-column mismatch. Extracted the listener body into refineCellValuePreview() and replaced the flag check with a targeted try/catch around the actual failure mode, independent of whether the datatype is writable. Verified: array-of-varstr, array-of-array-of-varstr, and compound-of-vlen-compound all read/write correctly via direct H5ScalarDS/H5CompoundDS probes and render correctly in the HDFView table view, against a from-source build of current HDF5 develop. All 163 existing object-module tests pass.
brtnfld
requested review from
jhendersonHDF,
lrknox and
mattjala
as code owners
September 11, 2026 15:00
| { | ||
| Object[] tempArray = new Object[(int)arraySize]; | ||
|
|
||
| for (int i = 0; i < arraySize; i++) { |
| for (int j = 0; j < arrayDims.length; j++) { | ||
| log.trace("allocateArray(): Array dims[{}]={}", j, arrayDims[j]); | ||
|
|
||
| asize *= arrayDims[j]; |
| catch (Exception ex) { | ||
| log.debug("CompoundDSCellSelectionListener: buildIndexMaps", ex); | ||
| } | ||
| baseIndexMap = maps[DataFactoryUtils.COL_TO_BASE_CLASS_MAP_INDEX]; |
| if (val == null) { | ||
| cellValueField.setText("Null"); | ||
| return; | ||
| if (val != null && ((String)val).compareTo("NULL") != 0) { |
| } | ||
| } | ||
| else if (valIsObjRef) { | ||
| if (val != null && ((String)val).compareTo("NULL") != 0) { |
| * only. For top-level CompoundData, this should be the entire width of the | ||
| * dataset. For nested CompoundData, nCols will be a subset of these columns. | ||
| */ | ||
| int nCols = (int)dataFormat.getWidth() * baseIndexMap.size(); |
| * dataset. For nested CompoundData, nCols will be a subset of these columns. | ||
| */ | ||
| int nCols = (int)dataFormat.getWidth() * baseIndexMap.size(); | ||
| int nRows = (int)dataFormat.getHeight(); |
| cellLabel.setText(String.valueOf(rowIndex) + ", " + fieldName + colIndex + " = "); | ||
| String strVal = null; | ||
| if (valIsRegRef) { | ||
| boolean displayValues = ViewProperties.showRegRefValues(); |
Contributor
Author
|
Closing in favor of contributing directly to #474 (see mattjala#2) rather than opening a parallel PR that duplicates the same commits already proposed there. #474 remains the canonical PR for this fix; my array-of-varstr fix and the remaining review-comment fixes will land as additional commits on that branch. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This resolves the customer-reported issue that #474 set out to fix (HDFView failing to display compound datasets containing nested/variable-length members), and additionally fixes a case #474 did not handle: arrays of variable-length strings, including arrays of arrays of them.
This branch is built by taking #474's (
nested_seq_cmpd) legitimate compound/vlen display fixes and replacing itsH5Dread_VLStrings/H5Dwrite_VLStringsworkaround for array-of-varstr with a fix at the actual root cause, plus fixing the underlying table-view indexing bug that workaround was papering over. If this is merged, #474 can be closed - the commits from that branch are included here (with original authorship preserved) as the first 8 commits, so this is not a duplicate/independent implementation.Root cause of the array-of-varstr bug
H5Datatype.allocateArray()'sARRAYbranch allocated a flat, narrowly-typed buffer (e.g.String[]) by delegating to the base type's own allocation. But per the JNI's documented buffer data model (translate_rbuf()/translate_wbuf()in HDF5'sh5util.c), anARRAYreads as anArrayListof its elements whenever the base is itself variable-length - a flat buffer can't hold that. The resulting type mismatch throwsArrayStoreExceptioninside the JNI'sSetObjectArrayElementcall, andtranslate_rbuf's own exception handling (CHECK_JNI_EXCEPTION(ENVONLY, JNI_TRUE)) silently discards it, leaving the buffer empty - no error surfaces anywhere.I confirmed this empirically with a standalone array-of-varstr reproducer built against current HDF5
develop: the genericH5DreadVL/H5DwriteVLpath returns empty strings on read and throwsh5validate_wbuf: expected a java.util.ArrayList elementon write, with the unmodified buffer-allocation code. Once the caller passes the documentedObject[]shape instead, the same JNI call reads and writes correctly - no JNI changes needed.Changes on top of #474's commits (net a code reduction)
H5Datatype: newcontainsVlenOrVarStr(), mirroring the JNI's own recursiveh5str_detect_vlen()/h5str_detect_vlen_str()(recurses throughARRAYbases andCOMPOUNDmembers to any depth).allocateArray'sARRAYbranch now allocatesObject[]instead of a flat buffer when the base needs it.H5ScalarDS: read/write dispatch now uses the same recursive check instead of the one-level-deepisArray() && base.isVLEN()test, which missedARRAYofARRAYof variable-length string. Fix display for nested/vlen cmpds #474'sH5Dread_VLStrings/H5Dwrite_VLStringsspecial-casing for arrays is reverted back to plainisVarStr()- not needed once the buffer shape is correct.DataProviderFactory(ArrayDataProvider): the table view's flattened-offset indexing assumed one contiguous buffer spanning all rows, which cannot describe a List-per-row layout once nesting is involved (row 0 was pulling in row 1's data, row 1 went out of bounds, both showing*ERROR*). Added a container-based retrieval path that addresses nestedListelements directly by index. The existing flat-buffer path is untouched and still used for plain numeric arrays.Also two smaller fixes from review comments on #474 that apply independently of the array fix:
DefaultBaseTableView:showUnsafeWriteNotice()'s one-dialog-per-second throttle is unnecessary -Tools.showInformation()opens a modal dialog, so the triggering event isn't re-entered until the user dismisses it.DefaultCompoundDSTableView: the cell-selection listener's early return keyed offunsafeForWritewas unrelated to write safety - it existed to dodge abaseIndexMap/expanded-column mismatch. Extracted the listener body intorefineCellValuePreview()and replaced the flag check with a targeted try/catch around the actual failure mode.Verification
Built HDF5
developfrom source (Java bindings enabled) and tested against it end to end:H5ScalarDS/H5CompoundDSprobes: array-of-varstr, array-of-array-of-varstr, and compound-of-vlen-compound (the original reported bug) all read and write correctly.[[r0e0a, r0e0b, r0e0c], [r0e1a, r0e1b, r0e1c]]exactly matching what was written, and the original compound-of-vlen file showsid/tags/nestedcolumns with correct values - no more*ERROR*anywhere.mvn test -pl object: 163/163 pass, no regressions.