feat(deps): upgrade dependencies across TensorFlow.NET - #1303
feat(deps): upgrade dependencies across TensorFlow.NET#1303samuelcaldas wants to merge 7 commits into
Conversation
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
… and CLAUDE.md Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
…ads, DTypes) - Upgrade TargetFrameworks to net8.0;net9.0 across core and test projects - Add P/Invoke bindings for TF_SetPayload, TF_ForEachPayload, TF_SetStatusFromIOError - Add P/Invoke bindings for TF_TensorBitcastFrom, TF_TensorIsAligned, TF_TensorDefaultAlignment, TF_TensorElementCount - Add FP8 and Sub-Byte DTypes to TF_DataType aligned with TF 2.22 SOT - Fix float16 / bfloat16 mapping bug in dtypes.cs - Resolve C# 13 operator ambiguity in GradientTest and GradientEagerTest Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
- Upgrade MethodBoundaryAspect.Fody to 2.0.150 - Upgrade Newtonsoft.Json to 13.0.4 - Upgrade OneOf to 3.0.271 - Upgrade Razorvine.Pickle to 1.5.0 - Upgrade Serilog.Sinks.Console to 6.0.0 - Upgrade System.Memory to 4.6.3 (netstandard2.0) - Upgrade HDF5-CSharp to 1.19.1 - Upgrade SharpCompress to 0.39.0 - Upgrade Microsoft.NET.Test.Sdk to 17.13.0 across test suites - Upgrade coverlet.collector to 6.0.4 across test suites - Upgrade BenchmarkDotNet to 0.14.0 and synchronize SciSharp.TensorFlow.Redist to 2.16.0 - Upgrade Microsoft.CodeAnalysis.CSharp.Scripting to 4.12.0 Co-Authored-By: Claude Code <noreply@anthropic.com>
There was a problem hiding this comment.
🟡 Changes recommended
A newly introduced TF_STRING dtype size of 8 bytes is inconsistent with the repo’s own TF_STRING storage/allocation (TF_TString=24 bytes) and can cause incorrect byte-length/element-count behavior, plus some new docs are inconsistent/duplicative.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Upgrades solution dependencies and target frameworks to modernize TensorFlow.NET (notably moving tools/tests to .NET 8 and core libraries to multi-target netstandard2.0/net8.0/net9.0), while adding newer TensorFlow C-API bindings and dtype support needed for newer TensorFlow runtime parity.
Changes:
- Retarget tools and test projects from
net6.0tonet8.0and upgrade several dev/test NuGet dependencies (BenchmarkDotNet, coverlet, Microsoft.NET.Test.Sdk, etc.). - Expand core TensorFlow C-API/dtype surface area (new TF_DataType enum members, dtype name mappings, size logic updates, and new status/tensor interop methods).
- Add/update roadmap and contributor guidance documentation (ROADMAP/README/CLAUDE).
File summaries
| File | Description |
|---|---|
| tools/TensorFlowNET.Console/Tensorflow.Console.csproj | Retarget console tool to .NET 8. |
| tools/TensorFlowNET.Benchmarks/Tensorflow.Benchmark.csproj | Retarget benchmarks to .NET 8 and bump BenchmarkDotNet + redist. |
| tools/Tensorflow.Redist.NativeLibrarySplitter/Tensorflow.Redist.NativeLibrarySplitter.csproj | Retarget splitter tool to .NET 8. |
| tools/Tensorflow.CodeGen/Tensorflow.CodeGen.csproj | Retarget codegen tool to .NET 8 and bump Roslyn scripting package. |
| test/TensorFlowNET.UnitTest/Tensorflow.Binding.UnitTest.csproj | Retarget unit tests to .NET 8 and bump test SDK/Fody. |
| test/TensorFlowNET.UnitTest/ManagedAPI/GradientTest.cs | Adjust gradient test expression to resolve tensor/operator typing. |
| test/TensorFlowNET.UnitTest/GradientTest/GradientEagerTest.cs | Adjust eager gradient test expression to resolve tensor/operator typing. |
| test/TensorFlowNET.Native.UnitTest/Tensorflow.Native.UnitTest.csproj | Retarget native tests to .NET 8 and bump test SDK + coverlet. |
| test/TensorFlowNET.Keras.UnitTest/Tensorflow.Keras.UnitTest.csproj | Retarget Keras tests to .NET 8 and bump test SDK + coverlet. |
| test/TensorflowNET.Hub.Unittest/Tensorflow.Hub.Unittest.csproj | Retarget Hub tests to .NET 8 and bump test SDK + coverlet. |
| test/TensorFlowNET.Graph.UnitTest/TensorFlowNET.Graph.UnitTest.csproj | Retarget graph tests to .NET 8 and bump test SDK + coverlet. |
| test/Tensorflow.UnitTest/Tensorflow.UnitTest.csproj | Retarget tests to .NET 8 and bump test SDK + coverlet. |
| test/TensorFlow.Kernel.UnitTest/TensorFlow.Kernel.UnitTest.csproj | Retarget kernel tests to .NET 8 and bump test SDK + coverlet. |
| src/TensorFlowNET.Text/Tensorflow.Text.csproj | Multi-target Text package for netstandard2.0/net8.0/net9.0. |
| src/TensorFlowNET.Recommenders/Tensorflow.Recommenders.csproj | Multi-target Recommenders package for netstandard2.0/net8.0/net9.0. |
| src/TensorFlowNET.Keras/Tensorflow.Keras.csproj | Update TFMs + warnings suppression and bump package references. |
| src/TensorflowNET.Hub/Tensorflow.Hub.csproj | Update TFMs and bump SharpCompress. |
| src/TensorFlowNET.Core/Tensors/TF_DataType.cs | Add newer TF_DataType enum values (FP8/sub-byte types). |
| src/TensorFlowNET.Core/Tensors/dtypes.cs | Add dtype aliases/mappings and extend dtype size + classification logic. |
| src/TensorFlowNET.Core/Tensors/c_api.tensor.cs | Add new tensor-related C-API P/Invokes (alignment, element count, bitcast). |
| src/TensorFlowNET.Core/Tensorflow.Binding.csproj | Multi-target core binding and bump dependency versions; add warning suppression for net8/9. |
| src/TensorFlowNET.Core/Status/Status.cs | Add managed wrappers for new TF_Status payload/I/O helpers. |
| src/TensorFlowNET.Core/Status/c_api.status.cs | Add status payload + I/O-related C-API P/Invokes. |
| ROADMAP.md | Add a roadmap document describing modernization/parity phases. |
| README.md | Update version relationship table and link to ROADMAP.md. |
| docs/plans/zippy-puzzling-knuth.md | Add a second roadmap-like document under docs/plans. |
| Directory.Build.props | Set repo-wide AllowUnsafeBlocks and LangVersion defaults. |
| CLAUDE.md | Add contributor guidance for build/test and architecture notes. |
Review details
Suppressed comments (1)
ROADMAP.md:46
- The numeric enum values listed for FP8/sub-byte dtypes here don't match the actual TF_DataType values used by the TensorFlow C-API (and the values added in src/TensorFlowNET.Core/Tensors/TF_DataType.cs in this PR). Hard-coding incorrect numbers in the roadmap will cause confusion and future mistakes.
- **Dynamic Multidimensional Slicing (`src/TensorFlowNET.Core/Tensors/Tensor.Indexing.cs`)**:
- Add full support for Ellipsis (`...`), `NewAxis` / `None`, negative step strides, and dynamic tensor-valued slice indices using the native `StridedSlice` kernel.
- Files reviewed: 30/30 changed files
- Comments generated: 4
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| public static int get_datatype_size(this TF_DataType type) | ||
| => type.as_base_dtype() switch | ||
| { | ||
| TF_DataType.TF_STRING => 8, |
| ## Build and Test Commands | ||
|
|
||
| ### Prerequisites | ||
| - Target framework: `.NET 6.0` / `.NET Standard 2.0` |
| ## Architectural Decisions & Foundations (Settled Design Tree) | ||
|
|
||
| 1. **Target SOT Baseline**: Staged rollout starting with **TensorFlow 2.16 LTS** (Milestone 1: Keras 3 engine + C-API modernization) progressing to **TensorFlow 2.22** (Milestone 2: Sub-byte FP8/Int4 quantization + latest TSL kernels). | ||
| 2. **Runtime Framework**: Pure **.NET 8.0 / .NET 9.0** baseline. Drops `.NET Standard 2.0` and `.NET 6.0` legacy constraints to leverage `System.Runtime.Intrinsics` (AVX-512 / ARM Neon), zero-copy `Span<T>` / `Memory<T>`, and native `Half` / `Int128`. |
| # Comprehensive Roadmap: Next Version Release of TensorFlow.NET (TF 2.16 - 2.22+ Parity) | ||
|
|
||
| ## Context & Objectives | ||
| This document establishes the official engineering roadmap to release the next major generation of **TensorFlow.NET** (`TensorFlow.NET`, `TensorFlow.Keras`, and `SciSharp.TensorFlow.Redist`). | ||
|
|
||
| The overarching goal is to achieve architectural, mathematical, and behavioral parity with modern Python TensorFlow (v2.16 LTS through v2.22+ located in `refs/py-tensorflow-sot`), transitioning to a modern **.NET 8.0 / .NET 9.0** high-performance runtime baseline while preserving backward compatibility for existing enterprise consumers. | ||
|
|
Summary of Changes
Modernizes and upgrades external NuGet package dependencies across the entire TensorFlow.NET solution targeting .NET Standard 2.0 / .NET 8.0 / .NET 9.0:
1. Core Runtime Libraries (
src/)MethodBoundaryAspect.Fody: 2.0.149 -> 2.0.150Newtonsoft.Json: 13.0.3 -> 13.0.4OneOf: 3.0.263 -> 3.0.271Razorvine.Pickle: 1.4.0 -> 1.5.0Serilog.Sinks.Console: 4.1.0 -> 6.0.0System.Memory(netstandard2.0): 4.5.5 -> 4.6.3HDF5-CSharp: 1.19.0 -> 1.19.1SharpCompress: 0.34.1 -> 0.39.02. Test Suites (
test/)Microsoft.NET.Test.Sdk: 17.6.3 -> 17.13.0 across all test projectscoverlet.collector: 3.1.2/3.2.0 -> 6.0.4 across all test projectsMethodBoundaryAspect.Fody: 2.0.149 -> 2.0.1503. Developer Tools & Benchmarks (
tools/)BenchmarkDotNet: 0.13.5 -> 0.14.0SciSharp.TensorFlow.Redist: 2.11.4 -> 2.16.0 (aligned withUnitTest.RedistHolder)Microsoft.CodeAnalysis.CSharp.Scripting: 4.6.0-1.final -> 4.12.0Verification
🤖 Generated with Claude Code