feat: heterogeneous Multi-Backend support (CUDA/ROCm) for Linux environments#545
Open
joanfgarcia wants to merge 5 commits intomicrosoft:mainfrom
Open
feat: heterogeneous Multi-Backend support (CUDA/ROCm) for Linux environments#545joanfgarcia wants to merge 5 commits intomicrosoft:mainfrom
joanfgarcia wants to merge 5 commits intomicrosoft:mainfrom
Conversation
…nd distillation envelope
This is a Red Pill project roadmap document, not a BitNet contribution. Relocated to the parent project where it belongs.
Red Pill project benchmark, not a BitNet upstream contribution.
Summary of changes: - Decoupled BitNet kernels from ggml.c into src/ggml-bitnet-mad.cpp - Established Sovereign Bridge in include/ggml-bitnet.h - Fixed CPU multi-threading telemetry (thread-aware debug) - Resolved CPU SEGFAULT via static buffer and internal quantization - Verified stable inference on Falcon3-10B-1.58b
Author
|
@microsoft-github-policy-service agree |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Abstract:
This PR extends the official
bitnet.cppinference framework to support full GPU acceleration on Linux via thellama.cppbackend. It stabilizes the interaction between the BitNet frontend and the underlyingggmlkernels, enabling seamless, high-performance execution on NVIDIA and AMD hardware.Highlights:
I2_Sdequantization kernels provided by thellama.cppsubmodule.Testing & Validation:
Validation was conducted across a diverse set of hardware (Mixed-GPU clusters). Local benchmark scripts can be found in
tests/bitnet_sovereign_bench.pyfor verification.References:
ggmladditions available in ourllama.cppfork: joanfgarcia/llama.cpp (fork).Acknowledgments:
Thank you for providing the community with these groundbreaking 1-bit LLM foundations. This contribution is offered with respect and professional commitment.
A Lannister always pays his debts.