OCI: offer the H100, H200, L40S, A100-v2, B200, B300 and RTX PRO shapes - #4302
Open
fede-kamel wants to merge 1 commit into
Open
fede-kamel wants to merge 1 commit into
fede-kamel wants to merge 1 commit into
Conversation
`SUPPORTED_SHAPE_FAMILIES` in the OCI backend still listed only the GPU shapes that existed when Bare Metal support was added (dstackai#1325): GPU3, GPU4 and A10. Every newer NVIDIA shape from the gpuhunt catalog was filtered out by `_supported_instances`, so `dstack apply` could never select an OCI H100, H200, L40S, A100-v2, B200, B300 or RTX PRO 6000 instance even with quota in place. dstackai#2343 asked for H100 and was closed as not a priority; the only real blocker at the time, Fabric Manager in the OS images (dstackai#2348), was fixed in dstackai#2355, and the images now install the open kernel modules and Fabric Manager for the whole CUDA driver line. Add those families. They are all x86-64 hosts with NVIDIA GPUs, which is what the published `dstack-cuda-*` Marketplace images run. Two groups of GPU shapes from the catalog are intentionally still left out, with the reason recorded next to the list so it is not mistaken for an omission: * BM.GPU.MI300X.8 and BM.GPU.MI355X.8 (AMD Instinct): dstack publishes no ROCm image in OCI Marketplace, and `create_instance` picks the CUDA image for every GPU offer. * BM.GPU.GB200.4 and BM.GPU.GB300.4 (Grace superchips): arm64 hosts, and the OCI images are x86-64 only. Also document why `SUPPORTED_REGIONS` is a fixed set of seven regions: the Marketplace community listings that carry the dstack images are regional, and those are the regions they are published in (verified: the `dstack-cuda-0.14` listing is visible in us-chicago-1 and absent in ca-toronto-1). Widening the set requires publishing the images first. Note that the catalog side needs dstackai/gpuhunt to collect the newer shapes; without that, this change only affects H100, H200, L40S and A100-v2, which the catalog already carries.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #4301.
SUPPORTED_SHAPE_FAMILIESin the OCI backend still lists only the GPU shapes that existed when Bare Metal support landed (#1325): GPU3, GPU4, A10. Every newer NVIDIA shape from the gpuhunt catalog was filtered out by_supported_instances, sodstack applycould never select an OCI H100, H200, L40S, A100-v2, B200, B300 or RTX PRO 6000 even with quota. #2343 asked for H100; its blocker (Fabric Manager, #2348) was fixed in #2355 and the images now install open kernel modules and Fabric Manager.This adds those families (all x86-64 hosts with NVIDIA GPUs, which is what the published
dstack-cuda-*Marketplace images run). Deliberately still excluded, with the reason recorded next to the list and in tests: AMD Instinct shapes (no ROCm image is published, andcreate_instancealways picks the CUDA image) and Grace GB200/GB300 shapes (arm64 hosts; the images are x86-64 only). It also documents whySUPPORTED_REGIONSis fixed: the Marketplace listings carrying the images are regional (verified:dstack-cuda-0.14present in us-chicago-1, absent in ca-toronto-1), so widening requires publishing images first.The catalog side is in dstackai/gpuhunt (dstackai/gpuhunt#262). Without it, this change already affects H100, H200, L40S and A100-v2, which the catalog carries today; B300 and RTX PRO 6000 appear once the gpuhunt change is merged and the hourly catalog republishes. Checked with dstack's pinned gpuhunt 0.1.30 that the new catalog rows convert through
catalog_item_to_offerwithout error.Tests: new
src/tests/_internal/core/backends/oci/test_compute.py(13 supported / 10 unsupported shapes); OCI backend suite 43 passed, whole backends suite 473 passed; ruff clean.