How many 3090 is too many 3090? Trick question - never enough 🤖.
text
How many 3090 is too many 3090? Trick question - never enough 🤖.
new_line
new_line
Working on rewiring for a 5th this weekend so need the quad in 4U to make this happen.  Definitely pushing air cooling to its limit but NVLink requires aggressive spacing between the cards.
text
Working on rewiring for a 5th this weekend so need the quad in 4U to make this happen.  Definitely pushing air cooling to its limit but NVLink requires aggressive spacing between the cards.
new_line
new_line
Fortunately up here in Canada our power is quite cheap, so stacking compute this way works.  While being annoying in terms of physical constraint the NVlink bridges really helps squeeze these cards - on smaller models at big batch the difference at -tp 2 can be as much as 50%. It's not bandwidth, it's latency! 
text
Fortunately up here in Canada our power is quite cheap, so stacking compute this way works.  While being annoying in terms of physical constraint the NVlink bridges really helps squeeze these cards - on smaller models at big batch the difference at -tp 2 can be as much as 50%. It's not bandwidth, it's latency! 
How many 3090 is too many 3090? Trick question - never enough 🤖.

Working on rewiring for a 5th this weekend so need the quad in 4U to make this happen.  Definitely pushing air cooling to its limit but NVLink requires aggressive spacing between the cards.

Fortunately up here in Canada our power is quite cheap, so stacking compute this way works.  While being annoying in terms of physical constraint the NVlink bridges really helps squeeze these cards - on smaller models at big batch the difference at -tp 2 can be as much as 50%. It's not bandwidth, it's latency! 
🔥
John6666
66719e03a4dd9ce916a82688
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/Tww_y0Qu_cfTnpk8_r6xI.png
CanQi Jin
likewendy
user