This post spurred me to look into Cerebras, and its approach is quite interesting. Its chips get full performance regardless of the batch size, which would make them ideal for deep RL experiments, unlike GPUs which are only suitable for supervised learning. But with GPUs you can at least buy or rent them them if you have the money, while the Cerebras' chip costs around 5 million according to what I've read on the ML sub.
So buying them is out for me and most other people, but I wonder if it would be possible to rent a part of them for small scale research costing no more than 1k per month?
My guess is that it wouldn't, but does anybody know for sure?
I'll be curious to see costs on H100 when they're more available. A100 is getting a bit old.
seems like you really enjoyed NeurIPS! have enjoyed your many tweet.s interesting aside - both Jasper and Cerebras are Foundation Capital companies.
This post spurred me to look into Cerebras, and its approach is quite interesting. Its chips get full performance regardless of the batch size, which would make them ideal for deep RL experiments, unlike GPUs which are only suitable for supervised learning. But with GPUs you can at least buy or rent them them if you have the money, while the Cerebras' chip costs around 5 million according to what I've read on the ML sub.
So buying them is out for me and most other people, but I wonder if it would be possible to rent a part of them for small scale research costing no more than 1k per month?
My guess is that it wouldn't, but does anybody know for sure?