July 2026 - Blablador-news - lists.fz-juelich.de

Kimi-K3
by Strube, Alexandre 30 Jul '26

30 Jul '26

TLDR: It’s here, and it will suck for you FOR NOW. IT WILL GET BETTER. So, we have managed to have Kimi-K3 running on Blablador. As far as I can tell, this is the only lab in the world which managed to have it running on A100 GPUs in any production-like deployment. Sure, there are people claiming that they can run it on a Mac Mini. It’s like saying that I have seen, dunno, Taylor Swift, when all I saw was a tattoo of her face on someone’s leg. Sure, fam. Go on. We are running the full thing. No quantizations, no optimizations, no tricks. Just as Moonshot AI recommended. Except we are using hardware they never expected it to run. For that, we did a number of crazy things here. I will document this eventually - it deserves at least a blog post, better a paper. Anyway, what I wanted to say is: hold your horses. This is EXPERIMENTAL. This is SLOW. Kimi has always been extremely verbose. It might take HOURS to get a reply. The replies are good, though. What I mean is: don’t point your agents to it and complain later that the thing takes forever. It does. And I will be killing the deployment multiple times before my vacations are over. It won’t be there as available as Qwen is. That being said, and I hope I tone down the expectations, the model is GOOD. It can go for hours on tasks, slowly for now. And, as I said, I will be killing the deployment to run experiments. Don’t expect stability for now. As it gets better, it will become part of our permanent roster of models. I also want it. Keep barking, Alex, the one who barks for random dogs and they bark back

1 0

About support tickets and latest models.
by Strube, Alexandre 29 Jul '26

29 Jul '26

Hello Blabladores, As my vacations are about to end, I would like to address a couple things I have noticed on the many unanswered tickets (well, I also deserve a rest every decade or so) about models, the current status of things and where we are going. - GLM 5.2 doesn’t fly well, and gets lost in long works: We know. The issue is that GLM is not made to run on the hardware we have - in fact, it asks for gpus 2 generations newer than the current A100 we have available for it. Remember, this is a trillion-sized parameter model to begin with. It won’t run on small stuff. In order to get it running, we used a number of patches on the underlying libraries, and used 4-bit quantization. Both would degrade the model’s quality in ways we can’t really measure unless we would be able to isolate each. This is not a job to be done away from Jülich on spare hours in the middle of the ocean - *EVERYONE* wants kimi-K3. In fact, I got requests for it way before it was even released. Same story: its minimum hardware requirements are 16 GB200 gpus. This is more than a million euros for a single model - without counting multiple users! Remember, this is a 2.8T parameter model. Blablador itself has been helping me make a special CUDA kernel so it would run on A100…. But it would require 1024 (!) gpus to run it at full context size and able to serve half a dozen users simultaneously… This is not a toy. I’m on it. I already have a preliminary kernel compiled, and there’s still stuff to do, and I hope Blablador helps me with that. But getting 1024 gpus available for testing is not something I can just dig every minute :-) There are more news coming soon, and not just about models. And it’s good news! :-) Best, Alex

1 0

Kimi-K3
by Strube, Alexandre 27 Jul '26

27 Jul '26

Kimi-K3 came out 5 minutes ago. It’s downloading for 4 minutes already. I don’t promise we’ll have it today, but it won’t be for lack of trying! Keep barking! Alex

1 0