Hello Blabladores, As my vacations are about to end, I would like to address a couple things I have noticed on the many unanswered tickets (well, I also deserve a rest every decade or so) about models, the current status of things and where we are going. - GLM 5.2 doesn’t fly well, and gets lost in long works: We know. The issue is that GLM is not made to run on the hardware we have - in fact, it asks for gpus 2 generations newer than the current A100 we have available for it. Remember, this is a trillion-sized parameter model to begin with. It won’t run on small stuff. In order to get it running, we used a number of patches on the underlying libraries, and used 4-bit quantization. Both would degrade the model’s quality in ways we can’t really measure unless we would be able to isolate each. This is not a job to be done away from Jülich on spare hours in the middle of the ocean - *EVERYONE* wants kimi-K3. In fact, I got requests for it way before it was even released. Same story: its minimum hardware requirements are 16 GB200 gpus. This is more than a million euros for a single model - without counting multiple users! Remember, this is a 2.8T parameter model. Blablador itself has been helping me make a special CUDA kernel so it would run on A100…. But it would require 1024 (!) gpus to run it at full context size and able to serve half a dozen users simultaneously… This is not a toy. I’m on it. I already have a preliminary kernel compiled, and there’s still stuff to do, and I hope Blablador helps me with that. But getting 1024 gpus available for testing is not something I can just dig every minute :-) There are more news coming soon, and not just about models. And it’s good news! :-) Best, Alex