October 2025 - Blablador-news - lists.fz-juelich.de

Phasing out Llama3-405
by Strube, Alexandre 27 Oct '25

27 Oct '25

Hello Blabladores, in order to improve the reliability of BLABLADOR, I am having to do some drastic changes, which required me to shutdown pretty much every single model and the server itself. OF COURSE while this happened, some GPUs decided do fail, which forces a full node reboot. I am bringing everything back up. If you are not working at the evening, just ignore this part. They are basically all up. Another thing is that I would like to phase out Llama3-405. It’s an obsolete model, it’s slow, and takes too many resources. Yesterday, Minimax-M2 came out, and it’s one of the best models out there. It’s ranked among models such as ChatGPT 5 and Claude 4.5. It is already running. It’s aliased as “alias-huge”. But as this is a reasoning model, you might filter what comes between the <think> and </think> on your code. Another thing is that this model might be able to generate images and music, but I need to figure it out. Let’s bark! Alex

1 0

Blabla fork for private use?
by Mario.Muellhaeuser＠dlr.de 17 Oct '25

17 Oct '25

Dear Alexandre, thank you for providing Blablador! I use it frequently for business. Now I am looking for a privacy friendly AI for friendly private use. I searched the internet and only found quite costly providers like mindverse, offering 250 tokens per month for 19 EUR. Can you suggest a more affordable AI aas provider, which offers privacy? Or are there plans or possibilities to open Blablador or a clone for public use, of course for a fee? Regards Mario -------------------------- 20 Jahre Forschungshubschrauber ACT/FHS - DLR Portal<https://www.dlr.de/content/de/artikel/news/2022/04/20221202_20-jahre-forsch…> DLR-Forschungshubschrauber: 20 Jahre Innovative Hubschrauberforschung - YouTube<https://www.youtube.com/watch?v=z_7RL3IsGrs&t=211s>

1 0

And we are good!
by Strube, Alexandre 16 Oct '25

16 Oct '25

Seems that most of the problems had been solved for now! Uhuu!

1 1

Outage of systems
by Strube, Alexandre 16 Oct '25

16 Oct '25

Hello Blabladores, you might have noticed that some models have been changing, coming and going. As they say, troubles like to come with company :-) So we have a couple issues together: 1 - The supercomputer where we run 4 of our models (GPT-OSS, Qwen3-235, Llama3-405 and Qwen-3-Coder with function calling) is offline. You can check the status of Jureca-HWAI on https://status.jsc.fz-juelich.de/ 2 - The change on the api server last friday. Given that we use mostly the VLLM backend, and VLLM has changed architecture recently. As mentioned here, https://docs.vllm.ai/en/latest/configuration/conserving_memory.html#quantiz…, "CUDA graph capture takes up more memory in V1 than in V0.” This in turn made many models run out of memory on the same hardware they ran before. So I am carefully reducing context size and the size of cuda graphs. It’s a manual, boring and slow process. I am sorry, working as fast as I can here so we can keep barking loud!! :-D Dr. Alexandre Strube a.strube(a)fz-juelich.de Helmholtz AI Jülich Supercomputing Centre Forschungszentrum Juelich GmbH 52425 Jülich, Germany Phone: +49 2461 61-3866 JSC is the coordinator of the John von Neumann Institute for Computing (NIC) and member of the Gauss Centre for Supercomputing (GCS)

1 0

Change of API server
by Strube, Alexandre 09 Oct '25

09 Oct '25

Hello Blabladores, many of you have heard that I have been testing a new api server on the past few months. It’s pretty stable, very performant, and it allows us to do way more with the models we have been running. So far, it has been on a http instance internal to FZJ, but I plan to move all models to it tomorrow (Friday), so this will be our official endpoint. If you are using the api.helmholtz-blablador.fz-juelich.de <http://api.helmholtz-blablador.fz-juelich.de/>, you don’t need to do anything. Only that more models will appear there. If you have been using the ip address of the secondary api server, just revert to the normal one. All models should appear there. Yay! Let’s push into production on a Friday, after all, WHAT COULD POSSIBLY GO WRONG? Let’s bark! Alex

1 0

Function Calling!
by Strube, Alexandre 09 Oct '25

09 Oct '25

(1) (gemini can’t write in vertical). the other models are worse Hello Blabladores! I have been trying around with enable models for function calling, as I this is something I wanted to do for a while. Please have a look at the model on the second api server - the name is "Qwen3 Coder 30B with function call” Once I get it into a stable working, it will be the new alias-code. I would like to run Qwen3-480, but it’s just too big and expensive to run. Let’s bark! Alex (1) Original picture from https://en.wikipedia.org/wiki/London_Calling

1 0

Models are back
by Strube, Alexandre 02 Oct '25

02 Oct '25

There were some changes on the authentication processes of UNICORE/Unity, and this kept the models which run on our supercomputers (Qwen3-235, GPT-OSS, Llama3.1 405b) out in the cold. This is fixed, and we’re back in business, baby! There are some special people at JSC. One of them is Bernd Schuller, the maintainer of Unicore. If you are reading this, thanks a lot! Keep barking, Alex

1 0