- Blablador-news - lists.fz-juelich.de

Blablador is DOWN
by Strube, Alexandre 03 Nov '25

03 Nov '25

The whole VM is down. I can’t access it myself. I’m going to the institute as soon as I can. Sorry for that. Alex

1 0

Phasing out Llama3-405
by Strube, Alexandre 27 Oct '25

27 Oct '25

Hello Blabladores, in order to improve the reliability of BLABLADOR, I am having to do some drastic changes, which required me to shutdown pretty much every single model and the server itself. OF COURSE while this happened, some GPUs decided do fail, which forces a full node reboot. I am bringing everything back up. If you are not working at the evening, just ignore this part. They are basically all up. Another thing is that I would like to phase out Llama3-405. It’s an obsolete model, it’s slow, and takes too many resources. Yesterday, Minimax-M2 came out, and it’s one of the best models out there. It’s ranked among models such as ChatGPT 5 and Claude 4.5. It is already running. It’s aliased as “alias-huge”. But as this is a reasoning model, you might filter what comes between the <think> and </think> on your code. Another thing is that this model might be able to generate images and music, but I need to figure it out. Let’s bark! Alex

1 0

Blabla fork for private use?
by Mario.Muellhaeuser＠dlr.de 17 Oct '25

17 Oct '25

Dear Alexandre, thank you for providing Blablador! I use it frequently for business. Now I am looking for a privacy friendly AI for friendly private use. I searched the internet and only found quite costly providers like mindverse, offering 250 tokens per month for 19 EUR. Can you suggest a more affordable AI aas provider, which offers privacy? Or are there plans or possibilities to open Blablador or a clone for public use, of course for a fee? Regards Mario -------------------------- 20 Jahre Forschungshubschrauber ACT/FHS - DLR Portal<https://www.dlr.de/content/de/artikel/news/2022/04/20221202_20-jahre-forsch…> DLR-Forschungshubschrauber: 20 Jahre Innovative Hubschrauberforschung - YouTube<https://www.youtube.com/watch?v=z_7RL3IsGrs&t=211s>

1 0

And we are good!
by Strube, Alexandre 16 Oct '25

16 Oct '25

Seems that most of the problems had been solved for now! Uhuu!

1 1

Outage of systems
by Strube, Alexandre 16 Oct '25

16 Oct '25

Hello Blabladores, you might have noticed that some models have been changing, coming and going. As they say, troubles like to come with company :-) So we have a couple issues together: 1 - The supercomputer where we run 4 of our models (GPT-OSS, Qwen3-235, Llama3-405 and Qwen-3-Coder with function calling) is offline. You can check the status of Jureca-HWAI on https://status.jsc.fz-juelich.de/ 2 - The change on the api server last friday. Given that we use mostly the VLLM backend, and VLLM has changed architecture recently. As mentioned here, https://docs.vllm.ai/en/latest/configuration/conserving_memory.html#quantiz…, "CUDA graph capture takes up more memory in V1 than in V0.” This in turn made many models run out of memory on the same hardware they ran before. So I am carefully reducing context size and the size of cuda graphs. It’s a manual, boring and slow process. I am sorry, working as fast as I can here so we can keep barking loud!! :-D Dr. Alexandre Strube a.strube(a)fz-juelich.de Helmholtz AI Jülich Supercomputing Centre Forschungszentrum Juelich GmbH 52425 Jülich, Germany Phone: +49 2461 61-3866 JSC is the coordinator of the John von Neumann Institute for Computing (NIC) and member of the Gauss Centre for Supercomputing (GCS)

1 0

Change of API server
by Strube, Alexandre 09 Oct '25

09 Oct '25

Hello Blabladores, many of you have heard that I have been testing a new api server on the past few months. It’s pretty stable, very performant, and it allows us to do way more with the models we have been running. So far, it has been on a http instance internal to FZJ, but I plan to move all models to it tomorrow (Friday), so this will be our official endpoint. If you are using the api.helmholtz-blablador.fz-juelich.de <http://api.helmholtz-blablador.fz-juelich.de/>, you don’t need to do anything. Only that more models will appear there. If you have been using the ip address of the secondary api server, just revert to the normal one. All models should appear there. Yay! Let’s push into production on a Friday, after all, WHAT COULD POSSIBLY GO WRONG? Let’s bark! Alex

1 0

Function Calling!
by Strube, Alexandre 09 Oct '25

09 Oct '25

(1) (gemini can’t write in vertical). the other models are worse Hello Blabladores! I have been trying around with enable models for function calling, as I this is something I wanted to do for a while. Please have a look at the model on the second api server - the name is "Qwen3 Coder 30B with function call” Once I get it into a stable working, it will be the new alias-code. I would like to run Qwen3-480, but it’s just too big and expensive to run. Let’s bark! Alex (1) Original picture from https://en.wikipedia.org/wiki/London_Calling

1 0

Models are back
by Strube, Alexandre 02 Oct '25

02 Oct '25

There were some changes on the authentication processes of UNICORE/Unity, and this kept the models which run on our supercomputers (Qwen3-235, GPT-OSS, Llama3.1 405b) out in the cold. This is fixed, and we’re back in business, baby! There are some special people at JSC. One of them is Bernd Schuller, the maintainer of Unicore. If you are reading this, thanks a lot! Keep barking, Alex

1 0

New models and api server changes
by Strube, Alexandre 02 Sep '25

02 Sep '25

Hello Blabladores! It’s been a while since I wrote last. Things haven’t been quiet, though. Many of you noticed that I added some models - like GPT-OSS, for example - and they do not appear on the API listing. That is not an accident. We have been preparing for some months a new, faster API server, which better capacities to support the upcoming new hardware we will get for Blablador. The models running on our cloud instead of the old cluster are pointing to this new server. At some point, all models will run into the new server, therefore the new server replaces the endpoint of the old one. If you want to check the models running on it, give it a try: http://134.94.199.27:8080/v1 This week and the next are pretty busy, so I most probably won’t change these right away. But this endpoint above will at some point disappear and we we’ll only have the default one, with all models. Another interesting thing is that we have the Hunyuan-MT-Chimera-7 translation model available. It’s the first time we have a model only for translation. I hope it’s useful! Let’s bark! Alex, el Blablador

1 0

New read_user scope for api token
by Strube, Alexandre 08 Aug '25

08 Aug '25

Hello Blabladores! Some of you mentioned that using the “Api” scope on helmholtz codebase for BLABLADOR was too much. Helmholtz kindly added the “read_user” token, so one can create a much more strict token and use it instead of the old “api” one. So if you want, just go to https://codebase.helmholtz.cloud/-/user_settings/personal_access_tokens and create a new token with this much more limited scope, and revoke the old one, and things should work the same, but without any access to anything on your Helmholtz Codebase! Let’s bark throughout the weekend! Alex

1 0