Turns out that this was a confusing one!
A while ago, I proposed changes to Gradio, the library which powers Blablador’s web ui, so we could generate a watermark when copying text from it.
This is to comply with the EU AI Act. They took the change in. So I updated the library on the server. All good.
At the same time, I removed the message about the 10-year old birthday some of you might have seen around.
But then, things got weird: It was suddenly happening that I had to write something first, then write again, so the system would respond!
As the two things were done roughly at the same time, I blamed Gradio - it wouldn’t have been the first time.
To make it even weirder, after wasting a night on this, I realized that SOME models worked.
After an even longer time, my investigation got weirder: models which didn’t work would work if I changed their names to something else.
At this point I was already running the server line by line on a debugger, and then it hit me: the system prompt of the birthday:
# A template with a one-shot conversation example
register_conv_template(
Conversation(
name="one_shot",
system_message="A chat between a curious human and an artificial intelligence assistant. "
"The assistant gives helpful, detailed, and polite answers to the human's questions.",
roles=("Human", "Assistant"),
messages=(
(
"Human",
"Got any creative ideas for a 10 year old’s birthday?",
),
(
"Assistant",
"""Of course! Here are some creative ideas for a 10-year-old's birthday party:
1. Treasure Hunt: Organize a treasure hunt in your backyard or nearby park. Create clues and riddles for the kids to solve, leading them to hidden treasures and surprises.
2. Science Party: Plan a science-themed party where kids can engage in fun and interactive experiments. You can set up different stations with activities like making slime, erupting volcanoes, or creating simple chemical reactions.
3. Outdoor Movie Night: Set up a backyard movie night with a projector and a large screen or white sheet. Create a cozy seating area with blankets and pillows, and serve popcorn and snacks while the kids enjoy a favorite movie under the stars.
4. DIY Crafts Party: Arrange a craft party where kids can unleash their creativity. Provide a variety of craft supplies like beads, paints, and fabrics, and let them create their own unique masterpieces to take home as party favors.
5. Sports Olympics: Host a mini Olympics event with various sports and games. Set up different stations for activities like sack races, relay races, basketball shooting, and obstacle courses. Give out medals or certificates to the participants.
6. Cooking Party: Have a cooking-themed party where the kids can prepare their own mini pizzas, cupcakes, or cookies. Provide toppings, frosting, and decorating supplies, and let them get hands-on in the kitchen.
7. Superhero Training Camp: Create a superhero-themed party where the kids can engage in fun training activities. Set up an obstacle course, have them design their own superhero capes or masks, and organize superhero-themed games and challenges.
8. Outdoor Adventure: Plan an outdoor adventure party at a local park or nature reserve. Arrange activities like hiking, nature scavenger hunts, or a picnic with games. Encourage exploration and appreciation for the outdoors.
Remember to tailor the activities to the birthday child's interests and preferences. Have a great celebration!""",
),
),
offset=2,
sep_style=SeparatorStyle.ADD_COLON_SINGLE,
sep="\n### ",
stop_str="###",
)
I could just delete this prompt, right? WRONG
Look at another one:
# A template similar to the "one_shot" template above but remove the example.
register_conv_template(
Conversation(
name="zero_shot",
system_message="A chat between a curious human and an artificial intelligence assistant. "
"The assistant gives helpful, detailed, and polite answers to the human's questions.",
roles=("Human", "Assistant"),
sep_style=SeparatorStyle.ADD_COLON_SINGLE,
sep="\n### ",
stop_str="###",
)
)
This is the culprit:
offset=2,
This skips showing the two first messages on the dialogue, since they are system messages and should be hidden. Which meant that the message I sent and the first reply would never show up.
Easy, right? Then I realized I was saving the files on my test instance and had my browser open on the production instance, which was not working because of course it wasn’t! I wasn’t changing anything in there.
So, that’s it. A simple change that made a hell of a weekend and made me lost some days of work.
But now you won’t hear about the birthday anymore :-D
Let’s bark!
Dr. Alexandre Strube
a.strube(a)fz-juelich.de
Helmholtz AI
Jülich Supercomputing Centre
Forschungszentrum Juelich GmbH
52425 Jülich, Germany
Phone: +49 2461 61-3866
JSC is the coordinator of the
John von Neumann Institute for Computing (NIC)
and member of the
Gauss Centre for Supercomputing (GCS)
