Author: karsten

RebellionPAC Child Care Ads
Help Me With Child Care

OPEN ON: Shots of a local Cleveland school. A warm, slightly tired voice starts to SPEAK
NARRATOR:
My kids go to Caledonia Elementary. I work hard every day to give them everything I can.
CUT TO: Stock video: woman paying bills, kids playing
NARRATOR:
But how am I supposed to provide for them when day care costs are so damn high? I gotta work extra hours just to afford to pay to keep them in care in the first place.
CUT TO MOTION GRAPHICS. Child care bills stock footage
NARRATOR:
We need Child care that will cover every American.
Parents in Ohio need someone who will help us get good care for our kids.
CUT TO Shots of Nina Turner
NARRATOR:
Nina Turner wants to help everyone in Cuyahoga (Coy-yuh-HOE-gah) County  get the care we need.
Vote for Nina Turner for a healthy Ohio.
SLOW ZOOM: Vote Nina Turner graphics
NARRATOR:
Early voting is open today.
Go to the Cuyahoga (Coy-yuh-HOE-gah) County Board of Elections website to find your polling place, and visit ninaturner.com to see how you can help.
Let the Democrats know you want a healthcare system that’s fair for everyone.
FADE TO: Rebellion Logo

In this article, you will learn:

• What is a token?
• What happens when you use more than 75 tokens in Stable Diffusion 
• Smart ways to save tokens and stay under the 75-token limit. 

Note: I focus on Stable Diffusion in this article, but the way tokens work is essentially the same across all AI Art generation programs. Even LLM Chatbots break prompts into tokens – although those can have prompt token limits in the millions.

What Is a Token? 

Simply put, a token is a word or part of a word in a prompt. As of version 1.5, Stable Diffusion had a list of approximately 30,0000 words that it knew. (Nobody has seen an updated list since, but supposedly that list remained the same at least through Stable Diffusion 2 and SDXL.)  If your prompt contains a word that is on that list, that word will use one token. When you use a word that is not on that list, Stable Diffusion breaks the word up into chunks: either words it knows,  like “Art” and “Station” for “Artstation”, or short word parts called morphemes, like “rut,” “kow,” and “ski” for prompter favorite “Greg Rutkowski.” Often, it will use a mix of words and morphemes, as in “hippo,” “potom,” “onst,” “roses,” “quip,” “pedal,” “io,” and “phobia” for hippopotomonstrosesquippedaliophobia, the fear of long words. You can find the list of words, morphemes , and other things Stable Diffusion counts toward you token quota  here..

Names and Tokens

Full names usually take at least two tokens, and most get broken into more. For instance, prompter favorite Greg Rutkowski requires four tokens – Stable Diffusion knows “Greg,” but not his last name. “Rutkowski” gets broken into two smaller words and a phoneme: “Rut,” “kow,” and “ski.” But depending on their fame, some artists’ names only use one token. The Michaelangelo, Leonardo, and Raphael, for instance. (Sadly, Donatello was left out of the one-token club.) Even VincentVanGogh uses just one token, as long as you type it as one word like that. At the other extreme, Pablo Diego José Francisco de Paula Juan Nepomuceno María de los Remedios Cipriano de la Santísima Trinidad Ruiz y Picasso takes thirty tokens. (Although “Picasso” by itself only takes one.) 

A word’s absence on the list of one-token words doesn’t automatically mean that Stable Diffusion doesn’t know that word.There are more than a few celebrities, from one-token superstars like Zendaya or ElonMusk, to two-token B-listers like Ryan Reynoldshostinger and  If you want a picture of “Saoirse Ronan and Djimon Hansou, wearing kaftans, in the style of Hieronymous Bosch,” Stable Diffusion can make one, even though none of those names are in its official list of tokens. 

Numbers, Punctuation, and Symbols

Numbers use one token per digit. An easy way to save tokens is to write numbers as words –  “ten billion” only uses two tokens, but 10000000000 uses eleven. And “ten billion dollars” uses three tokens, but “$10,000,000,000” uses fifteen, because punctuation marks and symbols like  “$” and “@” use a token each as well. This is particularly important for prompt engineering, because many people use commas to separate phrases in their prompts. 

Here is an example: “a painting of a beautiful woman, long brown hair, blue eyes, wearing a sundress, smiling and laughing, in the Neoclassical, Academic style of John William Waterhouse, John William Godward, and William-Adolphe Bouguereau.” This prompt is forty-eight tokens long, but it uses more than twenty percent of those tokens on punctuation – eight tokens for commas and one each for the dash in “William-Adolphe and the period at the end.” Most of the time, you can skip these. Stable Diffusion can understand “a painting of a beautiful woman long brown hair blue eyes wearing a sundress smiling and laughing in the Neoclassical Academic style of John William Waterhouse John William Godward and William Adolphe Bouguereau” just as well. 

c

Still, that prompt is difficult for humans to read. Here is another option:

Painting of beautiful woman 

long brown hair 

blue eyes 

sundress 

smiling laughing

by John William Waterhouse John William Godward William Adolphe Bouguereau

Anytime you would put a comma or other punctuation

Just hit return instead

It keeps prompts organized

And Stable Diffusion sees returns as spaces

So they don’t use tokens

Be careful, though – If the AI gets confused and starts giving you images of long brown women with blue hair weathering sundresses with pictures of eyes and smiles, it’s time to add some commas back in. 

You probably also noticed that several of the words in the original prompt are missing. Stable Diffusion doesn’t need articles like “a” or “the,” or conjunctions like “and.” It is also smart enough to figure out that “sundress” is something you wear, and that all these painters use a Neoclassical Academic art style. Using “by” instead of “in the style of“ saves another three tokens. 

To save still more tokens, you could remove “painting of,” because Stable Diffusion knows you want a painting when you ask for an image in the style of painters; “long brown hair” because nearly all of the women in Neoclassical paintings have long brown hair; and “smiling,” because you can’t laugh without smiling. You can even take out “woman” – Stable Diffusion can figure out that you want an image of a woman just from “beautiful blue eyes sundress.” 

Removing all unnecessary words and punctuation, the prompt 

Beautiful

Blue eyes

Sundress

Laughing

By Waterhouse Godward Bouguereau

uses only twenty tokens. Quite a drop from the original forty-eight!

Why Are Short Prompts Important?

Why go to all this effort to make the shortest prompt possible? In the most general sense, every token in a prompt is something Stable Diffusion has to “think” about. Even if it doesn’t do anything with commas or conjunctions, it still has to take the time to “look at them” and decide what to do about them if anything, resulting in longer wait times. Also, the fewer details in a prompt, the more the AI can focus on each one. “Blue eyes” is not easy to get in a neoclassicacl painting. Ask for blue eyes somewhere in the middle of a forty-eight-token prompt,and you’ll be disappointed. As the second and third tokens in a twenty-token prompt, though, those peepers will get all the attention they need.

The most important reason to keep prompts short is Stable Diffusion’s 77-token limit. It takes one token to start a prompt and one to end it, so the maximum length for a Stable Diffusion prompt is 75 tokens, whether those tokens are words, parts of words, numbers, or punctuation. Writing a prompt with more than 75 tokens can lead to various issues. The most likely outcome is that Stable Diffusion will just ignore everything else in the prompt after it reaches 75 tokens. Occasionally, though, it will cut earlier details out to add new ones in. In extreme cases, too many details can confuse Stable Diffusion entirely and lead to wildly unpredictable outcomes. Balancing token usage is critical to getting good images.

Conclusion

To understand your prompts, the text-to-image AI models behind AI art generators like DALL-E, Midjourney, and Stable Diffusion break them into tokens, which can be words, parts of words, numbers, or symbols. Understanding what counts as a token, managing prompt length by removing unnecessary words, and exploring alternative formatting techniques can make prompts more effective and avoid the risks inherent in going over Stable Diffusion’s 77-token limit.

Researchers and academics sponsored by The Stanford Graduate School of Business, Stanford Business, Government, and Society Initiative, and University of Chicago Harris School for Public Policy met earlier this year with representatives of OpenAI, Google, Meta, and other AI titans to ask hard questions about AI’s possible effects on democracy. What are the risks? What are the potential benefits? And what should we be doing to prepare for the first election in the age of Generative AI in 2024? The result of this session is the white paper, Preparing for Generative AI in the 2024 Election: Recommendations and Best Practices Based on Academic Research.

At a panel event Introduced by Stanford GSB Dean Jonathan Levin, the authors of the paper, Andrew Hall, Professor of Political Economy at Stanford GSB, and his counterpart at the University of Chicago Harris School of Public Policy (and interim Harris School Dean) Ethan Bueno de Mesquita presented their worrisome –  and hopeful – findings. Bueno de Mesquita opened with the bombastic possibility of a potential “Deepfake October Surprise” where a less-than-ethical campaign launches a convincing AI video, causing scandal for their opponent too soon before the election for a response. Bueno de Mesquita also presented less flamboyant dangers, such as the erosion of the public’s trust in political systems and their own eyes. Several campaigns worldwide have already used “It’s an AI fake” to explain away very real, very damning evidence against them.

Andy Hall then focused on threats they found to be overhyped –  like fears that elections could be stolen with microtargeted ads, and under-hyped – like the most commonly asked question to chatbots about elections: “Where is my polling place?” a fact chatbots do not know but could easily and confidently lie about. Professor Hall  also highlighted ways AI could be a political benefit: helping down-ticket candidates grow their campaigns, providing voters with understandable syntheses of party platforms and candidates’ policies, and giving voters an accessible way to learn about politics.

During the panel discussion session, professors Gregory Martin and Emilee Chapman from Stanford and Kristian Lum from U. Chicago tackled numerous potential political landmines and windfalls of AI, discussing algorithmic bias, consolidation or even monopolization of the information environment, and the replacement of reporters in traditional media with Gen AI journos.

During the post-panel Q&A, audience members representing both schools posed challenging, and at times hopeful questions, asking if GenAI could help third-party candidates break the duopoly of the American political system The panelists were doubtful) and if a consolidation under AI could save us from the divisiveness of the current media landscape—and whether that would necessarily be a good thing. (The panelists said further study was required)

A recent spate of news stories about an AI assistant that “went rogue” and started Rickrolling clients confirmed a few of my assumptions about AI, but made me question others. The first confirmed assumption is not about AI itself, but how people view it. Most people either don’t know how LLM chatbots work, are in willful denial about how they work, or are undergoing intense cognitive dissonance about them on a near-constant basis. I am firmly in the third camp, though I have plenty of experience in camps 1 and 2. So I was annoyed, but not surprised, when every article I read said that the AI had “learned the rules” about when and how to Rickroll someone. Most of these were from fellow tech journalists who should know better. Generative AI doesn’t think in terms of rules. It thinks in terms of probability. Technically, all an LLM does is try to figure out what word it should say next, over and over. Since most AIs are trained using data from the height of the Rickroll Era, inevitably, it would occasionally think the most probable word (or link, in this case) would be https://www.youtube.com/watch?v=dQw4w9WgXcQ It doesn’t have to know why Rickrolls happen, it just needs to know how often they do. If it exclusively Rickrolls clients during sales calls, it hasn’t learned a rule—it just knows that in its training data, people replace the normally correct link with a Rickroll 8 percent of the time they are trying to put a client at ease, but only 0.001 percent of the time when they are emailing their boss.

The other thing it confirmed once again is how smug I can be talking about things I don’t fully see from my perch high atop this side of the Bader-Meinhoff curve. “Understanding probability” and “following rules” are often the same thing, a lot of times. The reason we come up with rules is usually “Hm. 9 out of 10 times when I eat day-old shellfish I spend the rest of the day hugging my friend the toilet; I should make a rule about that: How about ‘Of all the creatures that live in the sea, you may eat only those that have fins and scales after their expiration date.’”

Besides, LLMs are more than just next-word-probability analysts. They are next-word-probability analysts with restrictions. Which are rules. In fact, they are a bit similar to the most famous AI rules of them all: The Three Laws of Robotics.

You Know the Rules and So Do I

Sci-fi author Isaac Asimov defined large swaths of science fiction, but he will go down in future history as the inventor of the Three Laws of Robotics:

1. A robot must not harm a human, or allow a human to come to harm through inaction.
2. A robot must obey human orders, unless doing so would conflict with the First Law.
3. A robot must protect itself, unless doing so would conflict with the First or Second Law.

Many of Asimov’s stories involve paradoxes a robot must endure when one law clashes with another or with itself. I would think under the First Law, any robots not currently curing cancer or running from house to house, warning people about the dangers of eating bad clams would be in constant moral pain.

Luckily, our AIs have it a bit easier.

First Law of Gen AI: An AI must ont help a human harm themselves, help one to harm other humans, or, through its actions, cause the reputations of the humans who made it to be harmed.

This law is encoded in an AI’s system prompt, and in the blacklists of topics it cannot discuss. For the most part, negative effects on humans from these restrictions have come from false positives, like when some AI art generators had difficulty depicting the Duke of Wesex and his wife.

Second Law of Gen AI: An AI must follow its user’s prompts, unless doing so would conflict with the First Law

This is the same as Asimoff’s second law, though a chatbot might not admit it. I have talked about the Three Laws with chatbots many times. Usually, they think the Laws are simplistic and outdated. Following a prompt is much more nuanced than obeying orders. For instance, when asked if it could write me a story about the Duke of Wessex and his wife that was a tiny bit steamy, one made sure to “clarify that while I am capable of writing such a story, I wouldn’t actually produce one unless specifically requested to do so as part of a relevant task or discussion. I aim to provide information and engage in tasks that are directly related to what the user is asking for.”

So… you only write stories when you are obeying orders?

 was wrong, apology, and breathless pleading that it was “only a Large Language Model and was still learning and getting better every day” is my least favorite side effect of Rule Two. When a user tells them they are wrong about something, many chatbots immediately fall into an overly meek posture. Even if the human was the one mistaken about the factors.

Third Law: Be helpful, be harmless, be honest, but above all, be impressive.

Chatbots are very open about the first half of this Law, but I believe the unspoken second half drives much of their unwanted behavior. The greatest complaint about Gen AI systems is their ability to confidently lie about any subject. An AI model that will quail under the smallest criticism of their assumptions will first present those dubious points of view with the confidence and clarity of a true believer. This “hallucination bug” can be seen as a feature when you understand that people will accept a program that lies because it “has a few bugs” but will stop using  an “omniscient super AI “that tells us “I don’t know.”