Lessons from an AI Recess
An Idea for Consideration
For the last couple of years, my friend Jeremy Utley has been researching generative AI and how organizations can use it. One of his insights, shared in a recent blog post, was that anyone interested in learning more about generative AI should do an “AI recess,” or a long stretch of time dedicated to experimentation.
The idea behind the recess is that hands-on-keyboard work is the only way to understand the power of these tools and what’s possible. I blocked off a whole day for an AI recess two weeks ago. Here’s what I found:
What’s my job?
I started by brainstorming on the question, “What’s my job?” The goal was to create some concrete use cases.
The main insight was that there are areas where I could use AI but don’t want to. For example, longer-form writing fleshes out what I think about certain topics. AI tools could help with writing tasks, but that would disrupt the higher-order purpose. Similarly, I could use AI to summarize a book, but slow-reading the book and then doing my own summary imprints the content on my memory such that I can access it in real-time with clients—i.e., when it’s most valuable for them.
Ultimately, that analysis was helpful because it illuminated a more fundamental question: What parts of my job are still valuable in an AI world?
Generative AI is most useful on topics where you’re an expert.
On a recent episode of his podcast, Ezra Klein interviewed Wharton professor Ethan Mollick and asked about his recommendation to spend at least 10 hours with AI tools.
Mollick said this: “So first off, I want to indicate the 10 hours is as arbitrary as 10,000 steps. Like, there’s no scientific basis for it. This is an observation. But it also does move you past the, ‘I poked at this for an evening,’ and it moves you towards using this in a serious way. I don’t know if 10 hours is the real limit, but it seems to be somewhat transformative. The key is to use it in an area where you have expertise, so you can understand what it’s good or bad at, learn the shape of its capabilities.”
That’s important because, while these tools will get better over time, the responses are not always good. For example, I asked the Claude model for some education statistics. Its responses were definitive (e.g., “The answer is 5.3%.”), but the data wasn’t present when I clicked through to the underlying source. Moreover, Claude couldn’t explain how it came up.
Being able to detect “that doesn’t sound right” and “ we can do better” is important to get the benefit from these tools, and you can only know that in areas where your knowledge is deep.
Treat it like an intern. The more you explain, the better it can help.
If you give a vague instruction—e.g., “Find me information on ___,” for example—you’ll usually get a less-than-useful response whose sourcing is unclear. That’s why you need greater specificity.
I learned to include prompts like these:
Please cite your sources.
Find references in these publications: New York Times, Washington Post, Wall Street Journal…
Find papers by professors at….
Give me one answer at a time.
What you need to know about me / my customer / the situation is…
I want you to play the following role:
I give all those examples because they show that generative AI tools, like any technology, have a learning curve. I now know barely 1% of what I’ll want to know about these tools in the future, but getting over the initial learning curve has left me far more confident.
(Technical note: The experts say that you have to subscribe to the paid versions of generative AI tools to see the power of the most advanced models. At just $20 a month, it’s cheap enough to play around with. However, the experts also say that future releases will be even better at understanding intent and require less detailed instructions.)
Once you get started….
Perhaps the most important part of the experiment was that once I started, the possibilities for AI-supported work became even more obvious. For example, I started an online order from my local ramen shop. But then it occurred to me: “Could AI help with this?”
The answer was “not really,” but the exciting part was the instinct to interrupt my normal operating mode to build a new set of habits.
Leadership Wisdom
“[A]s somebody who teaches at universities, like, lots of people are summarizing. [...] [B]ut before A.I., there were — best estimates from the U.K. that I could find, 20,000 people in Kenya whose full-time job was writing essays for students in the U.S. and U.K. People have been cheating and Sparknoting and everything for a long time. And I think that what people will have to learn is that this tool is a valuable co-intelligence, but is not a replacement for your own struggle.”
— Ethan Mollick
Something Fun
There’s a ChatGPT app, RizzGPT, that will help you come up with flirty responses. It’s designed for use in dating scenarios, but I have been using it with my wife. She texted me this statement: “I’m not stopping by [the restaurant where I was eating with friends]. Realized wouldn’t leave enough travel time for my hair appointment. Tell them I said hello.”
RizzGPT suggested I respond with this: “Missing out on your company, but I hope your hair turns out as fabulous as your excuses! Give my best to your hair stylist too! 😄”
That’s a much better response than the normal non-response or “OK” I would normally give to such a message!
Separately, I asked DALL-E to create an image of me doing the AI recess. On the left is the first picture it came up with—even though I’d uploaded a picture of myself to start!
On the right is what I got after a few minutes of asking for revisions. I kept asking DALL-E for “shorter hair” to match my own, but it couldn’t get closer until I used “nearly bald” in the prompt. Definitely a moment of humility!
