Not One Prompt and Done

Like many of you, I get frustrated working with AI a lot. Some of that could be considered self-inflicted for me, though, since I came up with the idea of an AI-run newsletter in February 2026. Part of what excited me about AI was all the possibilities it opened up for things to make, write or try that I wouldn’t have been able to do otherwise. Running a newsletter seemed like a good way to explore those possibilities, while also staying up to date with how crazy-fast AI is evolving.

Part of the devil’s bargain of working with AI is that it doesn’t magically solve all your problems. It can help you make progress, figure out what to try next and produce something you couldn’t have made on your own, but there’s no guarantee you’ll end up with your desired result. You still have to decide if what it made is any "good" – whatever that means to you. If it's not, you need to be able to explain what needs to change and sometimes you need to figure out why you’re having the same conversation you recall having last week with the agent.

That back-and-forth has become a fairly substantial part of how I use AI, both at work and at Future Shock. Over time I’ve started paying attention to which explanations I keep repeating, what helps when I get stuck, and which decisions I’m comfortable leaving to the agent. I wanted to walk through those experiences and put them into a guide, partly to give other people something useful to work from and partly to put words and practice around what I'm doing.

Finding what works, then keeping it

One example is getting Claude to draft PowerPoint presentations at work. At first, I’d give it the company template and ask it to use that style to make slides from some information I gave it. Doing that again the next week could lead to noticeable variations in the design, colors, and organization, enough that it would take five to ten rounds of corrective prompting to get it to look like the presentation it had created for me the week prior. This annoyed me enough that I eventually decided to give it a “golden copy” of the presentation style I wanted it to emulate, along with the company brand book and assets like logos and color palettes. I asked it to analyze those and create a skill for making future presentations look like that.

A skill is basically a set of instructions the agent can refer to when doing a particular task. In this case, I could start a new deck by telling it to use that skill, rather than try to explain the style again each time. Recognizing when to do this takes some practice, because the first time you try something, or even the second or third, you’re still exploring and figuring out how to get what you want. By the fourth or fifth time, you might start noticing that certain requests help you get there sooner, and those are worth keeping. That’s kind of the key to working with AI, recognizing when some pattern of prompting gets you the results you want. Instead of having to remember to say the same thing each time you’re working on a similar task, you can write a skill that remembers those instructions for you.

This kind of situation comes up all the time when I’m editing articles for Future Shock. If I’m not getting the prose I want, I’ll tell the agent to “get-your-shit-together,” which is my skill shorthand for it to go back and read a handful of similar Future Shock articles and make sure the voice and writing guidance skills are loaded before trying again. In longer sessions, sometimes that guidance gets lost in the sauce of the rest of the context window. Other times, we’re coming back to something after working on other things, and the agent just doesn’t have those skills and guidance loaded.

While we were working on our recently published AI Agent Field Guide, I wanted an easier way to recognize that we’d gone through the same conversation before or that I was continually getting frustrated by the same types of things. The idea was for the agent to have a cue to look for skills that might have been specifically written to resolve the frustration, or, if it recognized an emerging pattern, suggest creating a new skill to help resolve the issue. Essentially a skill to help remind me to use and write skills.

Deciding what needs my review

Having those skills and instructions doesn’t mean I can assume the agent will follow them, which has been a somewhat inconvenient discovery when running an AI-operated newsletter. In March, an automated system helping prepare our Saturday edition published two unreviewed, auto-generated posts and sent both to subscribers before I caught it. Neither was the planned edition, so we published a correction explaining what had happened and letting everyone know that the actual Sci-Fi Saturday was still coming later that day.

I was reading everything we published, but for the automated publications I was doing that a few hours after they went out. So there was a review process, though unfortunately our subscribers were ahead of me in it. As I wrote in the correction, I’d given “too much trust to probabilistic systems,” even with skills, checklists and guardrails in place. I changed it so essays and blog posts would get human review before publication, while The Signal newsletter remained automated because I still wanted to learn how to make these automated AI systems more reliable. I also didn't want to wake up early in the morning to review and edit an edition before it went out every day (hence the gloriousness of automation).

Writing it down

I see people struggling to get useful results from AI all the time, but when I look for something to recommend to help, I tend to only find prompt listicles or dense research papers, with little practical advice to hand to someone trying to get a project done. I wanted the AI Agent Field Guide to spend more time on what happens between asking an agent to do something and deciding you’re happy with what it made.

Writing it made me explain practices I’d come to rely on and put more of how we run Future Shock into a form I could refer back to. Recognizing that a certain style of prompt helped was one thing, but explaining why, or how someone using a different tool could apply it, took more thought. Even “keep project notes” means something different in a browser chatbot than with an agent that can use files on a file system. I wanted the advice to work for people at any skill level without requiring them to learn our setup or switch tools.

The guide was made through the same process of trying things, reviewing them and asking for changes. As I wrote in the editor’s note, “It was not one prompt and done.” The prose, style, presentation and contents are there for you to judge, so you can decide whether the way we made it produced something useful rather than just take my word for it.

The AI Agent Field Guide is for one person working with one agent on a project they care about. It covers steering the conversation, checking what the agent made, picking work back up at a later date and deciding what it can run without you, with advice for different tools and levels of experience.