AI tools have transformed how businesses and individuals approach tasks, but understanding how to use these tools effectively takes some practice. We reviewed three prompt engineering guides from OpenAI, Microsoft, and Google, focusing on their intended audiences and the skills needed to work with large language models (LLMs). Prompting is not just a skill but a strategy to produce your desired outputs, and these guides contain varying levels of complexity and focus.
The Prompting Guides
Prompting, or crafting instructions for AI, is both art and science. The process enables users to communicate their goals clearly to LLMs, which need plenty of context and data to be successful. This review breaks down three guides:
- OpenAI’s API Prompt Guide: Designed for developers creating custom applications.
- Microsoft’s Azure OpenAI Guide: Aimed at technical audiences with experience in few-shot training and integration.
- Google’s Gemini Guide for Workspace: Geared toward business users but places a significant burden on them to craft effective prompts.
The Developer’s Toolkit: OpenAI and Microsoft
Both OpenAI and Microsoft’s guides target a technical audience.
- OpenAI’s API Prompt Guide: Focuses on helping developers integrate generative AI into applications. It includes advanced techniques such as retrieval augmented generation (RAG), allowing users to ground AI outputs in specific datasets. This guide assumes familiarity with APIs and programming, making it ideal for software engineers building AI-powered workflows.
- Microsoft’s Azure Guide: Caters to developers and solution architects leveraging Azure’s AI capabilities. It highlights the nuances of prompt crafting, from few-shot examples to embedding business logic. The guide acknowledges that prompting requires both technical expertise and creativity.
Both guides excel in equipping developers with the tools to embed AI in robust, scalable applications. However, their technical focus may alienate non-technical users.
Google’s Gemini Guide: A Business User’s Perspective
Google’s Gemini guide takes a different approach, targeting a broader audience of business professionals. By integrating Gemini into Google Workspace tools like Gmail and Docs, the guide aims to democratize AI usage. However, this accessibility comes at a cost.
- Persona, Task, Content, and Format: Google emphasizes structuring prompts with these four elements. For instance, users are instructed to define their role (e.g., HR manager), describe the task, provide relevant context, and specify the desired output format (e.g., table or bulleted list). While these steps ensure clarity, they place a heavy burden on the user to manually input detailed information. Google has acknowledged this challenge, stating, “Based on what we’ve learned from our users so far, the most fruitful prompts average around 21 words with relevant context, yet the prompts people try are usually less than nine words.” This significant gap highlights how users often underutilize the potential of AI by providing insufficient details in their prompts.
- Iterative Refinement: The guide suggests users iterate on their prompts to achieve better results. While practical, this can be time-consuming and inefficient for routine tasks. The need to refine prompts repeatedly only compounds the burden on users, detracting from the overall efficiency AI is supposed to provide.
Unlike the developer-focused guides, Google’s guide underestimates the effort required from non-technical users, particularly when creating detailed prompts for everyday tasks. This reliance on user-crafted prompts underscores the need for solutions that automate these processes instead of shifting the workload onto users.
Challenges in Prompting
Across all three guides, certain limitations emerge:
- Learning Curve: Effective prompting requires training, which may be overwhelming for large teams or non-technical users.
- Contextual Burden: Users must often provide detailed context and references, which can negate the efficiency AI is supposed to deliver.
- Privacy Concerns: Including sensitive data in prompts raises questions about data security, particularly with cloud-based tools like Gemini.
Krista is Your Prompt Engineer
For technical users, OpenAI and Microsoft’s guides offer powerful frameworks for integrating AI into custom solutions. However, rather than teaching employees to become prompt engineers, Krista takes a different approach. Krista handles prompt engineering automatically by managing roles, providing context, leveraging retrieval-augmented generation, and orchestrating complete workflows—not just creating content. This eliminates the need for employees to manually craft and refine prompts, saving time and ensuring consistency.
Organizations can focus on achieving business goals without the overhead of training employees on prompt engineering or dealing with the inefficiencies of manual prompting. With Krista, automation handles the complexity, allowing teams to focus on strategic, high-value activities.
Choosing the Right Approach
Prompt engineering is evolving, and these guides reflect the varied needs of their audiences. Developers can leverage OpenAI and Microsoft’s technical depth to build sophisticated solutions. Meanwhile, Krista removes the cognitive and contextual burden from users entirely. Instead of requiring business users to craft detailed prompts as seen with Google’s Gemini, Krista automates prompt engineering, ensuring efficient and reliable workflows.
As AI becomes integral to workflows, choosing a solution like Krista that handles prompt engineering and orchestration can differentiate efficient teams from those mired in inefficiencies. The question isn’t whether to use prompting—it’s how to make it work for you.
Links and Resources
Speakers
Transcript
Scott King
Hey everyone, this is Scott and Chris. Thanks for joining us on this episode of the Union Podcast. How’s it going, Chris? You’ve got the lumberjack look going today.
Chris Kraus
It’s going well. Can you tell it turned a little bit cold in Dallas? It hasn’t frozen, but it feels like winter here.
Scott King
Cold in Dallas isn’t really cold compared to other places, so people make fun of us for that, but we can handle the heat. Today, we’re going to talk about some prompting guides we found. We came across three different guides, and we’ll discuss what’s in them, who they’re for, how to use them, and which might be best for your situation.
If you’re looking to use generative AI more effectively in your personal or work life, this is for you. Chris, we found the Microsoft prompting guide, which is inside Azure, right? Is it specific to OpenAI and Azure? Then there’s the OpenAI platform API guide.
Chris Kraus
Yes.
Scott King
And Google put out a Gemini prompting guide. From your perspective, let’s start with the Microsoft guide. Tell me who it’s geared for. I read through it and thought there’d be more documentation. Maybe I didn’t dive deep enough, but who is it for, and what’s in it?
Chris Kraus
The Microsoft guide is interesting because it explicitly states that it assumes a technical audience with familiarity in prompt engineering and few-shot training. That’s already complex for most people. Few-shot training involves giving the model examples of what you want it to recognize or categorize.
Scott King
What’s few-shot training?
Chris Kraus
It’s when an LLM uses its pre-trained knowledge to complete tasks with minimal examples. For instance, the model has read the internet, so it knows that “Humpty Dumpty sat on a…” is likely followed by “wall.” Few-shot training involves giving it specific examples to teach it how to handle something like categorizing a complaint or identifying a product return. You provide a few examples, and the model uses them to understand the task.
It’s a complex process. When you and Jeff did that webinar on the rise of the prompt engineer a year or two ago, this was the kind of complexity we discussed. Credit to Microsoft for clearly stating upfront that this guide isn’t for beginners—it’s highly technical, and you need to have that foundational knowledge to use it effectively.
Scott King
All right, so this one isn’t for the regular business user. We get a lot of questions like, “This AI thing is cool, but how can I use it?” They’re referring to generative AI. This guide isn’t for that audience because they’re looking for use cases. Would you say it’s developer-centric or maybe for a solution engineer—someone with a less code-heavy job?
Chris Kraus
No, it’s definitely not for a non-technical audience.
Chris Kraus
I think it’s aimed at someone who’s heavily code-focused and building applications. It’s for a developer audience—people embedding this technology into applications.
Scott King
And what kind of applications? Any application, or is it more Microsoft-specific?
Chris Kraus
It’s hosted in Azure, so you can leverage it through frameworks like .NET or Java. While it’s platform-independent, it’s specifically hosted as an LLM in Azure.
Scott King
Okay, moving on to OpenAI. I’d imagine this one is closely aligned with Microsoft’s guide and targets a developer audience, right? I looked at it, and my eyes glazed over—it’s not for me.
Chris Kraus
Yes, it’s very technical. They start by referring to APIs and then get into building solutions. If you’re looking for answers in a specific context, you need to write a retrieval-augmented generation (RAG) solution. It can even generate code to get you started, which you can then refine. It’s very developer-focused and walks through the building blocks of how a developer would approach using the tool.
Scott King
Remind everyone, what is retrieval-augmented generation?
Chris Kraus
Retrieval-augmented generation, which we call Document Understanding in Krista, is about getting answers to specific questions using your company’s data rather than relying on a pre-trained corpus or internet knowledge. For example, in customer service, you might want answers based on your company’s return policies. For accounts payable or receivable, you’d want answers grounded in your specific business rules and processes. It ensures the responses are accurate for your organization, not generic or potentially incorrect ones pulled from the internet.
Scott King
That’s very different from people asking about training their own model, right? Training your own model is expensive and resource-intensive—it’s not feasible.
Chris Kraus
Exactly. And if you don’t know what you’re doing, you can mess up the neural network and end up with random, inaccurate answers.
Scott King
Yeah, I’ve definitely messed up my neural network over the weekend! But seriously, this is very developer-centric. You’re building solutions to support specific business workflows. I remember talking with Jeff about the role of the prompt engineer. One of the first job postings we saw last year advertised a $900,000-per-year position for prompt engineering. It’s a skill, but hopefully, solutions are improving so they don’t require that level of expertise anymore.
Chris Kraus
Prompt engineering is both an art and a science. Unlike something structured like writing SQL statements—where you follow a specific pattern and understand how databases and indexes work—prompting involves understanding tone, the order of information, and how to craft effective instructions. That’s what sets Google’s guide apart from the others.
They provide a pattern or recipe to follow, which is helpful for getting started. However, there’s still an element of art involved, and results can vary. They even note in their guide, “Your mileage may vary, check for accuracy.” It’s not something you can just master by reading a book.
Scott King
Let’s move on to Google’s guide. It’s more tailored to business users, right? It gives a lot of examples. For instance, it includes persona-based examples. If you’re an admin, in marketing, HR, or management, you can find specific prompts.
Chris Kraus
Yes.
Scott King
Executives, too. The guide provides prompts for many roles. What I found interesting is that it tells you how to structure them. You need to define who you are, what you want, provide the right context, include the data, and then explain the outputs. That’s a lot of effort.
Chris Kraus
Exactly. They give you a recipe: define the persona, describe the task, provide supporting content, and specify the format for the response. It reminded me of learning to write a five-paragraph essay in school. The order of those elements, along with the tone, makes a difference.
Google’s guide says to treat the prompt as if you’re talking to another person but avoid slang and be very concise and specific. However, it feels contradictory. On one hand, they say act like you’re talking to someone, but on the other, they expect perfect, formal English and no slang. It doesn’t quite align with how most people naturally communicate.
Scott King
Yeah, or jargon, right?
Chris Kraus
Exactly. It feels like they want you to explain everything, but not actually explain it. It creates an awkward way to communicate.
Scott King
It’s like talking to a little kid. They don’t understand the important cues or context because they haven’t learned it yet. Similarly, LLMs don’t know the full backstory unless you explicitly tell them. It’s like when someone tells you a story but skips critical details, and you’re left trying to figure out who’s doing what.
When I looked at all the personas in the guide, it was repetitive. You’re essentially saying, “I’m in PR, I’m trying to do this, and I need an output for that,” or “Schedule an agenda.” It’s the same process over and over.
Chris Kraus
Exactly. If you’re asking for something like a press release, you’re giving it the format, which is probably well-known: three sentences about the company, their tagline, and a hook explaining why it’s important. But no one actually writes press releases that way. You’d start with a detailed template tailored to the company and its messaging.
I found it interesting because some of the examples in the guide didn’t feel practical. I might use an LLM to improve readability or tweak a hook, but not for the initial draft.
Scott King
Like you said, it’s useful for corrections. If you create the content yourself, you can put it into the LLM and ask it to improve readability or refine the hook. At that point, you’ve already provided all the context and content.
Chris Kraus
Exactly.
Scott King
When you provide the format upfront, you’re setting expectations instead of leaving it open-ended. If you start with just a vague prompt, I guarantee that most of the time, especially early on, you won’t like the output. It’s not good because you don’t know how to prompt effectively.
Over time, you get better. I have prompts that are half a page long. It took me hundreds of iterations to get them right, and even then, models change, and you have to fix them again.
Chris Kraus
Right.
Chris Kraus
The good thing is Google recognizes this. They say prompting should be iterative—start, refine, and keep improving. But I noticed something interesting: while defining the persona is straightforward (your job title or role), the tasks and examples were very generic. For instance, “create an agenda.”
But what kind of agenda? Is it for salespeople? A board meeting? A customer review? The output is generic unless you add more detail. Then you have to iterate again, describing specifics like what’s included in a welcome message.
Getting the right content into the prompt feels like an Achilles heel. I noticed many of the example prompts were short—around 31 characters—and the outputs weren’t much longer. Google even notes that prompts over 20 words work better, while shorter prompts of 7–9 words don’t provide enough context or content.
Scott King
Google mentioned in their data that the best prompts average 21 words with relevant context, but most people use prompts under nine words. Essentially, they’re just saying, “I want to do this,” and the system doesn’t understand what they’re asking for.
Chris Kraus
Exactly.
Chris Kraus
Google’s approach encourages including lots of details in prompts. For example, in their customer service scenario, the prompt might be: “I’m a customer service representative. I’ve received an email about broken headphones. I’ve reviewed the attached picture and confirmed the damage occurred during shipping. The customer is requesting expedited shipping.”
At that point, you’ve practically written the response yourself. What was interesting is that the model then decided to offer expedited shipping in its reply. But where did it get that from? Is there a policy or a procedure in place? Did it pull this decision from retrieval-augmented generation? For instance, it could determine that orders over $100 qualify for expedited shipping but exclude lower-value orders. It’s completing the response as it’s designed to, but the question remains—how does it determine the rules?
Scott King
Right, the response was grammatically correct, but whether it followed business rules is debatable. That’s why we emphasize orchestration. If a situation like this arises, you can escalate it to a person or someone with authority to make the decision if the model doesn’t have that capability.
Chris Kraus
Exactly.
Scott King
If the request is outside established boundaries, it should escalate to a person, and the system can learn from that. But what about privacy? One of the prompts mentioned executives sharing board meeting topics. Isn’t that insider information?
Chris Kraus
Yes, definitely.
Scott King
It’s also worth noting that Google’s guide only works with Google Docs and Gmail. If you’re using Microsoft tools, the prompts won’t integrate the same way.
Chris Kraus
Exactly.
Scott King
In the executive example, the prompt about board meeting topics felt like insider information. Doesn’t that raise privacy concerns?
Chris Kraus
You’ve highlighted two important aspects. First, you need to include references like a Google document or an email, which means you must find the right content yourself. For example, if there are 4,000 policy manuals, you need to locate the relevant one. Or, if you’re dealing with board agendas, you might have to review the last three for action items.
Second, there’s the issue of trust. If your data is hosted in Google Docs and integrated with Gemini, you’re trusting Google to protect it. But what happens if the context starts being used elsewhere? For instance, could leaked documents in Google Drive become accessible to others? Google doesn’t clearly outline how privacy works in these scenarios. Microsoft, on the other hand, suggests SharePoint security is inherited, but it’s not explicitly called out. Some companies might be more cautious, preferring not to upload sensitive data to third-party servers.
Scott King
For example, if you’re using Azure, the LLM isn’t inside your Azure—it’s hosted elsewhere, right?
Chris Kraus
Exactly, you’re crossing boundaries.
Scott King
For business users, let’s focus on what they should do depending on the tools they use—whether it’s Microsoft’s suite (Outlook, OneDrive, SharePoint) or Google Workspace (G Suite).
Chris Kraus
The key question is whether you’re trying to make one person 10% faster on their desktop or solve a broader problem. Incremental improvements like formatting or spell-checking won’t save much time overall, especially if you have to train 150 people to use it.
Instead, focus on automation. For example, certain emails—like those about damaged or delayed orders—should be processed automatically. Automation could extract the customer’s order number, verify it, and determine the appropriate response using retrieval-augmented generation. A machine can send an email offering options like a refund or a replacement without involving a person.
The value lies in letting software handle routine tasks, freeing people to focus on more complex or relationship-driven work. For major customers, personal touches—like a phone call—might still be needed, especially for accounts generating significant revenue.
Scott King
Exactly. Automation can handle basic pattern matching, like responding to common inquiries, but it can’t replace authenticity or empathy in customer interactions. Removing routine tasks helps reduce turnover, retain experienced employees, and improve the customer experience.
Teaching everyone how to write prompts seems like a waste of time when machines can eventually do this. Instead, focus on solutions that let machines handle prompting for you.
Chris Kraus
I completely agree.
Scott King
Thanks, Chris. We’ll include links to the prompting guides so everyone can take a look. If you need help, contact us at krista.ai. Until next time!
