Prompt Engineering

[Role] + [Context] + [Task] + [Format] + [Examples] + [Constraints]

You are an expert technical writer specializing in artificial intelligence.

Context: We are creating documentation for developers using our new NLP API.
The target audience has intermediate Python knowledge but limited NLP experience.

Task: Write a concise explanation of tokenization in NLP, suitable for our API documentation.
Explain what tokenization is, why it's important, and provide a simple Python example using our API.

Format: Use Markdown format with:
- Clear section headings
- Brief introduction
- Key points in bullet format
- Code example with comments
- Summary of benefits

Examples:
1. For word tokenization:
```python
from our_api import Tokenizer
text = "Hello world!"
tokens = Tokenizer.word_tokenize(text)
# Output: ['Hello', 'world', '!']

from our_api import Tokenizer
text = "Hello world! How are you?"
sentences = Tokenizer.sentence_tokenize(text)
# Output: ['Hello world!', 'How are you?']

## Prompt Engineering Techniques

### Basic Techniques

#### Zero-Shot Prompting
```markdown
Classify the following text as positive, negative, or neutral:

"The new AI model performs exceptionally well on various benchmarks."

Sentiment:

Classify the following texts as positive, negative, or neutral:

Text: "I love this product!" → Sentiment: Positive
Text: "This is terrible quality." → Sentiment: Negative
Text: "The item arrived on time." → Sentiment: Neutral

Text: "The new AI model performs exceptionally well on various benchmarks." → Sentiment:

Write a professional email to a client explaining a project delay.
Include the following information:
- Reason for delay: Unexpected technical challenges
- New estimated completion date: June 15, 2025
- Steps being taken to resolve issues: Additional testing and code review
- Apology for inconvenience
- Offer to discuss further if needed

Email:

Solve this math problem step by step:

A company has 120 employees. 60% work in development, 25% in marketing, and the rest in administration.
If 40% of development employees work remotely, how many development employees work on-site?

Let's think step by step:
1. Total employees = 120
2. Development employees = 60% of 120 =
3. Marketing employees = 25% of 120 =
4. Administration employees = 120 - (development + marketing) =
5. Remote development employees = 40% of development employees =
6. On-site development employees = development employees - remote development employees =

You are a senior data scientist with 10 years of experience in machine learning.
Explain the concept of overfitting to a junior developer who has basic statistics knowledge.
Use simple analogies and avoid complex mathematical formulas.
Include a practical example of how to detect and prevent overfitting.

Explanation:

First, let's analyze this customer review:
"The product works well but the interface is confusing. The setup took longer than expected."

1. Identify the positive aspects mentioned:
2. Identify the negative aspects mentioned:
3. Suggest improvements based on the feedback:
4. Write a response to the customer acknowledging their feedback:

Write a product description for a new AI-powered writing assistant.
Follow these constraints:
- Maximum 150 words
- Must include: grammar checking, style suggestions, plagiarism detection
- Target audience: university students
- Tone: professional but approachable
- Must not mention specific competitors
- Include a call-to-action

Product Description:

Explain [CONCEPT] to [AUDIENCE] with [BACKGROUND] knowledge.
Include:
- Definition in simple terms
- Key components/steps
- Practical example
- Common use cases
- Potential challenges

Constraints:
- Keep explanation under [WORD_LIMIT] words
- Use [TONE] tone
- Avoid [EXCLUDED_TOPICS]

Write [TYPE_OF_CONTENT] about [TOPIC] for [TARGET_AUDIENCE].
Include the following sections:
1. [SECTION_1]
2. [SECTION_2]
3. [SECTION_3]

Format: [FORMAT_REQUIREMENTS]
Style: [STYLE_GUIDELINES]
Length: [WORD_COUNT] words
Constraints:
- Must include [REQUIRED_ELEMENTS]
- Must not include [PROHIBITED_ELEMENTS]
- Use [SPECIFIC_TERMS] where appropriate

Analyze the following [DATA_TYPE] data about [TOPIC]:
[DATA_SAMPLE]

Perform the following analysis:
1. [ANALYSIS_TASK_1]
2. [ANALYSIS_TASK_2]
3. [ANALYSIS_TASK_3]

Provide:
- Key insights
- Visualization suggestions
- Potential business implications
- Limitations of the analysis

Format results as [OUTPUT_FORMAT]

Classify the following customer support tickets into these categories:
- Billing Issue
- Technical Problem
- Feature Request
- General Inquiry
- Complaint

For each ticket, provide:
1. The classification
2. Confidence level (Low/Medium/High)
3. Key phrases that influenced the classification

Ticket 1: "I can't log in to my account. It says my password is incorrect even though I'm sure it's right."
Classification:

Ticket 2: "When will you add dark mode to the mobile app? It's hard to use at night."
Classification:

Ticket 3: "I was charged twice for my last subscription renewal. Please refund the extra charge."
Classification:

Extract all named entities from the following text and classify them into these types:
- PERSON
- ORGANIZATION
- LOCATION
- DATE
- PRODUCT

Format the output as: [Entity] (Type)

Text: "Microsoft announced on March 15 that Satya Nadella will visit Paris next month to launch Azure AI Studio, their new cloud-based machine learning platform."

Extracted Entities:

Summarize this research abstract in 3 concise bullet points suitable for a non-technical audience.
Focus on the main findings and practical implications.

Research Abstract:
[INSERT ABSTRACT HERE]

Summary:
- Key finding 1:
- Key finding 2:
- Practical implication:

Write a Python function that [DESCRIBE_FUNCTION].
Requirements:
- Function name: [FUNCTION_NAME]
- Parameters: [PARAMETERS]
- Return value: [RETURN_DESCRIPTION]
- Handle these edge cases: [EDGE_CASES]
- Include docstring with [DOCSTRING_FORMAT]
- Follow [CODING_STANDARDS]

Example usage:
[EXAMPLE_INPUT] → [EXAMPLE_OUTPUT]

Function:

Explain the following Python code to a junior developer.
Break down:
1. What the code does overall
2. Purpose of each function
3. Key variables and their roles
4. How the algorithm works
5. Potential improvements or edge cases to consider

Code:
[INSERT CODE HERE]

Explanation:

Analyze this error message and code snippet.
Provide:
1. Root cause of the error
2. Step-by-step fix
3. Explanation of why this fix works
4. Prevention tips for similar errors

Error Message:
[INSERT ERROR MESSAGE]

Code Snippet:
[INSERT CODE SNIPPET]

Analysis:

Write 3 variations of a social media post promoting [PRODUCT/SERVICE].
Each variation should:
- Target [TARGET_AUDIENCE]
- Highlight [KEY_FEATURES]
- Include [CTA_TYPE]
- Use [TONE] tone
- Be under [WORD_LIMIT] words

Variation 1 (Educational):
Variation 2 (Benefit-focused):
Variation 3 (Urgency-driven):

Develop a 90-day marketing strategy for [PRODUCT] targeting [TARGET_MARKET].
Include:
1. Key objectives
2. Target audience analysis
3. Channel selection with rationale
4. Content plan (types and frequency)
5. Budget allocation
6. Success metrics
7. Potential challenges and mitigation

Format as a structured document with clear headings.

Generate 5 innovative feature ideas for [PRODUCT_TYPE] that solves [PROBLEM].
For each idea, provide:
1. Feature name
2. Brief description
3. Key benefits
4. Potential implementation challenges
5. User experience flow

Target user: [USER_PERSONA]
Technical constraints: [CONSTRAINTS]

from langchain import PromptTemplate, LLMChain
from langchain.llms import OpenAI

# Define prompt template
template = """You are an expert {domain} with {experience} years of experience.

Task: {task_description}

Context: {context}

Format the response as:
{format_instructions}

Constraints:
{constraints}

Response:"""

prompt = PromptTemplate(
    input_variables=["domain", "experience", "task_description", "context", "format_instructions", "constraints"],
    template=template
)

# Create LLM chain
llm = OpenAI(temperature=0.7)
chain = LLMChain(llm=llm, prompt=prompt)

# Use the chain
response = chain.run({
    "domain": "machine learning",
    "experience": "10",
    "task_description": "Explain transformer architecture to a software engineer",
    "context": "The engineer has experience with CNNs but not transformers",
    "format_instructions": "1. Introduction\n2. Key components\n3. Comparison with CNNs\n4. Practical applications",
    "constraints": "Keep explanation under 300 words. Avoid complex math. Use analogies."
})

print(response)

class PromptManager:
    def __init__(self):
        self.prompts = {}
        self.versions = {}

    def add_prompt(self, name, prompt_text, metadata=None):
        """Add a new prompt or version"""
        if name not in self.prompts:
            self.prompts[name] = []
            self.versions[name] = 0

        version = self.versions[name] + 1
        self.prompts[name].append({
            "version": version,
            "text": prompt_text,
            "metadata": metadata or {},
            "created_at": datetime.now()
        })
        self.versions[name] = version
        return version

    def get_prompt(self, name, version=None):
        """Retrieve a specific prompt version"""
        if name not in self.prompts:
            raise ValueError(f"Prompt {name} not found")

        if version is None:
            return self.prompts[name][-1]  # Return latest

        for prompt in self.prompts[name]:
            if prompt["version"] == version:
                return prompt

        raise ValueError(f"Version {version} not found for prompt {name}")

    def compare_versions(self, name, version1, version2):
        """Compare two prompt versions"""
        prompt1 = self.get_prompt(name, version1)
        prompt2 = self.get_prompt(name, version2)

        return {
            "version1": prompt1["text"],
            "version2": prompt2["text"],
            "differences": self._diff_text(prompt1["text"], prompt2["text"])
        }

    def _diff_text(self, text1, text2):
        """Simple text diff implementation"""
        import difflib
        differ = difflib.Differ()
        return list(differ.compare(text1.splitlines(), text2.splitlines()))

class PromptEvaluator:
    def __init__(self, model):
        self.model = model

    def evaluate_prompt(self, prompt, test_cases, metrics):
        """Evaluate a prompt on multiple test cases and metrics"""
        results = []

        for test_case in test_cases:
            # Generate response
            response = self.model.generate(prompt.format(**test_case["input"]))

            # Evaluate metrics
            case_result = {"test_case": test_case["id"], "response": response}
            for metric in metrics:
                case_result[metric] = self._calculate_metric(metric, response, test_case)

            results.append(case_result)

        return self._aggregate_results(results)

    def _calculate_metric(self, metric, response, test_case):
        """Calculate specific metric"""
        if metric == "accuracy":
            return self._calculate_accuracy(response, test_case["expected"])
        elif metric == "relevance":
            return self._calculate_relevance(response, test_case["input"])
        elif metric == "consistency":
            return self._calculate_consistency(response, test_case.get("previous_responses", []))
        elif metric == "completeness":
            return self._calculate_completeness(response, test_case["requirements"])
        elif metric == "conciseness":
            return self._calculate_conciseness(response)
        else:
            raise ValueError(f"Unknown metric: {metric}")

    def _calculate_accuracy(self, response, expected):
        """Calculate accuracy metric"""
        # Implement based on task type
        if isinstance(expected, list):  # Multiple possible correct answers
            return 1 if response.strip().lower() in [e.lower() for e in expected] else 0
        else:
            return 1 if response.strip().lower() == expected.lower() else 0

    def _calculate_relevance(self, response, input_data):
        """Calculate relevance metric (0-1)"""
        # Implement using semantic similarity or keyword matching
        return 0.8  # Placeholder

    def _aggregate_results(self, results):
        """Aggregate results across test cases"""
        aggregated = {"overall": {}, "per_case": results}

        # Calculate average for each metric
        metrics = results[0].keys() - {"test_case", "response"}
        for metric in metrics:
            values = [r[metric] for r in results]
            aggregated["overall"][metric] = sum(values) / len(values)

        return aggregated

# Example usage
evaluator = PromptEvaluator(model)
test_cases = [
    {
        "id": "case1",
        "input": {"question": "What is the capital of France?"},
        "expected": ["Paris", "paris"],
        "requirements": ["city name"]
    },
    {
        "id": "case2",
        "input": {"question": "Explain photosynthesis in simple terms"},
        "requirements": ["sunlight", "plants", "oxygen", "carbon dioxide"]
    }
]

metrics = ["accuracy", "relevance", "completeness", "conciseness"]
results = evaluator.evaluate_prompt("Answer the question: {question}", test_cases, metrics)

class PromptOptimizer:
    def __init__(self, model, evaluator):
        self.model = model
        self.evaluator = evaluator

    def optimize_prompt(self, initial_prompt, test_cases, metrics, iterations=5):
        """Optimize prompt through iterative refinement"""
        current_prompt = initial_prompt
        best_score = -1
        best_prompt = current_prompt
        history = []

        for i in range(iterations):
            # Evaluate current prompt
            results = self.evaluator.evaluate_prompt(current_prompt, test_cases, metrics)
            current_score = results["overall"]["accuracy"]  # Primary metric

            # Track history
            history.append({
                "iteration": i,
                "prompt": current_prompt,
                "score": current_score,
                "results": results
            })

            # Update best prompt
            if current_score > best_score:
                best_score = current_score
                best_prompt = current_prompt

            # Generate variations
            variations = self._generate_variations(current_prompt, results)

            # Select best variation for next iteration
            current_prompt = self._select_best_variation(variations, test_cases, metrics)

        return {
            "best_prompt": best_prompt,
            "best_score": best_score,
            "history": history,
            "optimization_report": self._generate_report(history)
        }

    def _generate_variations(self, prompt, evaluation_results):
        """Generate prompt variations based on evaluation"""
        variations = []

        # Variation 1: Add more examples if completeness is low
        if evaluation_results["overall"]["completeness"] < 0.7:
            variations.append(self._add_examples(prompt))

        # Variation 2: Clarify instructions if relevance is low
        if evaluation_results["overall"]["relevance"] < 0.7:
            variations.append(self._clarify_instructions(prompt))

        # Variation 3: Add constraints if responses are too verbose
        if evaluation_results["overall"]["conciseness"] < 0.6:
            variations.append(self._add_constraints(prompt))

        # Variation 4: Rephrase for better clarity
        variations.append(self._rephrase_prompt(prompt))

        # Variation 5: Add role definition
        variations.append(self._add_role(prompt))

        return variations

    def _select_best_variation(self, variations, test_cases, metrics):
        """Select the best variation based on evaluation"""
        best_score = -1
        best_variation = variations[0]

        for variation in variations:
            results = self.evaluator.evaluate_prompt(variation, test_cases, metrics)
            score = results["overall"]["accuracy"]

            if score > best_score:
                best_score = score
                best_variation = variation

        return best_variation

    # Helper methods for generating variations
    def _add_examples(self, prompt):
        """Add more examples to the prompt"""
        # Implementation depends on prompt structure
        return prompt + "\n\nExamples:\n1. [EXAMPLE 1]\n2. [EXAMPLE 2]"

    def _clarify_instructions(self, prompt):
        """Make instructions more explicit"""
        return prompt.replace("Write", "Write a detailed explanation with examples")

    def _add_constraints(self, prompt):
        """Add constraints to the prompt"""
        return prompt + "\n\nConstraints:\n- Keep response under 200 words\n- Use simple language"

    def _rephrase_prompt(self, prompt):
        """Rephrase the prompt for better clarity"""
        # Could use the model itself to rephrase
        rephrase_prompt = f"Rephrase this prompt to make it clearer and more effective:\n\n{prompt}"
        return self.model.generate(rephrase_prompt)

    def _add_role(self, prompt):
        """Add role definition to the prompt"""
        return f"You are an expert in the field. {prompt}"

    def _generate_report(self, history):
        """Generate optimization report"""
        report = {
            "initial_score": history[0]["score"],
            "final_score": history[-1]["score"],
            "improvement": history[-1]["score"] - history[0]["score"],
            "iterations": []
        }

        for iteration in history:
            report["iterations"].append({
                "iteration": iteration["iteration"],
                "score": iteration["score"],
                "prompt_length": len(iteration["prompt"]),
                "key_changes": self._identify_changes(iteration["prompt"], history[0]["prompt"])
            })

        return report

    def _identify_changes(self, new_prompt, original_prompt):
        """Identify key changes between prompt versions"""
        # Simple implementation - could be enhanced
        changes = []
        new_lines = new_prompt.split("\n")
        original_lines = original_prompt.split("\n")

        if len(new_lines) > len(original_lines):
            changes.append(f"Added {len(new_lines) - len(original_lines)} lines")
        elif len(new_lines) < len(original_lines):
            changes.append(f"Removed {len(original_lines) - len(new_lines)} lines")

        # Check for specific additions
        if "Examples:" not in original_prompt and "Examples:" in new_prompt:
            changes.append("Added examples section")

        if "Constraints:" not in original_prompt and "Constraints:" in new_prompt:
            changes.append("Added constraints section")

        if "You are an expert" not in original_prompt and "You are an expert" in new_prompt:
            changes.append("Added role definition")

        return changes if changes else ["Minor rephrasing"]