Module 6 Assessment — LLM APIs (Python)

Module 6 Assessment — LLM APIs (Python)#

This assessment tests both your practical skills (coding tasks) and conceptual understanding (written task).

Assessment Structure#

5 Coding Tasks (80 points): Build LLM client components
1 Written Task (20 points): Explain production LLM integration

Instructions#

Coding tasks: Complete the code cells with the exact variable names shown
Written task: Fill in the string variable with full sentences
Do not rename variables
Ensure the notebook runs top-to-bottom without errors
You may use the module content for reference

Setup#

Run this cell first to set up the environment. No external packages required.

import json
import time
from typing import Optional, Dict, Any, List, Callable, TypeVar

print("Setup complete!")

Setup complete!

Task 1 — LLM Client Class (20 points) [Coding]#

Build a basic LLM client class with proper initialization.

Create a class called LLMClient with:

An __init__ method that accepts base_url, api_key (optional, default None), and model (default "phi3:mini")
Store these as instance attributes (self.base_url, self.api_key, self.model)
The base_url should have any trailing slashes removed (use rstrip('/'))

Hint:

class LLMClient:
    def __init__(self, base_url: str, api_key: str = None, model: str = "phi3:mini"):
        self.base_url = base_url.rstrip('/')
        # ... store other attributes

# Task 1: Create the LLMClient class
# YOUR CODE HERE

class LLMClient:
    pass  # Replace with your implementation

# Verification (do not modify)
try:
    test_client = LLMClient("http://localhost:11434/", api_key="test-key", model="test-model")
    print(f"base_url: {test_client.base_url}")
    print(f"api_key: {test_client.api_key}")
    print(f"model: {test_client.model}")
except Exception as e:
    print(f"Error: {e}")

Error: LLMClient() takes no arguments

Task 2 — JSON Response Parser (15 points) [Coding]#

Create a function to safely parse JSON from LLM responses.

LLMs often return JSON wrapped in markdown code blocks or with extra whitespace. Create a function called parse_json_response that:

Strips leading/trailing whitespace
Removes markdown code blocks if present (```json and ```)
Parses the cleaned text as JSON
Returns the parsed dictionary
Raises ValueError with message "Invalid JSON" if parsing fails

Hint:

def parse_json_response(text: str) -> dict:
    text = text.strip()
    if text.startswith("```json"):
        text = text[7:]  # Remove ```json
    # ... handle other cases
    try:
        return json.loads(text)
    except json.JSONDecodeError:
        raise ValueError("Invalid JSON")

# Task 2: Create the parse_json_response function
# YOUR CODE HERE

def parse_json_response(text: str) -> dict:
    pass  # Replace with your implementation

# Verification (do not modify)
test_cases = [
    '{"status": "ok"}',
    '```json\n{"status": "ok"}\n```',
    '  {"status": "ok"}  ',
]

for tc in test_cases:
    try:
        result = parse_json_response(tc)
        print(f"Parsed: {result}")
    except Exception as e:
        print(f"Error: {e}")

Parsed: None
Parsed: None
Parsed: None

Task 3 — Retry with Exponential Backoff (20 points) [Coding]#

Implement a retry function with exponential backoff.

Create a function called retry_with_backoff that:

Takes a function fn (no arguments), max_retries (default 3), and base_delay (default 1.0)
Tries to execute fn()
If it raises an exception, waits base_delay * (2 ** attempt) seconds before retrying
After max_retries failures, re-raises the last exception
Returns the result of fn() on success

Important: The delay calculation should be:

After 1st failure: base_delay * (2 ** 0) = 1.0 seconds
After 2nd failure: base_delay * (2 ** 1) = 2.0 seconds
After 3rd failure: base_delay * (2 ** 2) = 4.0 seconds

Hint:

T = TypeVar('T')

def retry_with_backoff(fn: Callable[[], T], max_retries: int = 3, base_delay: float = 1.0) -> T:
    last_exception = None
    for attempt in range(max_retries + 1):
        try:
            return fn()
        except Exception as e:
            last_exception = e
            if attempt < max_retries:
                time.sleep(base_delay * (2 ** attempt))
    raise last_exception

# Task 3: Create the retry_with_backoff function
# YOUR CODE HERE

T = TypeVar('T')

def retry_with_backoff(fn: Callable[[], T], max_retries: int = 3, base_delay: float = 1.0) -> T:
    pass  # Replace with your implementation

# Verification (do not modify)
# Test 1: Successful call
result = retry_with_backoff(lambda: "success", max_retries=3, base_delay=0.01)
print(f"Test 1 - Successful call: {result}")

# Test 2: Failing call that eventually succeeds
fail_counter = [0]
def failing_then_success():
    fail_counter[0] += 1
    if fail_counter[0] < 3:
        raise RuntimeError("Not yet!")
    return "finally worked"

result = retry_with_backoff(failing_then_success, max_retries=3, base_delay=0.01)
print(f"Test 2 - Retry success: {result} (attempts: {fail_counter[0]})")

Test 1 - Successful call: None
Test 2 - Retry success: None (attempts: 0)

Task 4 — Schema Validation (15 points) [Coding]#

Create a function to validate JSON responses against a schema.

Create a function called validate_schema that:

Takes data (a dictionary), required_fields (a list of field names)
Checks if all required fields exist in the data
If any fields are missing, raises ValueError with message "Missing fields: [field1, field2, ...]" (list the missing fields)
Returns the data if validation passes

Hint:

def validate_schema(data: dict, required_fields: list) -> dict:
    missing = [f for f in required_fields if f not in data]
    if missing:
        raise ValueError(f"Missing fields: {missing}")
    return data

# Task 4: Create the validate_schema function
# YOUR CODE HERE

def validate_schema(data: dict, required_fields: list) -> dict:
    pass  # Replace with your implementation

# Verification (do not modify)
# Test 1: Valid data
try:
    result = validate_schema({"name": "Alice", "age": 30}, ["name", "age"])
    print(f"Test 1 - Valid: {result}")
except ValueError as e:
    print(f"Test 1 - Error: {e}")

# Test 2: Missing field
try:
    result = validate_schema({"name": "Alice"}, ["name", "age"])
    print(f"Test 2 - Should have failed!")
except ValueError as e:
    print(f"Test 2 - Expected error: {e}")

Test 1 - Valid: None
Test 2 - Should have failed!

Task 5 — Build JSON Prompt (10 points) [Coding]#

Create a function to build prompts that request JSON output.

Create a function called build_json_prompt that:

Takes task (string describing what to do) and schema (dictionary showing expected output format)
Returns a prompt string that instructs the LLM to return ONLY valid JSON
The prompt must include the schema as a JSON string
The prompt must instruct the LLM to return “ONLY valid JSON” with “no other text”

Example output:

Task: Analyze the sentiment

Return ONLY valid JSON matching this schema:
{"sentiment": "positive|negative|neutral"}

Rules:
- Return ONLY the JSON object
- No other text

# Task 5: Create the build_json_prompt function
# YOUR CODE HERE

def build_json_prompt(task: str, schema: dict) -> str:
    pass  # Replace with your implementation

# Verification (do not modify)
test_schema = {"sentiment": "positive|negative|neutral", "confidence": "float"}
prompt = build_json_prompt("Analyze the sentiment of the text", test_schema)

if prompt:
    print("Generated prompt:")
    print("=" * 50)
    print(prompt)
    print("=" * 50)
    print(f"\nContains 'ONLY': {'only' in prompt.lower()}")
    print(f"Contains schema: {'sentiment' in prompt}")
else:
    print("No prompt generated")

No prompt generated

Bonus — Test Your Client (Not Graded)#

If you have access to an LLM server, you can test your components together. This is for enrichment only.

Note: This cell is not graded and may not work depending on your setup.

# BONUS: Integration test (Not Graded)
# Uncomment and modify if you want to test against a real LLM

"""
import requests

# Configuration
LLM_BASE_URL = "http://localhost:11434"  # Or your Pinggy tunnel URL
LLM_MODEL = "phi3:mini"

# Test the full pipeline
prompt = build_json_prompt(
    task="Classify the sentiment of: 'I love this product!'",
    schema={"sentiment": "positive|negative|neutral"}
)

def call_llm():
    response = requests.post(
        f"{LLM_BASE_URL}/api/chat",
        json={
            "model": LLM_MODEL,
            "messages": [{"role": "user", "content": prompt}],
            "stream": False
        },
        timeout=(5, 60)
    )
    response.raise_for_status()
    return response.json()["message"]["content"]

# Use retry with backoff
raw_response = retry_with_backoff(call_llm, max_retries=2, base_delay=1.0)
print(f"Raw response: {raw_response}")

# Parse and validate
parsed = parse_json_response(raw_response)
validated = validate_schema(parsed, ["sentiment"])
print(f"Validated result: {validated}")
"""

print("Bonus cell - uncomment code above to test with a real LLM")

Bonus cell - uncomment code above to test with a real LLM

Task 6 — Production LLM Integration (20 points) [Written]#

Prompt: Explain why production LLM systems require defensive programming.

Include:

Why LLM APIs should be treated as unreliable external services (not simple functions)
Why retry logic with exponential backoff is important
Why you must validate LLM responses before using them
Why unit tests should never call live LLM APIs

Write 6–10 sentences in your own words.

# Task 6: Written explanation

production_explanation = """

"""

Submission#

Before submitting:

Restart kernel and Run All Cells to ensure everything works
Verify all coding tasks produce the expected outputs
Verify your written explanation is complete and in your own words
Save the notebook

How to Download from Colab#

Go to File → Download → Download .ipynb
The file will download to your computer
Do not rename the file — keep it as Module6_Assessment.ipynb

Submit#

Upload your completed notebook via the Module 6 Assessment Form.

Submission Checklist#

All coding variables are filled with working code
Written explanation is thoughtful and in your own words
Notebook runs top-to-bottom without errors
Downloaded as .ipynb (not edited in a text editor)
File not renamed

Module 6 Assessment — LLM APIs (Python)

Contents

Module 6 Assessment — LLM APIs (Python)#

Assessment Structure#

Instructions#

Setup#

Task 1 — LLM Client Class (20 points) [Coding]#

Task 2 — JSON Response Parser (15 points) [Coding]#

Task 3 — Retry with Exponential Backoff (20 points) [Coding]#

Task 4 — Schema Validation (15 points) [Coding]#

Task 5 — Build JSON Prompt (10 points) [Coding]#

Bonus — Test Your Client (Not Graded)#

Task 6 — Production LLM Integration (20 points) [Written]#

Submission#

How to Download from Colab#

Submit#

Submission Checklist#