IS4010: AI-Enhanced Application Development

Week 7: Working with External Data - Interactive Notebook

Instructor: Brandon M. Greenwell Course: IS4010 - AI-Enhanced Application Development

---

🌐 Working with External Data

From local files to the entire web 🎯

The progression: Your app → Files → JSON → APIs → The world

• Why this matters: No app exists in isolation - data flows between systems constantly
• Real-world reality: Twitter, Stripe, GitHub, OpenAI - all expose APIs you can use
• Career skill: API integration is one of the most in-demand programming capabilities
• This week: Build apps that persist data locally AND fetch live data from the internet

📝 Additional Notes: Session structure: Part 1 covers file I/O and JSON (the foundation of data persistence). Part 2 covers HTTP APIs (connecting to external services). Unified theme: JSON is the universal language that makes data portable across files and networks. Lab 07 application: You’ll combine both concepts - save/load local data AND fetch live API data.

Learning Objectives

By the end of this notebook, you will be able to: - Read and write files using Python’s context manager pattern - Work with JSON for structured data storage and exchange - Make HTTP requests to public APIs using the requests library - Parse JSON responses from APIs into Python data structures - Handle errors gracefully when working with files and network requests - Combine file persistence with API integration for real-world applications

How to Use This Notebook

• Run each cell by clicking the play button or pressing Shift+Enter
• Experiment with the code - modify values and see what happens
• Complete the exercises marked with 🏋️‍♀️ for hands-on practice
• Use AI assistants to help you understand concepts or explore variations

---

Part 1: Files & JSON 📁

🤔 The Problem: Programs Have Amnesia

When the script ends, everything disappears

Every program so far: Variables, lists, dictionaries - all gone when the program exits

Real apps need memory: - Contact lists (your phone remembers contacts even when apps close) - Game progress (video games save your state) - User preferences (VS Code remembers your recent files) - Shopping carts (e-commerce sites persist your cart)

The solution: Persistence - saving data to permanent storage (files, databases)

This enables: - Apps that remember state between runs - Data analysis on large datasets - Sharing data between programs

📝 Additional Notes: Professional context: All production applications need some form of data persistence, whether it’s files, databases, or cloud storage. Development workflow: File-based persistence is perfect for prototyping and small applications before scaling to databases.

Reading and Writing Files in Python

The standard pattern: with open() context manager

Python’s built-in file handling uses the context manager pattern for safe file operations:

• Why with?: Automatically closes the file even if errors occur
• File modes: "r" (read), "w" (write/overwrite), "a" (append)
• Best practice: Always use with - prevents file corruption and resource leaks
• 📚 Python file I/O documentation

# Write to a file (overwrites existing content)
with open("my_note.txt", "w") as f:
    f.write("Hello from our application!")

# Read the content back
with open("my_note.txt", "r") as f:
    content = f.read()
    print(content)  # Output: Hello from our application!

🏋️‍♀️ Exercise 1: File Writing Practice

1. Create a file called todo.txt with your top 3 tasks
2. Read it back and print the contents
3. Use append mode ("a") to add a 4th task

# Your code here

🚧 The Problem with Plain Text Files

Text is unstructured and hard to parse

When storing structured data in plain text files, you face several challenges:

• Custom delimiters: You have to invent your own format (like using | or ,)
• Fragile parsing: What if data contains your delimiter character?
• No nested data: How do you represent complex hierarchies?
• No type information: Everything is text - numbers, booleans, etc. need manual conversion

The real solution: We need a standard data format everyone agrees on!

# Saving a contact list to a text file - messy!
with open("contacts.txt", "w") as f:
    f.write("Alice|alice@example.com|555-1234\n")
    f.write("Bob|bob@example.com|555-5678\n")

# Reading it back - lots of manual parsing
with open("contacts.txt", "r") as f:
    for line in f:
        parts = line.strip().split("|")
        name, email, phone = parts
        print(f"{name}: {email}")

---

Enter JSON: The Universal Data Format 🌍

JavaScript Object Notation - The language of the web

JSON is a simple, human-readable text format for representing structured data:

• Created: Early 2000s by Douglas Crockford - became the web’s standard
• Why it won: Simple, language-agnostic, maps perfectly to most programming languages
• Ubiquity: APIs, config files, databases, logs - JSON is everywhere
• Python advantage: JSON maps directly to Python’s built-in types
• 📚 JSON.org official specification

📝 Additional Notes: Industry adoption: JSON has largely replaced XML for web APIs and configuration. Cross-language: Every major programming language has excellent JSON support. Human-readable: Unlike binary formats, you can open JSON files in any text editor.

JSON Structure Maps to Python

Almost identical syntax

JSON Type	Python Type	Example
Object {}	Dictionary	{"name": "Alice", "age": 30}
Array []	List	[1, 2, 3, 4, 5]
String	String	"hello"
Number	int/float	42, 3.14
Boolean	Boolean	true → True, false → False
Null	None	null → None

Key insight: If you know Python dicts/lists, you already know JSON!

Using Python’s json Library

Two key functions: dump() and load()

The json module provides simple functions for working with JSON:

• json.dump(obj, file): Python object → JSON file
• json.load(file): JSON file → Python object
  
• indent=4: Makes JSON human-readable (use in development, skip in production)

import json

# Python dictionary (complex, nested structure)
contacts = {
    "people": [
        {"name": "Alice", "email": "alice@example.com", "age": 30},
        {"name": "Bob", "email": "bob@example.com", "age": 25}
    ],
    "count": 2
}

# Write Python object to JSON file
with open("contacts.json", "w") as f:
    json.dump(contacts, f, indent=4)  # indent=4 makes it readable

print("✅ Saved contacts to contacts.json")

# Read JSON file back into Python object
with open("contacts.json", "r") as f:
    data = json.load(f)
    print(f"\nFirst contact: {data['people'][0]['name']}")
    print(f"Total contacts: {data['count']}")

🎯 What the Generated JSON Looks Like

The contacts.json file created above looks like this:

{
    "people": [
        {
            "name": "Alice",
            "email": "alice@example.com",
            "age": 30
        },
        {
            "name": "Bob",
            "email": "bob@example.com",
            "age": 25
        }
    ],
    "count": 2
}

Human-readable: You can open it in any text editor

Standard format: Any language can read this (JavaScript, Java, C#, etc.)

Portable: Email this file to a teammate, they can load it instantly

🏋️‍♀️ Exercise 2: JSON Practice

1. Create a Python dictionary representing your favorite movies (title, year, rating)
2. Save it to movies.json with nice formatting
3. Load it back and print the movie with the highest rating

# Your code here

💡 Why JSON Matters for APIs (Preview)

The connection to Part 2

Files are local: Your computer reads/writes JSON files

APIs are remote: Other computers send/receive JSON over the internet

Same format: The JSON you just learned works for BOTH

This means: Once you can work with JSON files, you can work with web APIs

Coming up: How to fetch JSON from remote servers using HTTP requests

📝 Additional Notes: Unified skill: JSON is the universal translator between systems. Your app speaks JSON, APIs speak JSON, databases speak JSON. Learning JSON once unlocks both file persistence AND web integration.

---

Part 2: Working with APIs 🔌

What is an API?

Application Programming Interface

An API is a set of rules that allows different software applications to communicate

Simple analogy: A restaurant menu - Menu (API) tells you what you can order (available operations) - You don’t need to know how the kitchen works (implementation details) - You just make a request, and get food back (data)

APIs define: - What operations are available - What data to send - What you get back

📚 What is an API? (MDN)

📝 Additional Notes: Examples you use daily: Instagram API (Buffer posts programmatically), Google Maps API (Uber/Lyft navigation), Stripe API (e-commerce payments), OpenAI API (ChatGPT-powered apps). Career relevance: Every modern application integrates with external services through APIs.

🌐 Web APIs: Apps Talking Over the Internet

HTTP as the communication protocol

Web APIs use HTTP (HyperText Transfer Protocol) to communicate over the internet

Same protocol: Your web browser uses HTTP to load websites

The exchange: 1. Your app sends an HTTP request to a URL 2. Remote server processes the request 3. Server sends back an HTTP response with data (usually JSON!)

Key insight: Reading API data is just like reading a JSON file, but from a remote server

🔍 Anatomy of a Web API Request

URL structure and HTTP methods

https://api.github.com/users/octocat/repos
└─┬─┘ └────────┬─────────┘ └─────┬────────┘
  │          base URL         endpoint path
protocol

Components: - Base URL: https://api.github.com - the server you’re talking to - Endpoint: /users/octocat/repos - the specific resource you want - HTTP method: GET (retrieve data), POST (send data), PUT (update), DELETE (remove)

For now: We’ll focus on GET requests to retrieve data

📚 HTTP methods explained

Installing the requests Library

Python’s most popular HTTP library

The requests library makes HTTP requests simple in Python:

• Not built-in: Unlike json, we need to install it first
• Why requests?: Clean, simple API - much easier than Python’s built-in urllib
• Industry standard: Used by millions of Python developers

Installation (run in your terminal, not in this notebook):

pip install requests

Let’s verify it’s installed:

# Import and check requests library
try:
    import requests
    print(f"✅ requests library installed (version {requests.__version__})")
except ImportError:
    print("❌ requests library not installed. Run: pip install requests")

Making Your First API Request

GET request to a public API

Let’s fetch data from the PokeAPI - a free, public Pokemon database:

• requests.get(url): Makes HTTP GET request, returns response object
• response.status_code: HTTP status code (200 = success)
• response.json(): Parses JSON response → Python dict (same as json.load()!)

import requests

# Make a GET request to the PokeAPI
url = "https://pokeapi.co/api/v2/pokemon/pikachu"
response = requests.get(url)

# Check if the request was successful
if response.status_code == 200:
    # Parse the JSON response into a Python dictionary
    data = response.json()
    
    print(f"Name: {data['name'].title()}")
    print(f"Height: {data['height']} decimetres")
    print(f"Weight: {data['weight']} hectograms")
    print(f"\nAbilities:")
    for ability in data['abilities']:
        print(f"  - {ability['ability']['name']}")
else:
    print(f"Error: Received status code {response.status_code}")

🏋️‍♀️ Exercise 3: Explore PokeAPI

1. Fetch data for a different Pokemon (replace “pikachu” with another name)
2. Print the Pokemon’s types (hint: look at data['types'])
3. Try an invalid Pokemon name - what happens?

# Your code here

📊 HTTP Status Codes

The server’s way of telling you what happened

Code	Meaning	Example
200	OK - Success	Data retrieved successfully
201	Created	New resource created
400	Bad Request	Invalid data sent
401	Unauthorized	Need authentication
404	Not Found	Resource doesn’t exist
500	Server Error	Something broke on server

Always check: response.status_code before processing data

Error handling: Different codes need different responses

📚 HTTP status codes reference

🔗 The JSON Connection: Same Format, Different Source

Comparison: Files vs APIs

Reading JSON from a file:

import json

with open("data.json", "r") as f:
    data = json.load(f)
    print(data["name"])

Reading JSON from an API:

import requests

response = requests.get(url)
data = response.json()
print(data["name"])

Same result: Both give you a Python dictionary

Same skills: Working with dicts, lists, accessing nested data

Different source: One is local, one is remote

Key takeaway: JSON knowledge transfers directly between files and APIs!

🚀 Real-World API Examples

APIs you can use right now (no API key needed)

These public APIs are perfect for learning and practice:

• PokeAPI: Pokemon data (what we just used)
• REST Countries: Country data (population, flags, currencies)
• The Dog API: Random dog pictures and breed info
• JokeAPI: Random jokes and puns
• Chuck Norris API: Random Chuck Norris jokes
• Cat Facts API: Random cat facts

Thousands more: Public APIs directory

📝 Additional Notes: Lab 07 application: Students will choose one of these APIs to build a data viewer application. API keys: Some APIs require free API keys (OpenWeatherMap, NASA) - simple signup process. Best practice: Always explore API documentation before coding.

🏋️‍♀️ Exercise 4: Try Different APIs

Pick one of the APIs above and: 1. Read its documentation to find an interesting endpoint 2. Make a GET request to that endpoint 3. Parse and display the interesting data from the response

# Your code here - explore a new API!

🛡️ Error Handling with APIs

Networks are unreliable - plan for failure

APIs can fail for many reasons: - Network connection issues - Server downtime - Invalid requests - Rate limiting - Slow responses

Professional code handles errors gracefully:

import requests

url = "https://pokeapi.co/api/v2/pokemon/charizard"

try:
    response = requests.get(url, timeout=5)  # 5 second timeout
    response.raise_for_status()  # Raises exception for 4xx/5xx codes

    data = response.json()
    print(f"✅ Success: {data['name'].title()}")
    print(f"   Types: {', '.join(t['type']['name'] for t in data['types'])}")

except requests.exceptions.Timeout:
    print("❌ Error: Request timed out")
except requests.exceptions.ConnectionError:
    print("❌ Error: Could not connect to server")
except requests.exceptions.HTTPError as e:
    print(f"❌ Error: HTTP {e.response.status_code}")
except requests.exceptions.JSONDecodeError:
    print("❌ Error: Response was not valid JSON")

Error Handling Best Practices

• Timeouts: Set with timeout= parameter (seconds) - don’t let your app hang forever
• raise_for_status(): Converts error codes (4xx, 5xx) into Python exceptions
• Specific exceptions: Catch different error types for specific handling
• User-friendly messages: Don’t show raw error traces to end users

⚡ API Best Practices

Being a good API citizen

Read the docs first: Every API has different rules and endpoints

Respect rate limits: Most free APIs limit requests (e.g., 1000/day) - GitHub API: 60 requests/hour (unauthenticated), 5000/hour (authenticated) - OpenWeatherMap: 1000 requests/day (free tier) - Exceeding limits → 429 Too Many Requests error

Cache responses: Don’t request the same data repeatedly

Use timeouts: Don’t let your app hang forever waiting for response

Check status codes: Handle errors gracefully

API keys: Keep them secret (use .env files, never commit to git)

📚 API development best practices

📝 Additional Notes: Professional context: Rate limiting is essential for API sustainability. Excessive requests can get your IP address banned. Caching: Save API responses locally to reduce calls and improve performance.

🎯 Putting It All Together: Files + APIs

A complete data workflow

This example demonstrates the full pattern: Fetch → Process → Store

Why cache API data locally? - Faster loading: No network delay when reading from disk - Works offline: Data available even without internet - Reduces API calls: Avoid hitting rate limits - Real apps do this: Weather apps, news readers, social media feeds

📝 Additional Notes: Production pattern: Most real applications cache API responses with expiration timestamps. Database alternative: For simple apps, JSON files work great as a lightweight database.

import json
import requests

# 1. Fetch live data from API
print("📡 Fetching Pokemon data from API...")
response = requests.get("https://pokeapi.co/api/v2/pokemon/ditto")
pokemon_data = response.json()

# 2. Process the data (extract what we need)
simplified = {
    "name": pokemon_data["name"],
    "height": pokemon_data["height"],
    "weight": pokemon_data["weight"],
    "types": [t["type"]["name"] for t in pokemon_data["types"]],
    "abilities": [a["ability"]["name"] for a in pokemon_data["abilities"]]
}

# 3. Save to local JSON file for offline use
with open("pokemon_cache.json", "w") as f:
    json.dump(simplified, f, indent=4)

print("✅ Data fetched from API and saved locally!")
print(f"\n{simplified['name'].title()} has been cached:")
print(f"  Height: {simplified['height']} decimetres")
print(f"  Weight: {simplified['weight']} hectograms")
print(f"  Types: {', '.join(simplified['types'])}")
print(f"  Abilities: {', '.join(simplified['abilities'])}")

# 4. Later, load from cache instead of making another API call
print("\n📂 Loading from cache...")
with open("pokemon_cache.json", "r") as f:
    cached_data = json.load(f)
    print(f"Cached: {cached_data['name'].title()} (no API call needed!)")

🏋️‍♀️ Exercise 5: Complete Workflow

Build a complete data workflow: 1. Choose an API from the list above 2. Fetch data from that API 3. Process/filter the data to extract interesting information 4. Save it to a JSON file 5. Load it back and display a formatted report

# Your complete workflow here

---

Summary and Next Steps

What You’ve Learned 🎯

1. File I/O basics - Reading and writing files with the with open() context manager
2. JSON format - Universal data exchange format that maps to Python dictionaries
3. API fundamentals - HTTP requests, status codes, and JSON responses
4. Requests library - Making GET requests and parsing responses
5. Error handling - Timeouts, exceptions, and graceful failure
6. Complete workflow - Fetching data, processing it, and caching locally

Key Takeaways ✨

• Persistence: Files let your programs remember state between runs
• JSON: Universal data format - works for files AND APIs
• APIs: Let your apps communicate with other systems over the internet
• HTTP: The protocol that powers the web and most APIs
• Requests library: Makes HTTP requests simple in Python
• Error handling: Always check status codes and handle failures gracefully
• The connection: json.load() (files) and response.json() (APIs) both give you Python dicts

Career Relevance 🚀

Technical Interview Preparation

• API integration: “How would you integrate Stripe payment processing?”
• Data handling: “Design a weather dashboard that caches forecasts”
• Error handling: “How do you handle network failures gracefully?”

Industry Applications

• Every modern app: Integrates with external services (payments, auth, maps, notifications)
• Job requirements: “Experience with RESTful APIs” appears in 80%+ of backend/full-stack postings
• Real companies using APIs:
  • Stripe: Payment processing API ($95B valuation)
  • Twilio: SMS/Voice API ($12B valuation)
  • Mapbox: Maps API (powers Snapchat, Instacart)

Next Steps 🎯

1. Complete Lab 07 - Build contact book + API data viewer
2. Explore APIs - Browse Public APIs directory
3. Cache strategically - Implement expiration times for cached data
4. Practice error handling - Make your API code production-ready
5. Build a project - Create a CLI tool that uses your favorite API

Resources for Continued Learning 📚

Official Documentation

• Python file I/O tutorial
• JSON module documentation
• Requests library docs
• HTTP methods reference

Learning Resources

• Real Python: Working with APIs
• Public APIs directory
• HTTP status codes
• API design best practices

AI-Assisted Development

• ChatGPT/Claude prompts: “Help me understand how to parse this API response: [paste JSON]”
• GitHub Copilot: Generate API request boilerplate and error handling
• Code review: Ask AI assistants to review your error handling patterns

---

Final Thoughts 💭

Working with external data - whether from files or APIs - is fundamental to modern software development. The skills you’ve learned here enable you to build applications that persist data, integrate with external services, and provide real value to users.

JSON is your universal translator - master it once, use it everywhere. Files, APIs, databases, configuration - JSON is the common language that connects systems.

Practice building real applications! Choose APIs that interest you, cache the data intelligently, and build tools that solve actual problems. This is how you develop portfolio-worthy projects.

Now go fetch some data! 🌐📊