Scenario

A product inventory API returns data in paginated format. You need to fetch all pages and combine them into a single dataset for analysis.

Task

Write a Python script at /home/interview/fetch_products.py that fetches all pages from the API at http://api.products.local/products, combines the data from all pages, and saves the complete dataset to /home/interview/products.csv.

Example

Expected output format in /home/interview/products.csv:

id,name,category,price,stock
1,Product Name,Electronics,299.99,50
2,Another Product,Home & Garden,49.99,120
...

Step 1: Test the API endpoint

curl http://api.products.local/products

Review the JSON structure to understand pagination metadata and data format.

Step 2: Create the Python script

nano /home/interview/fetch_products.py

Write a script that fetches all pages and combines the data:

import requests
import pandas as pd

# Fetch first page to get pagination info
response = requests.get('http://api.products.local/products?page=1')
first_page = response.json()

total_pages = first_page['total_pages']
all_products = []

# Collect data from first page
all_products.extend(first_page['data'])

# Fetch remaining pages
for page in range(2, total_pages + 1):
    response = requests.get(f'http://api.products.local/products?page={page}')
    page_data = response.json()
    all_products.extend(page_data['data'])

# Convert to DataFrame and save as CSV
df = pd.DataFrame(all_products)
df.to_csv('/home/interview/products.csv', index=False)

print(f"Fetched {len(all_products)} products from {total_pages} pages")

Step 3: Run the script

python3 /home/interview/fetch_products.py

Step 4: Verify the output

wc -l /home/interview/products.csv
head /home/interview/products.csv

Should show 151 lines (150 products + header) with all product data combined.