Scenario

An Avro file contains user records with an embedded schema. You need to extract the schema definition for documentation or validation purposes.

Task

Write a Python script at /home/interview/extract_schema.py that reads /home/interview/data.avro, extracts its embedded schema, and saves it as a formatted JSON file to /home/interview/schema.json.

Note: The avro Python package is already installed.

Example

Expected output format in /home/interview/schema.json:

{
  "type": "record",
  "name": "User",
  "fields": [
    {"name": "id", "type": "int"},
    {"name": "name", "type": "string"},
    ...
  ]
}

Step 1: Examine the Avro file

python3 -c "from avro.datafile import DataFileReader; from avro.io import DatumReader; reader = DataFileReader(open('/home/interview/data.avro', 'rb'), DatumReader()); print('Records:', sum(1 for _ in reader)); reader.close()"

Confirms the Avro file contains records and has an embedded schema.

Step 2: Create the Python script

nano /home/interview/extract_schema.py

Write a script to extract and save the schema:

from avro.datafile import DataFileReader
from avro.io import DatumReader
import json

# Open Avro file and read schema
with DataFileReader(open('/home/interview/data.avro', 'rb'), DatumReader()) as reader:
    schema = reader.meta.get('avro.schema').decode('utf-8')

# Parse schema string to dict
schema_dict = json.loads(schema)

# Save schema as formatted JSON
with open('/home/interview/schema.json', 'w') as f:
    json.dump(schema_dict, f, indent=2)

print("Schema extracted successfully")

The schema is stored in the Avro file's metadata under the key avro.schema as a UTF-8 encoded byte string.

Step 3: Run the script

python3 /home/interview/extract_schema.py

Step 4: Verify the schema file

cat /home/interview/schema.json

Should show the complete schema definition including nested records, arrays, and union types.ss