Python

 

 

 

 

Python - Image Meta Data

 

This program is written for my personal necessity but I decided to share since this would be useful for others as well. I have a lot of images taken by mobile phone and backed up in harddisk. As I am getting more and more images backed up in my harddisk, it is getting more and more difficult to figure out which of the image file is for what and where it was taken etc.

So I thought it would handy if I can generate html table that shows each of the image shown as a thumbnail and additional information about the image (e.g, location, date/time etc) as shown below. I also wanted to get an enlarged image when I click on a thumbnail.

 

 

To do this, I need to implement two impartant functionalities :

  • The function to extract meta data (GPS location, Date/Time etc) from the image
  • The function to get the human readable location (address) from the GPS data.

I requested chatGPT (GPT 4) to write a phython script for this on Apr 25, 2023. Of course, it was not done by single shot and went through many many rounds of back-and-forth until I get some working in meaninful way.

For the first function (i.e, extract meta data from the image), I could do everything with Python with a couple of module installation. But for the second function (i.e, get human readable address from GPS information), I needed to use Google MapAPI.

Folliwng is the final verion of the code that was proved to work as I intended.

 

NOTE : If you are not familiar with how to use Google Map API, I would suggest you to check out this note first.

NOTE : This program raise a request for each of the image that contains GPS information. Each of the API call to Google Map generate some cost. So when you are just testing the code, use a test directory which contains a small number of images. Otherwise, you would get charged a lot just during the test.

 

NOTE : Challenges

The first version that chatGPT produced was pretty good in terms of overall framework, but it didn't work well for my images. What I spent most of my time for debugging were as follows :

  • The image that I have from various different devices (very old Blackberry device, old iPhone, old SamSung and another relatively new android phone) and each of the device gives a little bit different set of meta data information. I needed to ask chatGPT to add more flexibility e.g) handling missing values etc.
  • Not all devices gives the values with the exactly same format.
  • Division by Zero error in _convert_to_degrees(value). The root cause of this problem was not clear and went through so many trial and error.

 

 

GetImageMetaInfo.py

# This code is a Python script designed to extract metadata from a collection of image files in various formats

# (such as JPEG, PNG, GIF, and BMP) and generate an HTML table containing this metadata. The main features of

# the script include:

#

# 1. Extracting EXIF metadata from images, particularly GPS information (latitude, longitude, etc.) using the Python

#    Imaging Library (PIL) module.

# 2. Reverse geocoding the GPS coordinates to retrieve the human-readable address of the location where the image

#     was taken using the Google Maps Geocoding API.

# 3. Generating a base64-encoded thumbnail version of each image.

# 4. Creating an HTML table containing metadata for each image, including the human-readable location, thumbnail,

#     and other EXIF tags.

# 5. Saving the generated HTML table to a file named "image_metadata.html" in the same directory as the source

#     images.

#

# The script takes the path of the source directory containing the images and a Google Maps API key as inputs.

# The main function iterates through

# the image files in the source directory, extracts the metadata, and then calls helper functions to create the HTML

# table and save it to a file.

 

import os

import requests

from PIL import Image

from PIL.ExifTags import TAGS

import html

import base64

from io import BytesIO

from fractions import Fraction

 

CUSTOM_GPSTAGS = {

    0: "GPSVersionID",

    1: "GPSLatitudeRef",

    2: "GPSLatitude",

    3: "GPSLongitudeRef",

    4: "GPSLongitude",

    5: "GPSAltitudeRef",

    6: "GPSAltitude",

    7: "GPSTimeStamp",

    29: "GPSDateStamp",

}

 

 

# get_exif_data(image_path) is a function that takes an image file path as its argument and extracts the EXIF

# metadata from the image using the Python

# Imaging Library (PIL) module. It then returns a dictionary containing the extracted metadata.

#

# Here's a step-by-step breakdown of what the function does:

#

# 1. Open the image file using Image.open(image_path) from the PIL module.

# 2. Initialize an empty dictionary called exif_data.

# 3. Check if the image has EXIF data by calling img._getexif(). If it does, proceed with the following steps;

#     otherwise, print a message stating that no metadata was found for the image.

# 4. Iterate through the key-value pairs of the EXIF data using a for loop.

# 5. For each key-value pair, get the human-readable tag name by looking up the key in the TAGS dictionary from

#    PIL.ExifTags. If the key is not found in TAGS, use the key itself as the tag name.

# 6. Check if the current tag is "GPSInfo". If it is, create a dictionary called gps_data and iterate through its

#    key-value pairs.

#    i) For each key-value pair in the GPS data, get the custom GPS tag name by looking up the key in the

#       CUSTOM_GPSTAGS dictionary. If the key is not found, use the key itself as the tag name.

#    ii) Add the GPS tag and its corresponding value to the gps_data dictionary.

#    iii) Add the "GPSInfo" tag to the exif_data dictionary with gps_data as its value.

# 7. For all other tags, add the tag and its corresponding value to the exif_data dictionary.

# 8. Return the exif_data dictionary containing the extracted EXIF metadata.

#

def get_exif_data(image_path):

    img = Image.open(image_path)

    exif_data = {}

 

    if img._getexif():

        for tag_id, value in img._getexif().items():

            tag = TAGS.get(tag_id, tag_id)

            if tag == "GPSInfo":

                gps_data = {}

                for t in value:

                    gps_tag = CUSTOM_GPSTAGS.get(t, t)

                    gps_data[gps_tag] = value[t]

                exif_data[tag] = gps_data

            else:

                exif_data[tag] = value

    else:

        print(f"No metadata found for {image_path}")

 

    return exif_data

 

 

# reverse_geocode(lat, lng, api_key) is a function that takes latitude (lat), longitude (lng), and a Google Maps API

# key (api_key) as its arguments.

# The function uses the Google Maps Geocoding API to convert the given GPS coordinates (latitude and longitude)

# into a human-readable address.

#

# Here's an overview of what the function does:

#

# 1. Construct the URL for the Google Maps Geocoding API by interpolating the latitude, longitude, and API key into

#    the URL template.

# 2. Make an HTTP GET request to the API using the requests.get(url) method from the requests library.

# 3. Parse the JSON response using the response.json() method to obtain the geocoding data.

# 4. Check if the status of the API response is "OK". If it is, proceed with the following steps; otherwise, return None.

# 5. Extract the human-readable address from the API response data. This is typically found in the

#    "formatted_address" field of the first result.

# 6. Print the address to the console.

# 7. Return the human-readable address.

#

# This function essentially takes GPS coordinates and converts them into a more understandable location description

#  (e.g., street address) by querying the Google Maps Geocoding API.

#

def reverse_geocode(lat, lng, api_key):

    url = f"https://maps.googleapis.com/maps/api/geocode/json?latlng={lat},{lng}&key={api_key}"

    response = requests.get(url)

    data = response.json()

 

    if data["status"] == "OK":

        print("Address : ", data["results"][0]["formatted_address"])

        return data["results"][0]["formatted_address"]

    else:

        return None

 

 

# get_human_readable_location(metadata, api_key) is a function that takes a dictionary containing image metadata

# (metadata) and a Google Maps API key

# (api_key) as its arguments. The function aims to retrieve a human-readable location description (e.g., street

#  address) using the GPS information found in the metadata, if available.

#

# Here's an overview of what the function does:

#

# 1. Check if the "GPSInfo" key is present in the metadata. If it is, proceed with the following steps; otherwise,

#     print a message stating that no GPSInfo tag was found for the metadata and return None.

# 2. Extract the GPS information from the metadata.

# 3. Initialize variables for latitude_key, latitude_ref_key, longitude_key, and longitude_ref_key as None.

# 4. Iterate through the keys in the GPS information dictionary.

#    For each key, check if it corresponds to one of the following GPS data components: GPSLatitude,

#    GPSLatitudeRef, GPSLongitude, or GPSLongitudeRef.

#    If it does, store the key in the appropriate variable (e.g., latitude_key, latitude_ref_key, etc.).

# 5. Check if all four GPS data components have been found (latitude_key, latitude_ref_key, longitude_key, and

#     longitude_ref_key). If so, proceed with the following steps; otherwise, return None.

# 6. Extract the latitude, latitude reference, longitude, and longitude reference values from the GPS information.

# 7. Convert the latitude and longitude values to decimal degrees using the _convert_to_degrees(value) helper

#    function.

# 8. Adjust the sign of the latitude and longitude values based on the reference values (e.g., if latitude_ref is not "N",

#     make the latitude value negative).

# 9. Call the reverse_geocode(lat, lng, api_key) function with the decimal latitude and longitude values, as well as

#     the API key, to obtain the human-readable location.

# 10. Return the human-readable location.

#

# This function is responsible for extracting GPS coordinates from the metadata of an image and converting them

# into a location description that is more easily understood by humans, such as a street address.

#

def get_human_readable_location(metadata, api_key, filename):

    if "GPSInfo" in metadata:

        gps_info = metadata["GPSInfo"]

        latitude_key = None

        latitude_ref_key = None

        longitude_key = None

        longitude_ref_key = None

 

        CUSTOM_GPSTAGS = {k: v for k, v in TAGS.items() if v.startswith("GPS")}

 

        for key in gps_info.keys():

            if key == "GPSLatitude":

                latitude_key = key

 

            elif key == 'GPSLongitude':

                longitude_key = key

 

            elif key == 'GPSLatitudeRef':

                latitude_ref_key = key

 

            elif key == 'GPSLongitudeRef':

                longitude_ref_key = key

 

        if latitude_key and latitude_ref_key and longitude_key and longitude_ref_key:

            latitude = gps_info[latitude_key]

            latitude_ref = gps_info[latitude_ref_key]

            longitude = gps_info[longitude_key]

            longitude_ref = gps_info[longitude_ref_key]

 

            lat = _convert_to_degrees(latitude)

            if lat is not None and latitude_ref != "N":

                lat = -lat

            lng = _convert_to_degrees(longitude)

            if lng is not None and longitude_ref != "E":

                lng = -lng

 

            if lat is not None and lng is not None:

                location = reverse_geocode(lat, lng, api_key)

                return location

            else:

                #print(f"Error converting latitude and/or longitude to degrees for {metadata}")

                print(f"Error converting latitude and/or longitude to degrees for {filename}")

    else:

        #print(f"No GPSInfo tag found for {metadata}")

        print(f"No GPSInfo tag found for {filename}")

 

    return None

 

 

 

# _convert_to_degrees(value) is a helper function that takes a tuple (value) containing three components

#  - degrees, minutes, and seconds - and converts them into decimal degrees.

#

# Here's a step-by-step breakdown of what the function does:

#

# 1. Separate the input tuple (value) into three variables: d, m, and s, representing degrees, minutes, and seconds,

#     respectively.

# 2. Convert the degrees (d), minutes (m), and seconds (s) to floating-point numbers.

# 3. Calculate the decimal degrees using the formula: degrees = d + (m / 60.0) + (s / 3600.0). This formula converts

#     the minutes and seconds components  to their equivalent degrees and then adds them to the original degrees

#     value.

#    In case of a ZeroDivisionError, set the degrees to None.

# 4. Return the calculated decimal degrees.

#

# This function is used to convert GPS coordinates from their degrees, minutes, and seconds (DMS) format into

# decimal degrees, which are easier to work with when making API calls or performing calculations.

#

def _convert_to_degrees(value):

    d, m, s = value

 

    # Ensure that denominators of Fraction objects are not zero

    d = d.limit_denominator() if isinstance(d, Fraction) else d

    m = m.limit_denominator() if isinstance(m, Fraction) else m

    s = s.limit_denominator() if isinstance(s, Fraction) else s

 

    try:

        d = float(d)

        m = float(m)

        s = float(s)

    except ZeroDivisionError as e:

        print(f"Error: {e}")

        return None

 

    degrees = d + (m / 60.0) + (s / 3600.0)

 

    return degrees

 

 

 

# generate_table_headers(data) is a function that takes a dictionary containing image metadata (data) as its

# argument. The function is responsible for

# generating a sorted list of unique table headers, which represent the metadata fields found across all images

# in the dataset.

#

# Here's an overview of what the function does:

#

# 1. Initialize an empty set called headers.

# 2. Iterate through the metadata dictionaries in the data dictionary values using a for loop.

#    i) For each metadata dictionary, iterate through its keys.

#    ii) Add each key to the headers set. Since sets only store unique values, duplicate keys will not be added.

# 3. Convert the headers set to a sorted list and return it.

#

# This function is used to create a list of unique metadata fields (table headers) that will be used to build an HTML

# table displaying the metadata for each image in the dataset.

#

def generate_table_headers(data):

    headers = set()

 

    for metadata in data.values():

        for key in metadata:

            headers.add(key)

 

    return sorted(headers)

 

 

 

# create_html_table(source_dir, data, headers, api_key) is a function that takes four arguments: the source directory

#  path containing the images (source_dir), a dictionary containing image metadata (data), a list of table headers

#  representing metadata fields (headers), and a Google Maps API key (api_key).

# The function generates an HTML table displaying the metadata and location information for each image in the

#  dataset and saves it as a file named "image_metadata.html" in the source directory.

#

# Here's an overview of what the function does:

#

# 1. Define the initial HTML content, including the table, CSS styles, and JavaScript for opening an image in a new

#    window.

# 2. Add table header rows to the HTML content, with columns for the file name, location, and each metadata field

#    in the headers list.

# 3. Iterate through the images and their metadata in the data dictionary:

# 4. Obtain the human-readable location for the image using get_human_readable_location(metadata, api_key)

#     function.

# 5. Create a thumbnail of the image using the create_thumbnail(image_path) function.

# 6. Add a table row to the HTML content for each image, displaying its file name, thumbnail, location, and metadata

#    fields.

# 7. Close the table, body, and HTML tags in the HTML content.

# 8. Write the generated HTML content to a new file named "image_metadata.html" in the source directory.

#

# This function is responsible for creating a visually appealing HTML table that displays the metadata and location

#  information for each image in the dataset.

# Users can view the table in a web browser to easily access and understand the metadata associated with the

# images.

#

def create_html_table(source_dir, data, headers, api_key):

    html_content = '''

<html>

<head>

<style>

    table, th, td {

        border: 1px solid black;

        border-collapse: collapse;

    }

    th, td {

        padding: 15px;

    }

</style>

<script>

    function openImage(imgSrc) {

        var windowWidth = 800;

        var windowHeight = 600;

        var windowLeft = (screen.width / 2) - (windowWidth / 2);

        var windowTop = (screen.height / 2) - (windowHeight / 2);

        

        var newWindow = window.open("", "_blank", "width=" + windowWidth + ", height=" + windowHeight + ", left=" + windowLeft + ", top=" + windowTop + ", resizable=yes, scrollbars=yes");

        newWindow.document.write('<html><head><title>Image Viewer</title></head><body><img src="' + imgSrc + '" style="max-width:100%; max-height:100%; display:block; margin:auto;"></body></html>');

    }

</script>

</head>

<body>

'''

 

 

    html_content += "<table><tr><th>File</th><th>Location</th>"

 

    for header in headers:

        html_content += f"<th>{header}</th>"

 

    html_content += "</tr>"

 

    for image, metadata in data.items():

        location = get_human_readable_location(metadata, api_key,image)

        if location:

            location = html.escape(location)

        else:

            location = ""

        thumbnail = create_thumbnail(os.path.join(source_dir, image))

        original_image_url = os.path.join(source_dir, image).replace('\\', '/')

        html_content += f'''<tr>

            <td>

                {image}<br>

                <img src='data:image/jpeg;base64,{thumbnail}' onclick="openImage('{original_image_url}')" style="cursor: pointer;"/>

            </td>

            <td>{location}</td>'''

        for header in headers:

            value = metadata.get(header, '')

            escaped_value = html.escape(str(value))

            html_content += f"<td>{escaped_value}</td>"

        html_content += "</tr>"

 

    html_content += "</table></body></html>"

 

    with open(os.path.join(source_dir, "image_metadata.html"), "w") as f:

        f.write(html_content)

 

 

 

# create_thumbnail(image_path) is a function that takes an image file path (image_path) as its argument. The

# function creates a thumbnail of the input image with a maximum size of 200x200 pixels and returns it as a

# base64-encoded string.

#

# Here's an overview of what the function does:

#

# 1. Open the image file using the provided image_path with the Python Imaging Library (PIL) Image module.

# 2. Create a thumbnail of the opened image by calling the thumbnail() method with a tuple containing the maximum

#     width and height (200, 200).

#    This method automatically maintains the aspect ratio of the original image while resizing it to fit within the

#     specified dimensions.

# 3. Create a new BytesIO buffer to temporarily store the thumbnail image data.

# 4. Save the thumbnail image to the buffer in JPEG format using the save() method.

# 5. Encode the buffer's content as a base64 string using the base64.b64encode() function and decode it to

#     a UTF-8 string.

# 6. Return the base64-encoded thumbnail string.

#

# This function is used to create smaller, base64-encoded versions of images that can be easily embedded in an

# HTML table for quick previewing.

# By using thumbnails, the HTML table loads faster and consumes less memory, providing a better user experience

# when viewing the metadata and image previews.

#

def create_thumbnail(image_path):

    img = Image.open(image_path)

    img.thumbnail((200, 200))

    buffer = BytesIO()

    img.save(buffer, format="JPEG")

    thumbnail_base64 = base64.b64encode(buffer.getvalue()).decode('utf-8')

    return thumbnail_base64

 

 

# Following is the test code

#

def main():

 

    # NOTE : When you are in test stage, put a small number of image files in the directory because each image would

    #           trigger a API request and each API request generate Cost.

    source_dir = "C:\\test"   # Specify any directory where your image files are stored

    

    api_key = "YOUR_API_KEY"   # Copy and Paste your Google Map API Key

    

    metadata_dict = {}

    for file in os.listdir(source_dir):

        if file.lower().endswith(('.png', '.jpg', '.jpeg', '.gif', '.bmp')):

            image_path = os.path.join(source_dir, file)

            metadata_dict[file] = get_exif_data(image_path)

    

    headers = generate_table_headers(metadata_dict)

    create_html_table(source_dir, metadata_dict, headers, api_key)

 

if __name__ == "__main__":

    main()

 

 

Result