In an attempt to migrate my iPhoto library to Lightroom, I found this topic is interesting yet messy. All image metadata, file format, schema are well maintain and published as some kind of standard, but there seems no easier way to make sense these topics easily. Consider the target is just to ensure I can migrate and manage the metadata in a rational and sensible way, it does not make sense to go through all the information with days of effort.
Hence, I try to summarize this information in a concise manner (hope this can be digested within 1 hour for people with technical background). If you are technical person but just new to image metadata, and you want to have a sound foundational knowledge, you should benefit from this article (at least this is my goal).
This article will first go through various kind of Metadata, following technical details on how this metadata embedded in the image file. Finally, the document will illustrate how to use exiftool tools to better understand the data.
Types of Metadata
EXIF is probably the earliest types of data. It primarily focus is technical information (here technical in the sense of photography). Sample information include capturing time, capture location (GPS), exposure setting (aperture,ISO,shutter), camera/lens maker and model. The standard defined the tag schema (what is the meaning for each tag) as well as on binary level how EXIF meta data can be represented.IPTC is a metadata schema originated by news/media agency. The focus on IPTC is support annotation and organization (for example Keywords). There are two generation of IPTC, IPTC IIM (Information Interchange Model) and IPTC4XMP. Usually when people refers IPTC tags, it refers to IPTC IIM. (We will come back to IPTC4XMP later on when we discuss XMP). IPTC IIM also define the both the logical schema and binary representation.
XMP define a logical schema for representing media metadata. Very general in nature. The serialization format is XML/RDS. It defines how the standard can be extended to cover different vocabulary (via namespace). The standard consists of 3 parts, the first part different some serialization details (for example how to represent scalar values and tuples/lists. Part 3 define some details on how to represent the XMP data in binary files. This also define how XMP can exists as a sidecar file.
IPTC4XMP bring the vocabulary of IPTC under the XMP standard. It consists of IPTC-Core and IPTC-Extension. In additions to IPTC/XMP, XMP also include dulin core (XMP-DC namespace), Dublin Core are general XML schema for metadata. It originated for librarian purpose.
There are some overlap between IPTC and Dublin Core (given both try to solve the problem for information organization). A few field are defined as inter-changeable between two schema. The most important one probably is: IPTC:Keywords vs DC:Subject).
Image File Format for Embedding Metadata
JPEG file consists of 'segment', each segment starts with 2 bytes to denote the type of segment. For example, 0xFF0xD8 refers to Start of Image, 0xFF0xE1 refers to APP1 segment, 0xFF0xEC refers to APP13 segment. The 'App' segment is generic mechanism for application extension. Usually App segment also follow with additional application tag. The 0xFF0xE1 App1 segment is followed with EXIF to denote the segment is an EXIF block. The block content will then be encoded according to EXIF.Similarly, 0xFF0xEC refers to App3 which is used by IPTC IIM, while 0xFF0xE1 with a segment header "http://ns.adobe.com/xap/1.0/\x00" denote a 'XMP' segment.
A typical JPEG pictures will both embedded with EXIF segment, IPTC-IIM segment and XMP segment. While the XMP segment will include IPTC core elements, Dublin Core elements as well as application specific extension like XMP:LR namespace will contains Lightroom specific extension under XMP format. XMP is serialized as XMP hence you can see an XML text embedded in the JPEG file if you open it with a hex or text editor.
Different metadata schema have some overlap, application usually will synchronize the meta when saving the information. In Lightroom, when an external tools update the Subject tag (part of XMP:DR), LR can be instructed to read this, when LR save the metadata, it will populate the value into both IPTC-IIM:Keyword and XMP-DC:Subject, as well as XMP-LR:HierarchicalSubject.
TIFF have similar mechanism, and most all RAW format is derived from TIFF (include CR2, ARW, and DNG). While RAW files have extension mechanism for embedding metadata, however, Lightroom allow metadata to be written as a sidecar file (all metadata, include post processing settings, are serialized in XMP format, as a separate file seat aside the original file).
Playing with Metadata
EXIFTool is a friendly CLI tools to manipulate image metadata. You can use it inspect file in details or update the metadata (hence you can write simple script and batch update a particular tag)To read the tags, simple dump
exiftool myPhoto.jpg
Each tag is prefixed with the group (ie: whether it is EXIF, IPTC, XMP, etc)
exiftool -G myPhoto.jpg
To print all meta data in XMP
exiftool -X myPhoto.jpg
To update a particular tag
exiftool -Subject='OuterSpace' Photos/IMG_4882.jpg
To make Lightroom recognize meta data is updated, need to update IPTC Digest tag
exiftool -Subject='OuterSpace' -IPTCDigest=new Photos/IMG_4882.jpg
Further References
In additions to reference provided within the text above, some other sites provide very indepth information:
