HPR4697: Correcting the Dates of Files

This show has been flagged as Clean by the host.

I recently had an experience where UNIX tools proved very useful. A relative had an old mobile phone running Android that stopped connecting to the carrier's network and bought a new one to replace it. I took on the job of trying to copy their files (consisting of just photos and videos) off of the old phone.

Google's software was desperate to convince me to upload everything to the cloud, but I wasn't interested. It offered the option of copying the files over to an SD card, but failed on repeated attempts to do that. The option I tried next was to transfer them to another device via Bluetooth—that one did actually work, although it was slow and would only handle sending about 100 files at a time.

They came over to my laptop OK, but the problem with that method was that all of the file times were set to the time when they were transferred. I'm not super familiar with how mobile apps manage metadata, but would presume that they look to file times for organizing photos by date. Fortunately, the names of each of the files included the date and time they were created. I recognized that I could write a bit of shell script to parse the filenames and set the file times accordingly.

While there were over 800 files, the good news is that there were only three different categories of filenames, so the logic to extract the information needed was relatively simple. Each file had eight numerical digits representing the date and six digits representing the time. It would definitely be an option to come up with a more sophisticated parser that could handle a wide variety of filenames, but I went the lazy way and just handled those three cases. Another nice aspect was that none of the filenames contained spaces, which allowed me to be a bit less careful when using them in command lines. I didn't need to worry about time zones because my laptop was set to the same time zone as the phone—also, if a time was off a by a few hours it wouldn't make a practical difference.

Examples of the three different types of filenames I had to deal with, labeled with the relevant values: YYYY=year, MM=month, DD=day, hh=hour, mm=minute, and SS=second.

00001IMG_00001_BURST20250525140124.jpg
 YYYYMMDDhhmmSS

IMG_20220223_124023.jpg
VID_20221017_095024.mp4
 YYYYMMDD hhmmSS

20191224_195939.jpg
20161021_122620-1.jpg
20191130_134317_Burst01.jpg
20200129_223612_010.jpg
YYYYMMDD hhmmSS

I considered awk as an option (see Whiskeyjack's comment on HPR episode 4657 ), but realized it has no built-in way to change file times, so I set it aside. Don't worry, I will come back to that later.

My approach was to use an if-then shell construct to choose how to treat the three categories of filenames. For the if condition, I fed the filename into the grep -q command with an appropriate regular expression to test whether it matches. The -q option to grep causes it not to output anything—it returns a zero exit status if there's a match and a status greater than zero if there isn't. Then, there is an elif statement with another grep -q test for the second category of filenames. Finally, an else statement is followed by the command to run for all other filenames. The whole thing is wrapped in a for loop that runs over all the files in the current directory.

The touch command , when used with the -t option, can be given a string consisting of the year, month, day, hour, minute, and second. These are all numerals that are run together, except that a period sits between the minute and second. So we need a way to extract these numbers and to insert the period.

That's where the cut utility comes in. It can be given a set of characters to select, and I specified a different set representing the appropriate ones depending on which category a filename fit into. To insert the period, I used sed to replace the last two characters with a period followed by those characters.

The first script was to test out that I was getting the correct results.

for fn in *
do
 if echo "$fn" | grep -q BURST
 then
 printf "$fn "
 echo $fn | cut -c '21-34' | sed 's/..$/.&/'
 elif echo "$fn" | grep -q -E '^(IMG_|VID_)'
 then
 printf "$fn "
 echo $fn | cut -c '5-12,14-19' | sed 's/..$/.&/'
 else
 printf "$fn "
 echo $fn | cut -c '1-8,10-15' | sed 's/..$/.&/'
 fi
done

This one actually sets the file times. The -c option to touch prevents it from creating a file if one with that name doesn't already exist.

for fn in *
do
 if echo "$fn" | grep -q BURST
 then
 touch -c -t "$(echo $fn | cut -c '21-34' | sed 's/..$/.&/')" "$fn"
 elif echo "$fn" | grep -q -E '^(IMG_|VID_)'
 then
 touch -c -t "$(echo $fn | cut -c '5-12,14-19' | sed 's/..$/.&/')" "$fn"
 else
 touch -c -t "$(echo $fn | cut -c '1-8,10-15' | sed 's/..$/.&/')" "$fn"
 fi
done

The script ran over all the files in less than 15 seconds and correctly set the file time on each. Job done, right? Well, after I did this, it struck me that there was room for improvement. The script would probably run more quickly if I used a case construct instead of an if construct that called grep multiple times. While the pattern-matching notation used with case is not as flexible and can handle fewer situations than the regular expression syntax available with grep , in this case (see what I did there?) it is sufficient. Testing it out, using case reduced the running time by 45%.

Replacing if with case —the commands to be executed for each category of filename can remain exactly the same.

for fn in *
do
 case "$fn" in
 *BURST*)
 printf "$fn "
 echo $fn | cut -c '21-34' | sed 's/..$/.&/'
 ;;
 IMG_*|VID_*)
 printf "$fn "
 echo $fn | cut -c '5-12,14-19' | sed 's/..$/.&/'
 ;;
 *)
 printf "$fn "
 echo $fn | cut -c '1-8,10-15' | sed 's/..$/.&/'
 esac
done

for fn in *
do
 case "$fn" in
 *BURST*)
 touch -c -t "$(echo $fn | cut -c '21-34' | sed 's/..$/.&/')" "$fn"
 ;;
 IMG_*|VID_*)
 touch -c -t "$(echo $fn | cut -c '5-12,14-19' | sed 's/..$/.&/')" "$fn"
 ;;
 *)
 touch -c -t "$(echo $fn | cut -c '1-8,10-15' | sed 's/..$/.&/')" "$fn"
 esac
done

I couldn't completely put awk out of my mind, though, and I eventually came up with an awk script for the same purpose. This is far faster, probably because everything can be done within awk except actually modifying the file times, which is possible using the system() function to call touch . I was able to knock 90% off the running time, which for 800 files isn't a big deal but might make a difference if you have hundreds of thousands of files.

The awk counterparts to both scripts above. Unlike those, ls is used to feed it with the list of filenames. We have the full power of extended regular expressions available to use for matching against the filenames. The next statement causes awk to skip any remaining pattern-action pairs and go to the next line of input.

ls | awk '/BURST/ { print $0, substr($0, 21, 12) "." substr($0, 33, 2)
 next }
/^(IMG_|VID_)/ {
 print $0, substr($0, 5, 8) substr($0, 14, 4) "." substr($0, 18, 2)
 next }
{ print $0, substr($0, 1, 8) substr($0, 10, 4) "." substr($0, 14, 2) }'

ls | awk '/BURST/ {
 system("touch -c -t " substr($0, 21, 12) "." substr($0, 33, 2) " " $0)
 next }
/^(IMG_|VID_)/ {
 system("touch -c -t " substr($0, 5, 8) substr($0, 14, 4) "." \
 substr($0, 18, 2) " " $0)
 next }
{ system("touch -c -t " substr($0, 1, 8) substr($0, 10, 4) "." \
 substr($0, 14, 2) " " $0) }'

A further optimization that came to me later was to not call system() from within awk , but to instead just have awk print out a set of command lines. These can then be piped to sh to actually be executed. This cut the running time down by 95% compared to my original script.

The fastest version I was able to come up with. If you run it without the | sh on the end, you can check that it's outputting the right information before actually modifying anything. The backslash on the end of a couple lines causes the subsequent line to be treated as a continuation of the existing line. Normally I would just keep everything on one line even if it runs longer than 80 columns, but for display purposes this looks nicer.

ls | awk '/BURST/ {
 print "touch -c -t " substr($0, 21, 12) "." substr($0, 33, 2) " " $0
 next }
/^(IMG_|VID_)/ {
 print "touch -c -t " substr($0, 5, 8) substr($0, 14, 4) "." \
 substr($0, 18, 2) " " $0
 next }
{ print "touch -c -t " substr($0, 1, 8) substr($0, 10, 4) "." \
 substr($0, 14, 2) " " $0 }' | sh

It is probably true that this could have been carried out just as easily on Windows using Microsoft's PowerShell. I'm not very familiar with it, but would imagine (or hope) that it includes commands for managing these basic things like text manipulation and modifying file times. If you are stuck in an environment where you don't have a UNIX-like system available, investigate how to accomplish a task with the tools you do have.

While I had the necessary information in the filenames to use, that might not be the case in all situations. You could look for other sources of dates—most digital cameras will add EXIF tags to a JPEG file giving the date and time it was created. (Hopefully, the clock in the camera will be set accurately.) While there is no standard UNIX utility to read those tags, free and open source software tools are widely available for that purpose. I found one called exiftags that included the utility exiftime , which specifically outputs EXIF data relating to time. The output format was a little trickier to handle, but awk was able to manage it with a little coaxing.

Example of output produced by exiftime . Note that the first line with the filename is only printed if more than one filename is given as an argument. Also, for amusing-sign.jpg , apparently I edited that photo after taking it and the editing software updated the "created" tag but left the others intact. Not all images will necessarily have created, generated, and digitized tags; we will just take whichever ones exist. I redirected standard error to /dev/null to get rid of error messages for files that don't have EXIF tags; we'll handle those below.

$ exiftime *.jpg 2>/dev/null
20260508_154743.jpg:
Image Created: 2026:05:08 15:47:43
Image Generated: 2026:05:08 15:47:43
Image Digitized: 2026:05:08 15:47:43

20260508_155044.jpg:
Image Created: 2026:05:08 15:50:44
Image Generated: 2026:05:08 15:50:44
Image Digitized: 2026:05:08 15:50:44

3704a78e771c2a25a894ef2f0b5a2a629f1eba80.jpg:

amusing-sign.jpg:
Image Created: 2017:01:24 23:14:04
Image Generated: 2017:01:24 21:18:07
Image Digitized: 2017:01:24 21:18:07

dscf3011.jpg:
Image Created: 2015:01:01 00:02:19
Image Generated: 2015:01:01 00:02:19
Image Digitized: 2015:01:01 00:02:19

window-view.jpg:
$ 

We can take advantage of the fact that different records are separated by a blank line. In awk , when RS is set to a null string and FS is set to a newline character, each set of non-blank lines is treated as a record and each line within those sets is treated as a field. One or more blank lines separate each record. For the output of exiftime , this means that $1 will contain the filename and $2 will contain the first line after the filename. For those files without an EXIF date tag, $2 will be a null string, which is treated by awk as FALSE, so the pattern will not match, the action will not be taken, and nothing will be printed. If a file has multiple tags, I will just use the first one reported by exiftime (contained in $2 ). The sub() function call removes the colon that exiftime prints after the filename, and the gsub() function call removes all non-numeric characters from the date and time in the tag. (After a comma within a print statement, a backslash is not necessary to continue a line.) Also, this time I bothered to print quotation marks around the filename in case it contains spaces.

$ exiftime *.jpg 2>/dev/null | awk 'BEGIN { FS = "\n" ; RS = "" }
$2 { sub(":$", "", $1)
 gsub("[^0-9]", "", $2)
 print "touch -c -t", substr($2, 1, 12) "." substr($2, 13, 2),
 "\"" $1 "\"" }'
touch -c -t 202605081547.43 "20260508_154743.jpg"
touch -c -t 202605081550.44 "20260508_155044.jpg"
touch -c -t 201701242314.04 "amusing-sign.jpg"
touch -c -t 201501010002.19 "dscf3011.jpg"
$ 

I would imagine that there's some photo management program out there that I could have used to accomplish this. But then I would have had to locate it, verify that it wasn't some malware-loaded garbage, download, and install it. And chances are it would want to take over all the photos on my laptop. Instead, with standard UNIX tools and shell capabilities like if , case , process substitution, and pipelines, I was able to complete the task without having to install anything.

The techniques I described can be used in different circumstances and with the output of different utilities. My intention was not just to explain how to solve this specific problem, but to hopefully teach you some things that you can apply in many situations. Perhaps if you use them to tackle a challenge of your own, you'll record an episode for HPR to share what you know.

Provide feedback on this episode.