/dev/schnouki

Tag: howto

Optimizing JPEG pictures

I recently realized that during our vacation in London, my girlfriend and me took about 4 GB of pictures. Since I currently have 30 GB of storage space on [rsync.net][] to do my backups, 4 GB is quite a lot. Fortunately, there are several solutions to reduce their size.

The first one would be to resize them or to increase their compression ratio / decrease their quality. But I don't want such a lossy method: I want to keep my pictures at the best quality available so I can print them in high resolution if I want to.

The other solution is to "optimize" them. Once again, several methods: removal of unnecessary data (EXIF markers and other metadata), conversion to progressive JPEG, or Huffman table optimization. Since I don't want to lose metadata (mostly because I add many tags to my pictures in Shotwell and they are stored in these metadata), I only use the other two methods.

Most of my photos are taken with my camera (Panasonic Lumix FZ100) or with my girlfriend's (Nikon Coolpix S8000).

I first tried to use jpegoptim to do this task. It only optimizes Huffman tables, and it does it well. However, this tool only supports EXIF and IPTC metadata, and on pictures taken with my camera, Shotwell stores its tags in the XMP "Subject" marker. And jpegoptim erases XMP markers when processing them, resulting in many lost tags...

So I tried to use jpegtran to do the same. It also supports progressive JPEG, and is apparently much better at not destroying metadata when not asked to do so :) Here is the command I use to optimize my pictures with it:

parallel -u 'echo {}; jpegtran -optimize -progressive -perfect -copy all -outfile {}.tran {} && mv {}.tran {}' ::: *.JPG

parallel is GNU Parallel, a tool which is very useful to speed things up by using the 16 cores of my work PC to do the job :)

Using jpegtran this way, I reduced the size of my "London" folder from 4.0 GB to 3.5 GB, i.e. a 12.5% reduction with absolutely no quality loss. Not bad!

Now, some funny things I noticed while doing this:

  • the Lumix FZ100 does not do any optimization to its JPEG files: jpegtran always reduced them by at least 13%, sometimes more. It also create some EXIF and XMP markers in its files, but no IPTC tag.

  • the Coolpix S8000 does a much better job at optimizing its files: jpegtran could only reduce their size by 0.6 to 0.8%, 1.2% at best. It creates EXIF, XMP and IPTC markers.

  • when Shotwell stores tags directly into pictures, it will use the IPTC "Keywords" marker only if there already are IPTC data in the file. This is why jpegoptim lost tags on pictures taken with my camera: the FZ100 only added XMP markers, which were then wiped out by jpegoptim. For pictures taken with the S8000, tags were stored both in XMP and IPTC markers, so when the XMP ones were removed, Shotwell still took the IPTC version into account.

    Not sure if it's a bug or a feature...

HOWTO Backup your GnuPG secret key on paper

Paper is a safe way to backup a secret key: you can't hack into it remotely, you can hide it very easily, and you will still be able to use it in 50+ years. No USB stick can do that...

If you want to store your GnuPG secret key on a paper sheet, it is quite simple to do. You can use PaperKey, a small tool that strips all the useless data from a secret key and formats it into a printable result. This is great, but the result can be quite long: printing my 2048 bits secret key would take 3 pages.

But there is a nice way to store more data on a small surface: 2D barcodes, for example in the DataMatrix format, using the great libdmtx library. For small keys, this is really easy:

gpg --export-secret-key KEY_ID | paperkey --output-type raw | dmtxwrite -e 8 -f PDF > secret-key.pdf

If your key is bigger (like my 2048 bits key), you will need to split it in several parts, because the result of the paperkey command will be too big to be encoded in a single DataMatrix. Here is a simple method:

# Generates key-aa, key-ab, ...
gpg --export-secret-key KEY_ID | paperkey --output-type raw | split -b 1500 - key-

# Convert each of them to a PNG image
for K in key-*; do
    dmtxwrite -e 8 $K > $K.png
done

You now have several PNG images that you can print together on a single page.


To restore your key, it's just as simple: scan each DataMatrix into a separate image, decode them with dmtxread, concatenate all the resulting files (cat...), and use paperkey:

cat my-scanned-keys | paperkey --pubring ~/.gnupg/pubring.gpg > secret-key.gpg

Source: TPK Archival (by David Shaw, creator of PaperKey).