"Junk" data in JPEG data stream

[AndyWolan]

I have the a6000 and have been using it for almost a year now, and I love it. 

 

This weekend, I have been going through old photos, and purging those images that I don't need.  While toying around with lossless editing tools, I noticed that many of my images have an additional 500KB to 1MB of "junk" attached to the end of the JPEG image's data stream. I say "junk" because I have no idea what this data is. I imagine it is "metadata", but that much? Using some lossless editing tools, I have been able to safely strip this data off. 

 

My question to you is this: what is this "junk" data, and how do I prevent it from being written in the first place? This junk alone is wasting 10-20% of my disk space!

 

For the record, I pull photos off of my camera by popping the SD card into my PC and pulling them off of it. I don't connect the camera to the PC.

 

Refer to my attached image. There, I use "WinMerge" to compare the cleaned and original files.

Welcome, dear visitor! As registered member you'd see an image here…

Simply register for free here – We are always happy to welcome new members!

[Guest all8]

Play around with ExifTool.

[AndyWolan]

Yes, but what is the camera adding to the JPEG data stream? And is there a setting to prevent it from adding this "junk" in the first place?

[Guest all8]

Play with ExifTool and you might figure it out.

 

No.

[AndyWolan]

Under "Maker" I see something called "PreviewImage: (Binary data 714056 bytes, use -b option to extract)". The size specified for "preview image"  is roughly the size I recovered when I strip data "junk" data from file. In the case of this file, the stripped/cleaned file is 720,523 bytes smaller.

 

So, the culprit is a "preview" image being written to the JPEG file itself. Does anyone know if there is an option to prevent the preview image from being written? I understand that adding a small preview image can be helpful to image editing applications, but adding 700KB to a 3.4MB image is A LOT! (20% to be exact.)

[AndyWolan]

Ok, I tried playing with EXifTool and I figured out how to delete "preview image", but now my "last modified" time stamp on the file system is messed-up.  Exiftool supports an option to set filesystem modified time to the creation time of the photo as recorded in the metadata. But when I run that command after removing the large preview image, exiftool give me an error of "cannot read preview image", and thus can't adjust the date. I get the same error from EXIFTool if I strip the preview image using JPEGCrop. 

 

ARG! Why does Sony include a freaking 700KB preview image with each photo I take? It's such a waste of disk space! It's bad enough trying to remove said data is painful.

[Guest all8]

Should give you some ideas:

 

http://superuser.com/questions/292630/how-can-i-change-the-timestamp-on-a-file

 

 

Given that a GB of disk storage, enough for 1500 preview images, is costing around 30 cents ... for most people the benefits probably exceed the cost. However, you can resolve the issue yourself with these tools, and possibly do many other things.

[AndyWolan]

I'd accept the 10-20% hit in file size if there was a reason. But what is the purpose of including such a large "preview" image in the JPEG stream itself? Why not read the regular data stream if you want to render a high quality preview?

 

BTW, my old Canon camera does not do this. Is this some kind of new trend?

[Guest all8]

For 30 cents worth of storage I just don't care. Sony probably thinks along the same lines, so call it a trend if you want.

 

I think the real problem is that you are using "lossless editing tools" and "JPEG" in the same sentence       Try using RAW files perhaps?

[AndyWolan]

People forget the cost of archiving isn't necessarily the cost of buying a HDD, but rather the cost to manage and maintain the data and their backups. The less data you have to manage, easier it is to manage your data. Yes, disk space is cheap, but manpower is not. Also, online storage is not as cheap either. (Cost to upload large file via ISP ... some land-line ISPs like Comcast charge by the GB, cost of web storage, cost to recovery data from an archive, etc.) So no, the cost of extra storage is not "30 cents", over time it's far more than that.

 

BTW, there exists lossless editing tools for JPEG images that perform simple editing tasks, such as: grid-based cropping, right-angle rotations and even mirroring. All done losslessly. (IE, without re-compressing the image.) For more sophisticated editing, yes, "lossless" is not going to work. (Red-eye removal, non-right angle rotations, pixel-perfect cropping, etc.)

 

Finally, why should I capture and archive 24-28 MB RAW images when a 4-8MB JPEG image will do fine? With the exception to the occasional red-eye photo, I rarely need to perform any non-lossless post-production work to my images. And the a6000 does a great job at making JPEGs (minus the stupid "preview image"), so why should I bother with post-processing of RAW images? I don't have the time.

[AndyWolan]

Anyhow, back to my ORIGINAL question:

 

is there an option to prevent the a6000 from embedding a "preview image" into the JPEG files it writes? I'm not interested in "going RAW" or using data manipulation tools like "EXIFtool", which creates more problems than they help resolve. I just don't want the camera to create said embedded image. That is all.

[VTC]

I stopped using floppies and worrying about 500k worth of data years ago. If I were obsessed I'd sooner start pitching obvious trash images to reclaim space rather than spending hours on some voodoo 'striptease' act that well may corrupt the image and prevent it from even being opened at some future date.

[AndyWolan]

Uh, ya, I stopped using floppies as backup media in late 1990's. However,  500KB * 1,000 images = 500MB. That's nothing to sneeze at. And yes, I DO go through my photos and purge bad/unwanted photos. 

 

My question is not about how to perform a "voodoo striptease" to purge the "preview image" from a JPEG.  My question is this: is there a setting on the a6000 to prevent it from embedding a "preview image" in the first place?

[illogict]

This preview image is the one that is shown in the camera file manager. It is much faster to read and decode a small preview than the full image.

 

My previous Canon DSLR also did that. I’m pretty sure most, if not all cameras also embed a preview.