Micro 4/3 I made a software to find settings that match the JPEG output between cameras

[ibd]

And I used it to replicate some of Fujifilm's film simulations on the Olympus Pen-F.

The Pen-F is quite unique (along with its younger sister, the E-P7) in that it allows a lot of control over JPEG output in-camera. It's the only camera that I know of, besides smartphones and phone-camera hybrids, that allows setting the saturation of individual hues.
This is one of the features that make the Pen-F so popular, I believe. Many people have used this functionality to great effect.

Among the more popular profiles is the "Kodachrome 25" profile, originally created by user GarethB over on the mu-43.com forums: My "kodachrome/classic chrome" Pen-F settings (update)
Rob Trek has even made a tutorial video that explains the process in more detail:

These settings can be replicated in Olympus / OM System's "Workspace" raw converter software. Here are all the relevant settings at a glance:

Now, there are a couple of websites that test a great variety of cameras. They usually provide raw files for their test scene. What is more, they include color calibration targets such as the "ColorChecker":

This example is from DP Review. Imaging-Resource is another source of data.
The regular structure of the colored swatches makes it possible to automatically detect the ColorChecker in the camera frame and extract statistical color data from it.

This and the original Kodachrome 25 thread inspired me to write a collection of Python scripts that make it possible to match the JPEG color response between cameras as closely as possible.
As the Pen-F offers many tunable controls, it's a great candidate for this application. For cameras that do not offer so many options in-camera, there is less motivation to follow this approach -- it's more rewarding to simply shoot raw, then apply the desired color profile with full flexibility on the PC.

The software consists of three parts:

1. Iterative global optimization. This first script analyzes the target color, the source color, and adjustment changes in the JPEG output to find an optimal transformation in 3D L*a*b* space as a linear combination of adjustments. Since the effects of color adjustments in the JPEG output are highly non-linear, this step is repeated 5-10 times until a rough convergence is reached.
2. Hill climbing. This is a local optimization that has its starting point in the output from the first script. It offers an easy way to quickly test small adjustments, allowing you to fine-tune the profile to reach an optimal solution.
3. Evaluation and plotting. This script visualizes the alignment of the ColorChecker squares in the L*a*b* color space. This is useful for analysis.

Of course, the Pen-F itself does not offer full freedom in its adjustments. It's only possible to change the saturation of specific hues. It's not possible to change the lightness of different hues, or to shift hues around the color wheel altogether (only the "workspace" raw converter offers that).
As such, a perfect alignment for color profiles that do more complex color space distortions is not possible. One prime example is Fujifilm's "Classic Negative" profile, which has extreme hue shifts.
Still, the scripts allow you to find a solution that's as close as possible (under some mathematical definition related to the L*a*b* color space) to the target color profile.

Here's an example of a color profile matching for Fujifilm's "Astia" profile. The two images show the alignment before making any adjustments (using a stock Olympus profile), and after making adjustments with help of the scripts described above:

The left side shows a "top down" view where red-green is on one axis, and yellow-blue is on the other axis, with luminosity not visible.
The right side shows chroma on the x-axis, and luminosity on the y-axis.
Corresponding color swatches from the calibration target are connected with a thin black line.

As you can see, the adjusted color profile is much closer to what "Astia" looks like on average.

In a laborious effort, I did the alignment procedure for all film simulations of the Fujifilm X100V camera.
Thanks to my Dad who was able to provide me with JPEGs for the different film simulations -- Fuji's raw converter won't let you use them if you don't own the camera!
All the source data I've used can be downloaded here:

• DPR data: fujifilm_x100v_fs_dpreview - Google Drive
• IR data: fujifilm_x100v_fs_imaging_resource - Google Drive

Here are all the profiles in a table:

This table is available as a PDF here: Film simulations for Olympus Pen-F.pdf

The source code of my software is hosted on GitHub: GitHub - fishcu/color_chart
This is currently only a loose collection of Python scripts. If there is interest, I can write some tutorials on how to use the software, or even make it easier to use by programming a GUI for it.

I would be very happy to receive feedback on some of these profiles (and the software itself for users who are comfortable with Python)!

 

[ibd]

Here are some photo examples. I shot these photos some years ago, right in the beginning of 2020.

These were shot with the E-M1 II and then the profiles were applied in the Olympus Workspace software -- but with the Pen-F, you can store and apply these profiles directly in the camera! 🙂

Acros + red filter:

Join to see EXIF info for this image (if available)

Astia:

Join to see EXIF info for this image (if available)

Classic Chrome:

Join to see EXIF info for this image (if available)

Join to see EXIF info for this image (if available)

Classic Negative:

Join to see EXIF info for this image (if available)

Join to see EXIF info for this image (if available)

Pro Neg Hi:

Join to see EXIF info for this image (if available)

Pro Neg Std:

Join to see EXIF info for this image (if available)

Velvia:

Join to see EXIF info for this image (if available)

 

[comment23]

This is a really interesting and clever idea. I’d like to try these in a few files when I find some time and hope others post some examples here too.

 

[ibd]

I got a question about applying different recipes to the same photo, so one can judge the changes that are applied more clearly.

Here are all the profiles applied to three different photos I took in Istanbul, along with the Olympus stock profile export.

The photos are in the order that they appear in the Fujifilm film selection menu, that is:

1. (Olympus stock)
2. Provia
3. Velvia
4. Astia
5. Classic Chrome
6. Pro Neg Hi
7. Pro Neg Std
8. Classic Neg
9. Eterna
10. Acros
11. Monochrome

 

[ibd]

Second example:

 

[ibd]

Third example:

 

[Aviator]

Thank you for taking the time developing these settings, I am sure many will appreciate them.

I have a Pen-F and even if I always shoot raw, I'm certainly going to try some and post some pictures!

 

[sasquatchphotog]

Fascinating work, had no idea that the Pen-F has such granular colour controls.

 

[Bogansson]

Great work, thank you very much! Other such profiles are often based purely on feeling, perception, your method is something different, data-based, replicable (although I dont understand the code behind). I would like to ask you, though - if these profiles are based on PEN F, you surely end up somewhere a little bit else on any other Olympus camera (different sensor)? What do you see regarding for example E-M1 II? What about creating profiles of highly regarded PEN F mono/color profiles for other Olympus cameras?

 

[Bogansson]

Oh, I forgot to specify - use of profiles for different cameras is meant via OM Workspace (I guess only PEN F allows them in-camera).

 

[ibd]

Great work, thank you very much! Other such profiles are often based purely on feeling, perception, your method is something different, data-based, replicable (although I dont understand the code behind). I would like to ask you, though - if these profiles are based on PEN F, you surely end up somewhere a little bit else on any other Olympus camera (different sensor)? What do you see regarding for example E-M1 II? What about creating profiles of highly regarded PEN F mono/color profiles for other Olympus cameras?

Hi Bogansson, thanks for your interest!

The whole process is based around the assumption that OM Workspace gives me identical results to processing the file in-camera. I did a very brief test earlier and this seems to hold true at least for the Pen-F camera. See the two attached photos, one is the output from the camera, the other is a re-processing of the raw file in OM Workspace. They're identical except for some tiny, tiny differences visible in the noise structure if you zoom in a lot.

The question if profiles can be transferred from one camera to another is a great one, which I didn't explore yet in detail. In my test photos above, I had assumed that the E-M1 II behaves roughly similar to the Pen-F so I simply applied the same profiles on it.
To figure out some numbers how close they really are, I did a brief check with the Pen-F profile for the "Velvia" simulation.
We can evaluate the discrepancy on the DPR test scene for several cameras.

Here's the color patch view for the Pen-F itself. The discrepancy score is 339. A lower discrepancy score means a better resemblance to the reference that is trying to be matched.

Here's the E-M1 mark II. The discrepancy is 808.

Finally, here's the OM-5 with a score of 782.

Even though we can assume that the OM-5 and the E-M1 mark II share the same sensor, the score will always be a bit different between the two simply because they are different exposures.
Given that the scores are less than 30 units apart, though, indicates that they behave very similarly. Given that the score measures discrepancy in L*a*b* color space, we can say that a difference of 30 in the error is basically imperceptible to humans.
The difference between the Pen-F and the other two cameras is visible in theory. But it's hard to say from a single shot. 🙂

From the plots and from the scores themselves we can see that using the profiles developed for the Pen-F give the best score when used on the Pen-F raw file, as expected.
For the color chart on DPR which contains 24 colors, I found the following rough guideline to interpret the scores:
1000 or less: Rough fit.
500 or less: Good fit.
300 or less: Excellent fit.
This is arbitrary and just derived empirically.
If we apply that, the Velvia settings are a good fit on the Pen-F itself (for which it was optimized), and a rough fit for the E-M1 II and the OM-5.
This means that we would get slightly better results by refining the profiles for each specific camera (or sensor).

Regarding Black and White profiles:
Yes, it would be possible to "transfer" the B&W from the Pen-F to other cameras as most of them also have these settings.
Note that the "score guideline" above doesn't apply the same way for B&W shots, as you're basically only measuring the error in one dimension instead of three (luminosity only = matching the tonality).

Transferring color profiles to other cameras is possible, but would be restricted to the use within OM Workspace only as other (Olympus) cameras don't have the saturation settings. For me, if the color profile cannot be applied in-camera, there is less motivation as great "film presets" already exist for example in Lightroom, and my work here would probably not be as accurate as those film presets, for example the ones that VSCO used to make.

 

[ibd]

I have created two color profiles, a neutral and a warm one, to simulate the old Olympus E-1 camera:


Some more information and sample photos in this and the following posts on mu-43.com:

My "kodachrome/classic chrome" Pen-F settings (update)

First of all many thanks for your effort to make my wish come true .:bowdown: I will try the profile and let you know. Meantime if you finish the other profile (automatic WB with the warm look) I will try to borrow my friend's E-1 and set a real comparison with both profiles in the weekend. I...

www.mu-43.com

 

[Tili]

Oh my! That's amazing! Would it be possible to add the newer film sims as well? Like Nostalgic neg, Bleach bypass or Reala ACE?

 

[ibd]

Oh my! That's amazing! Would it be possible to add the newer film sims as well? Like Nostalgic neg, Bleach bypass or Reala ACE?

Yes. I would need the following files:

• JPEGs of the film simulations to be matched showing a color calibration target.
• Raw file of the same scene shot with the Pen-F

I have recently updated the software so that it can work with IT-8 color targets. Actually, the Olympus E-1 profiles were created in that way.
I found that the resulting profile calibration is more stable and probably more accurate, too.

I am now exploring ways to better match the tonality of the target camera. For that reason, I am experimenting with simultaneously matching the base exposure and also +/-1 EV results.
In summary, I would need shots of a color target that were developed with nostalgic neg, bleach bypass, and reala ace.
You can display this image on your monitor and take a photo of it: https://www.dpreview.com/files/p/articles/2484506686/Samples/Colourspace/8G350268-standard.jpeg
Do a +/-1 EV exposure bracketing and save the JPEG files. Then, take the same shots with the Pen-F and save the raw files.

I have asked the author of Fuji-X-Weekly, Ritchie, for help with these files two times already. Unfortunately, he wasn't responsive.

 

[Balinus]

Nice work!

How is that supposed to be working? I mean at the command level. I see python script, but I'm unsure how to proceed. Cheers!

 

[ximi]

This is great. It's fantastic that people still invest time into this camera many years after its release.
A Portra 400 simulation would be amazing too, but not sure how feasible it is (one issue with film is that the scanning process will affect the final colours as well)

 

[mtsuisse]

Hello fellow photogs 🖖😎
New Pen-F owner here and pretty much interested in this topic.

Thank you @ibd for your effort and this nice idea. Have tried some of your first film sims and like the results. As I still have a Fuji X-t30 and some of the Fuji X Weekly presets are working on that camera I would invest time and submit photographs from both models and work through the list of Fuji X Weekly.
Sure no new film sims possible but it is a start.

If you just add needed additional values like white balance, tint shifts, exposure compensation or whatever for accurate results let me know. I would submit images of a colour checker.

My time is a little bit constrained but I think I could manage 1 to 3 submissions per week to expand the table of Pen-F film simulations.

So far, let me know how and when and we'll give that small power house a second life 😁

 

[ibd]

Hi all!

This project has been dormant for a bit as I have been busy with other stuff.
I plan to come back to it soon, though!

Nice work!

How is that supposed to be working? I mean at the command level. I see python script, but I'm unsure how to proceed. Cheers!

I may record a brief video explaining the commands and the overall process. I'm still learning about which software would be the most suitable for this kind of task.

This is great. It's fantastic that people still invest time into this camera many years after its release.
A Portra 400 simulation would be amazing too, but not sure how feasible it is (one issue with film is that the scanning process will affect the final colours as well)

The basic process would be the same. However, the analog process would introduce additional variables which are eventually sources of error, as you point out.

Hello fellow photogs 🖖😎
New Pen-F owner here and pretty much interested in this topic.

Thank you @ibd for your effort and this nice idea. Have tried some of your first film sims and like the results. As I still have a Fuji X-t30 and some of the Fuji X Weekly presets are working on that camera I would invest time and submit photographs from both models and work through the list of Fuji X Weekly.
Sure no new film sims possible but it is a start.

If you just add needed additional values like white balance, tint shifts, exposure compensation or whatever for accurate results let me know. I would submit images of a colour checker.

My time is a little bit constrained but I think I could manage 1 to 3 submissions per week to expand the table of Pen-F film simulations.

So far, let me know how and when and we'll give that small power house a second life 😁

• What do you mean by "If you just add needed additional values like white balance, tint shifts, exposure compensation or whatever for accurate results let me know."?
• Running the software and producing the profiles is much more time intensive than taking the shots and producing the input images! 🙂 In that sense, whatever pace you could manage is probably still above what I could manage to keep up with! There's hope that if I make an explanatory video of the whole process, people would be able to do it by themselves, though!

 

[ximi]

I have a follow up question: Say I find a Portra 400 lightroom preset that I like and I take a raw picture of a color target with my Pen F (even off my computer screen) and then I convert the raw file to jpeg using the Portra 400 lightroom preset. If I provide you with both the original raw and the converted lightroom jpeg could you generate the settings required to reproduce the lightroom preset on the PEN F?

 

[ibd]

Yes, that could work in theory.
Taking a photo off your screen should be perfectly fine.
Make sure to defocus slightly to avoid moiré and false colors.
Turn off all lights in your room to make sure there are no reflections on your screen.