FAQ FAQ  Forum Search   Events   Register Register  Login Login

DIY Lens MTF Testing

Page  123>
Author
craig66 View Drop Down
Senior Member
Senior Member

Joined: 03 April 2012
Country: Australia
Status: Offline
Posts: 637
Post Options Post Options   Quote craig66 Quote  Post ReplyReply Direct Link To This Post Topic: DIY Lens MTF Testing
    Posted: 21 December 2013 at 08:02
I recently came across a free open source software product called "MTF Mapper". This piqued my interest. It measures MTF50 and produces various output charts from images of a test target made with the camera/lens combination under test. It is very easy to use. It also has facilities to assist AF micro adjustment that I haven't looked at because my A65 has no such capability. It can handle RAW files directly (it uses dcraw).

For anybody who might feel inclined to waste some time testing their gear out of curiosity or whatever, I though my initial experiences might be useful.

I made a test target by using the supplied test chart generated by a supplied utility. It outputs SVG that I converted to PDF and then printed over four A4 sheets that I glued to a foam board backing. These were printed on a 12 yr old low end HP Officejet with a black ink cartridge that has been in the printer for several years on ordinary copy paper. So the test target does leave a little to be desired.

MTFMapper uses the slanted edge method to estimate MTF and requires a chart with black rectangles on a white background. This is what my test chart looks like:



Various outputs are produced, but the one with the most eye appeal is the 3D surface plot of MTF50 across the entire field. This is a kind of inverse of slrgear plots that show "blur units" across the entire field.

So here's some sample output. Note the scale on each chart - the colours are relative to each charts scale.

A65 Sony DT 50mm f1.8 @ f5.6 ISO 100 RAW




A65 M42 SMC 50mm f1.4 Takumar @ f5.6 ISO 100 RAW. Lighting and exposure was different from those with the Sony lens and focus may not be absolutely exact in either case, so I would definitely not use these to draw any conclusion as to the relative merits of the two lenses.




And finally the 50mm Tak wide open at f1.4. This lens is soft wide open and the MTF chart confirms that. The chart is skewed which may be due to not being properly perpendicular to the test target, or some misalignment due to the M42 adapter or the lens itself.




The "canyon" in the MTF plots is I believe an artifact of the test target and would probably just go away with a higher quality target.

Overall MTFMapper is very easy to use and does what it says on the tin. With a better test target it could be quite useful for lens testing and comparison. In particular if a baseline is established with measurements of lenses known to be good, it could be used as a quick and quite demanding test on any new lenses as they are acquired.
 



Back to Top
Alex H View Drop Down
Senior Member
Senior Member
Knowledgebase Contributor

Joined: 06 November 2008
Country: Sweden
Location: Stockholm
Status: Offline
Posts: 1591
Post Options Post Options   Quote Alex H Quote  Post ReplyReply Direct Link To This Post Posted: 21 December 2013 at 08:14
Thanks for sharing. It might come in handy, especially to test old second hand lenses for improper assembly.
Back to Top
craig66 View Drop Down
Senior Member
Senior Member

Joined: 03 April 2012
Country: Australia
Status: Offline
Posts: 637
Post Options Post Options   Quote craig66 Quote  Post ReplyReply Direct Link To This Post Posted: 21 December 2013 at 08:37
Originally posted by Alex H Alex H wrote:

Thanks for sharing. It might come in handy, especially to test old second hand lenses for improper assembly.


Yes. I'm going to follow this up a bit further. The trick is to make a test chart with high quality edges. I think the MTFs on the above charts are a bit low for those lenses on a 24 mpix APS-C sensor. And I think the poor edges on rectangles with a small slant causes problems.

Imatest say that ordinary razor blades are their gold standard for edges - better than any of their (expensive) test charts. I'm thinking it may be possible to just glue razor blades to translucent material such as white acrylic sheet and illuminate from behind.
Back to Top
Bob J View Drop Down
Admin Group
Admin Group
Dyxum Administrator

Joined: 23 December 2005
Country: United Kingdom
Location: London
Status: Offline
Posts: 26430
Post Options Post Options   Quote Bob J Quote  Post ReplyReply Direct Link To This Post Posted: 21 December 2013 at 09:28
Originally posted by craig66 craig66 wrote:

printed over four A4 sheets that I glued to a foam board backing...

...The "canyon" in the MTF plots is I believe an artifact of the test target and would probably just go away with a higher quality target.


I note that there is a vertical and an horizontal 'canyon' that would seem to conform to the edges of the A4 sheets; I'd imagine printing on one sheet of A2 might solve some of the problems that might come from having to align the four sheets.
Back to Top
craig66 View Drop Down
Senior Member
Senior Member

Joined: 03 April 2012
Country: Australia
Status: Offline
Posts: 637
Post Options Post Options   Quote craig66 Quote  Post ReplyReply Direct Link To This Post Posted: 21 December 2013 at 09:50
Originally posted by Bob J Bob J wrote:

Originally posted by craig66 craig66 wrote:

printed over four A4 sheets that I glued to a foam board backing...

...The "canyon" in the MTF plots is I believe an artifact of the test target and would probably just go away with a higher quality target.


I note that there is a vertical and an horizontal 'canyon' that would seem to conform to the edges of the A4 sheets; I'd imagine printing on one sheet of A2 might solve some of the problems that might come from having to align the four sheets.


That's certainly a logical question to ask, but I also tried the test chart printed on a single A4 page and it also produced a "canyon". It also resulted in lower MTF figures. I don't think the placement of the black rectangles matter much but the edge quality matters a lot.

In the end there's only one cure - procure a better chart.
Back to Top
Bob J View Drop Down
Admin Group
Admin Group
Dyxum Administrator

Joined: 23 December 2005
Country: United Kingdom
Location: London
Status: Offline
Posts: 26430
Post Options Post Options   Quote Bob J Quote  Post ReplyReply Direct Link To This Post Posted: 21 December 2013 at 10:13
Well, I was inspired to look up the definitions of Sagittal and Maridional; from what I can make out Sagittal is ventral-dorsal (back-front?) while Meridional is North-South/up-down... I have to confess I'm none the wiser about what each is showing on the chart other than I wonder if Meridional is showing something to do with astigmatism

Ventral-dorsal has me really stumped: Is there a simple explanation that a bear of little brain might understand?

 



Back to Top
craig66 View Drop Down
Senior Member
Senior Member

Joined: 03 April 2012
Country: Australia
Status: Offline
Posts: 637
Post Options Post Options   Quote craig66 Quote  Post ReplyReply Direct Link To This Post Posted: 21 December 2013 at 10:43
Originally posted by Bob J Bob J wrote:

Well, I was inspired to look up the definitions of Sagittal and Maridional; from what I can make out Sagittal is ventral-dorsal (back-front?) while Meridional is North-South/up-down... I have to confess I'm none the wiser about what each is showing on the chart other than I wonder if Meridional is showing something to do with astigmatism


There's a nice explanation with clear diagrams here in the section on MTF:

http://www.cambridgeincolour.com/tutorials/lens-quality-mtf-resolution.htm

And a little further down a section on the connection with astigmatism, also with a couple of good illustrations.
Back to Top
Blame View Drop Down
Senior Member
Senior Member

Joined: 14 October 2010
Country: United Kingdom
Location: London UK
Status: Offline
Posts: 2626
Post Options Post Options   Quote Blame Quote  Post ReplyReply Direct Link To This Post Posted: 21 December 2013 at 10:55
Results will usually be poorer using a single a4 print as non-macro lenses are not particularly sharp close focused. Still worth finding out how much they would loose for a tight head shot.

It might not be that expensive to order a professional A1 print.
A900, Min 24-105, 35-105, Samyang 14/2.8, 35/1.4, Sig 70/2.8 Macro, ISCO Ultra 125/2, Tam 180/3.5 Macro, Sig 400/5.6 TeleMacro
Back to Top
stiuskr View Drop Down
Moderator Group
Moderator Group

Joined: 01 September 2006
Country: United States
Location: West Virginia
Status: Offline
Posts: 11417
Post Options Post Options   Quote stiuskr Quote  Post ReplyReply Direct Link To This Post Posted: 21 December 2013 at 11:38
It's been a while since I've searched for test charts and I haven't found any inexpensive ones anywhere. But I just ran across these free charts for anyone to use, the only problem is that you have to be flying high above them!

http://petapixel.com/2013/02/15/there-are-giant-camera-resolution-test-charts-scattered-across-the-us/
Rob Suits Jr.
a99M2 a99 a77 a700 KM7D|Min24/2.8 Min35/2 So50/1.4 So50/2.8 Min85/1.4G Tam90/2.8 Tam180/3.5|Tam17-50 CZ24-70G2 KM28-75D So70-200G1 So70-300G So70-400G1| SonyF60 AD200R2
Back to Top
Blame View Drop Down
Senior Member
Senior Member

Joined: 14 October 2010
Country: United Kingdom
Location: London UK
Status: Offline
Posts: 2626
Post Options Post Options   Quote Blame Quote  Post ReplyReply Direct Link To This Post Posted: 21 December 2013 at 11:44
Hey - proof that that those UFO's have cameras. They don't just leave crop circles you know. I wonder if they use Sony's?

It explains those alien abductions too. Probably trying to find Sony executives. Like "any sign of intelligence?" "none yet they just discontinued the OVF".

Edited by Blame - 21 December 2013 at 11:53
A900, Min 24-105, 35-105, Samyang 14/2.8, 35/1.4, Sig 70/2.8 Macro, ISCO Ultra 125/2, Tam 180/3.5 Macro, Sig 400/5.6 TeleMacro
Back to Top
craig66 View Drop Down
Senior Member
Senior Member

Joined: 03 April 2012
Country: Australia
Status: Offline
Posts: 637
Post Options Post Options   Quote craig66 Quote  Post ReplyReply Direct Link To This Post Posted: 21 December 2013 at 14:29
Originally posted by Blame Blame wrote:

Results will usually be poorer using a single a4 print as non-macro lenses are not particularly sharp close focused. Still worth finding out how much they would loose for a tight head shot.


Yeah that may be a part of it, but mostly I think it's just a poor test chart. I pulled the camera back further so the chart was not filling the frame and I'm getting 50+ lp/mm @ f5.6 which is more like the kind of numbers you should expect from a prime.   

It might not be that expensive to order a professional A1 print.


That's probably the best move. A quick bit of googling suggests a price of about $40 for A1 on 200 gsm glossy photo paper. If they would accept an SVG file that would be ideal, PDF second best but I'm a bit suspicious of converting to jpeg. Will follow up on Monday.
Back to Top
Blame View Drop Down
Senior Member
Senior Member

Joined: 14 October 2010
Country: United Kingdom
Location: London UK
Status: Offline
Posts: 2626
Post Options Post Options   Quote Blame Quote  Post ReplyReply Direct Link To This Post Posted: 21 December 2013 at 19:14
You could possibly do better than $40. A quick check on UK ebay showed £13.60 for one or £22 for 2 -- and that was A0.

I would avoid gloss is you can. Ideally you want no reflections so mat would be best although I have only seen gloss or satin.

Simple geometric patterns should be ideal as jpg format. Could you generat jpg but at really high definition like, say 512MP ? That would be about 600dpi for A0.

Edited by Blame - 21 December 2013 at 19:19
A900, Min 24-105, 35-105, Samyang 14/2.8, 35/1.4, Sig 70/2.8 Macro, ISCO Ultra 125/2, Tam 180/3.5 Macro, Sig 400/5.6 TeleMacro
Back to Top
Blame View Drop Down
Senior Member
Senior Member

Joined: 14 October 2010
Country: United Kingdom
Location: London UK
Status: Offline
Posts: 2626
Post Options Post Options   Quote Blame Quote  Post ReplyReply Direct Link To This Post Posted: 22 December 2013 at 09:13
Forgive me but I have had a closer look at your chart. If you look closely at it there is a largish horizontal gap between the rows about halfway down. The is also a smaller vertical gap between horizontal rows where the two sheets meet. That is, in both cases, the gaps are larger than they should be.

Also by rights every square should have a face directly facing a central point but they don't. It isn't off by that much but if you take the ones close to the center you can see that lower squares point to a different position to the upper squares.

I am sure that it also could do with higher definition but the gaps do explain the centered horizontal & vertical oddities in your results. There is something wrong in the test chart generation program. Maybe like it was meant to be 4 sheets with overlap (because printers never print to the edge)but you have printed as 2 (horizontal gap) and then not quite got the overlap right between them (vertical gap).

Edited by Blame - 22 December 2013 at 09:26
A900, Min 24-105, 35-105, Samyang 14/2.8, 35/1.4, Sig 70/2.8 Macro, ISCO Ultra 125/2, Tam 180/3.5 Macro, Sig 400/5.6 TeleMacro
Back to Top
craig66 View Drop Down
Senior Member
Senior Member

Joined: 03 April 2012
Country: Australia
Status: Offline
Posts: 637
Post Options Post Options   Quote craig66 Quote  Post ReplyReply Direct Link To This Post Posted: 08 February 2014 at 09:44
Following up on this ...

I was not really happy with the surface charts for a couple of reasons. First is that there is no absolute scale and no absolute colour key, green on a chart for one lens is at a different level from another. Casual comparisons of lenses from these surface charts is easily open to misinterpretation.

A second reason is that I do not think they are sufficiently smoothed. The lumpiness is an artefact of the test system and not a quality of the device under test. mtf_mapper uses gnuplot to generate the charts. I had a look at taking the raw mtf data from mtf_mapper and changing the charting, but gnuplot does not seem to offer much in the way of smoothing for this kind of chart. It needs to be done elsewhere and things start to get quite a bit harder.

These two factors and some other issues with grid style test charts (both those generated my mtf_mapper and by a program I wrote) lead me to take a bit of a different tack. I created a test chart to mimic the one used by photozone. It's not exactly the same but quite similar. Edge slant is one in ten. I also added some Siemens stars as a focusing aid.

The command line version of mtf_mapper is used to generate a file of raw MTF values. To process this, I wrote a Java program that uses exiftool to extract camera and lens names, focal length and f stop from the raw images. The program massages the raw MTF data into suitable format for gnuplot and generates and executes gnuplot script to produce charts like these:







The number in brackets after the f stop on the x axis is the number of test images examined for the camera/lens/focal length/fstop combination. Multiple images can result from focus bracketing, or changing the point of focus on the test chart from centre to corner. In all these cases above, I refocused on each corner as well as the centre.

At each fstop the program finds the best edges for each rectangle measurements and uses those to produce the output barchart. The "error bars" are just an indication of spread of measurements - don't take much notice of them.

Compared to photozone results (especially for tests of lenses on Nex-7), the results above look reasonable to me and in the right ballpark. Given the spread in MTF measurements you see around the web, asking for more might be a bit much to hope for. The border and corner measurements may be a bit low - a subject for further study.

I'm pleased with this outcome. Just shoot the (carefully focused) test images and the rest is automated by script. Half an hour max per lens. This is what I was looking for. Next step is to get a professionally printed test chart(s) instead of the mickey mouse setup of 9 A4 copy paper sheets sticky taped to an A1 backing.
Back to Top
Dyxum main page >  Forum Home > Equipment forums > Lens Talk Page  123>

Forum Jump Forum Permissions View Drop Down



This page was generated in 0.094 seconds.

Monitor calibration strip

Dyxum.com - Home of the alpha system photographer

In memory of Cameron Hill - brettania

Feel free to contact us if needed.