FWIW, I never minded the 20mm's AF when shooting it on the E-PM1 - but what I didn't like was the unruly, ratchety aperture mechanism. The 20mm is no speed demon, but pretty accurate; I still find it more than adequate to use on the GF1 where the aperture doesn't rattle so much. I guess that's happening if the camera gets the lens ready to shoot - on Olympus bodies, AF efficiency seems to be "prioritised", which may make the aperture open up again after user adjustments and metering ... I'm not sure, but that's what it looks like.
However, I now own the 27mm f/2.8 - and the 20mm isn't far behind at all, neither in IQ (where I think it's equally good, with the advantage of a brighter aperture) nor in AF speed. Only deploy time is lagging (it takes its time to wake up on Olympus bodies).
But as you know: All other lenses you mention (14mm, 15mm, 17mm) *clearly* beat the 20mm for speed. I'm glad I'm not facing the conundrum you've put yourself into, David: For me, the advantages of the 15mm over the 17mm optically are fortunately obvious, and the 15mm does away with all weaknesses the 20mm has, whereas as a compact pancake lens, the 20mm holds its own, especially since its optics are really something worthwhile, always have been. That's not the case for the 17mm in my opinion - nice enough, but not on the same level (aberrations are too high for my taste, resulting in only *mostly* pleasing images). Wether you keep the 14mm or the 15mm (or both), the 20mm makes the better companion in my view. Of course, YMMV.
If Panasonic would take the time to upgrade the electronics (motor, control) of the 20mm, I'd be all over the result ...
M.