Open up to f/1.8 and you should find the clinicality greatly alleviated
Edit:
Assuming these shots are taken at ISO 200, f/4 and 1/2 sec,
at ISO 800 (+ 2 stops)
at f/1.8 (+ 2.3 stops)
at 1/30 sec (- 4 stops)
Should be fine. If the CL is too lightweight to stably hold you might get blur but that's how it would go for on a heavy M body.