I typically shoot M, so the most important thing for me is quick access to shutter speed, aperture, and ISO.
The G9 is great in this regard, with the front and rear dials controlling shutter speed and aperture and rear control for ISO.
Cameras with front and rear dials plus one button access to ISO are also very good. GX85 and Pentax K-3 fit this category.
Cameras with one dial that toggles between shutter and aperture plus one button access to ISO are still very workable. This is where the diminutive Pentax Q shines vs. similar sized cameras
Cameras below this are still workable but I often choose not to fight the camera and instead treat it like a point-and-shoot with exposure compensation. Lumix GM1, Nikon 1 V1 and J5, and other cameras fit this category.
Everything else - grip, display, EVF, menus, etc... - is secondary to me.