Oh no. The 23mm is a must. It has taken me the thick end of five years to get back to where I was in terms of my perfect compact travel kit - M7.85 and 35mm F2.0 Zeiss Biogon, 50mm Elmar-M F2.8 and 90mm F2.8 Elmarit translates to X-Pro2, Fujinon 23mm f2.0, 35mm F2.0 and 60mm F2.4.
But there may well be "room" for that 16mm... I usually travel with the 14mm but the extra speed would be most welcome.