Vinvin,
So, I shot with a full-on pro Canon kit (1D-series bodies), all the big white glass for years before jumping in to Fuji with the purchase of my X-Pro1.
So, if I may, I'd like to share my thoughts and recommendations, and my experiences which differ a bit with Lightmancer's regarding the X100T and my experiences with my Olympus M4/3s system.
If you were starting out with interchangeable lens Fujis, as much as I love my X-Pro1, I would recommend starting out with the X-T1. Why? Because this body is much more functional with the new pro zooms that Fuji is coming out with right now, and you may at some point decide to get one of those lenses. I just got the XF 50-140/2.8 (which is amazing), and I can tell you that using this lens on an X-Pro1 would be pretty difficult; the AF system wouldn't be able to keep up with the lens, and you'd have to use the EVF on the X-Pro1 for any focal length past 50mm. Also, the optical viewfinder with the X-Pro1 is really best suited for the standard/wider primes; the framing box in the OVF gets pretty small at 55mm. I personally think using the X-Pro1 is wonderful using the 14, 23, and to a lesser extent the 35 (it's AF motor is slow), but the 56 would have a small frame box if you are using the OVF. Again, the X-T1 would be better in this application. Regarding your choice of lenses, I would recommend passing on the 35/1.4. Why? Even though it's image quality is superb, it's AF motor is slow, with the lens hunting for focus back and forth. I have one and I hardly ever use it as I find the 18-55 to be a much more useful lens, and the image quality of this lens is so good, I've never thought to myself, "Oh, I should have used the 35 prime here." The 14 and 23 primes are different, those lenses are brilliant and I absolutely love them.
Also, the X-T1 has a 14-bit imaging engine, as opposed to the 12-bit one in the X-Pro1. Now, don't get me wrong, I love, love, love the image quality from my X-Pro1; it has a *something* to it that is absolutely magical. But, I think using it is best suited to applications where one is working in the classic rangefinder use scenario, with wide/standard fast primes, the 14, 23 and 35 lenses.
I would never put anyone off getting an X-Pro1, but I think if you're starting out, an X-T1 is a more versatile choice.
For a much broader range of use professional/editorial/commercial scenarios, it is tough to beat the X-T1, because it can use those primes as well as the new zooms and the longer focal lengths much more effectively. The EVF in it is so good that in the year I have been using it, I realized the other day when shooting with it that I had completely forgotten that I was not using an optical viewfinder of a DSLR (see my go-karting photos I just shot with the XF 50-140 native lens showcase forum).
Regarding the X100/X100S and the X100T. I rented both the X100 and X100S for two week intervals and shot with them extensively. Like Lightmancer, I never bonded with those two cameras. The X100T, IMO, is a whole other story. I shot with one at a local camera store and immediately fell in love. This model is notably zippier in operation, has snappier AF, and I love the small but important design changes Fuji made with the knurling on the aperture ring and control dials, and the the refinements made on the shutter speed/comp dials (each one turns with a different level of force; brilliant!), the improved and revised buttons and D-pad, the WiFi connectivity, and the brilliant small EVF that pops up in to the OVF window. As opposed to the X100/X100S, I absolutely love this camera, and Fuji has refined it to a "T" (pun intended!). My X100T goes with me EVERYWHERE.
Regarding the Olympus M4/3 system. I have an E-M1, and four M4/3 lenses, including the really nice Panny 12-35/2.8 and 35-100/2.8 G series zooms. While I think the camera body, build quality, and performance is excellent, as are the lenses, the truth of the matter is I hardly ever shoot with it. Why? Because the Fuji X-camera image quality is so much better. The images from the Olympus are truly excellent, but the images from the Fuji are magical. Also, the E-M1 doesn't have the auto white balance performance of the Fuji, and the noise performance at higher ISO can't compare.
With all that said, I can't think of a reason why you should wait to plunge into the Fuji X-system; I think you will love it as much as the rest of us, and will be really happy with your decision.