As big as the Contax market was (i.e. not too big), but it is likely a halo product, which can shine on their other offerings. They're not going to be able to release a DSLR, and they don't have the muscle likely to go truly head-to-head with NEX, so niche is the right move for Fuji. I bet they get the want-it-but-cant-afford-a-Leica crowd (er, that'd be me), the "I never got off film yet" crowd, the "I want something different/retro/hipster crowd" and maybe some other small segments. I'll bet they do pretty well, though, and certainly seem in a better position than Oly for 2012, as an example (as sad as it may be).
I agree that (assuming the leaked specs are on the money) the X-Pro is exactly where Fuji should be in the market following the X100, but I can't see the actual X-Pro camera itself troubling the scorers too much when they add up sales figures at the end of the year.