The reason why I would use manual with a bounce flash is because I don't know how the camera is supposed to calculate the correct exposure on it's own. It has no way of knowing how high the ceiling is, what colour, what texture, etc. You basically need to dial in a good baseline setting and then adjust it as necessary. This can be a bit awkward if you are in a room with multiple ceiling heights and textures. It is a good idea to set the flash intensity for close-up shots with the diffuser in place, and then retract it for shots at longer distances. This is a quicker way to control intensity than to adjust it on the flash itself! This may be a compromise and won't always be exactly right, but assuming that Canon didn't lose the plot with the 60D in terms of dynamic range any slight under or overexposures are easily recoverable.
If using the flash head directly I would always be using the diffuser for anything within a few metres of the camera, especially on a flash like the 580EX which has the light output of your average streetlamp.
I have no issues with your assertion that the X100 was suitable for this scenario, I just don't understand why Canon guy was struggling so much.
I've known a few people who were of the opinion that one of the requirements for obtaining a driver's license should be to understand the physics of what a car does, and I have developed a similar opinion relating to cameras. I have no problems using automatic functions on cameras, and I am sure that I would find it too much hard work if I didn't, but as much as possible I try to understand the workings and limitations of those functions so that I am not blaming the camera instead of myself when things aren't working properly.
This is me coming from an engineering background however so it may just be a personal oddity.
I agree. Just like a racing driver should know every component of his car, I have been taught to listen, read, experiment and question my assumptions.
Canon E-TTL II is an advanced pre-flash TTL metering system. The Speedlite (or groups of slaved Speedlites) sequentially discharge a series of preflashes, which are metered through the lens by the camera's matrix meter. Each flash group's preflash is evaluated to determine how it contributes to the overall lighting and the flash power is adjusted accordingly. Therefore, whether you have one Speedlite on camera pointed directly at the subject, or bouncing off a coloured or textured wall, shot through a gobo, or several flashes all discharging from various positions, the system is designed to adjust each group to deliver the exposure required. Old school manual metering, mental math and trial and error strobe adjustment still offers more subjective control, but it's too ponderous for walkabout shooting at a dynamic event. On the whole, E-TTL II works rather well.
I've set up my own EOS DSLR with the same flash and lens and done some experimental shots at home. The Speedlight works as intended. -3EV is just barely noticeable, -2 fills in most of the shadow, -1 EV is just under balance and +0EV balances and almost starts to dominate the ambient. In the range +0 to -1EV, surface reflections are notable.
The X100's unmodfied flash exposure is altogether more aesthetically pleasing. Fujifilm has its proprietary "Super Intelligent Flash" programme. If I told you that I totally understood "Super Intelligent Flash", I'd be lying. None of the research I've done explains it fully. All I can tell you is that the X100 has exposed the ambient a stop above the EOS through higher ISO and the shadows appear like the EOS at -1EV flash exposure compensation. At the same time, far shadows are less underexposed. Manually recreating the cameras' ambient auto exposure but without flash shows a much healthier histogram and subjectively better exposed scene. Flash specular highlights are less harsh. I think the more detailed ambient shadows and less obtrusive flash highlights are the result of the X100 sensor's higher dynamic range (DxOMark 12.4 vs 11.9) and the RAW-utilising dynamic range compression (2 stops at DR400) done through automatic in-camera post processing. Photoshopping of RAW images using curves in Photoshop just about brought the lighting up to the X100 standard.
So you see, we do both know our cameras, how they perform and how to use them better than most. The X100 is not only technically on par with a top prosumer DSLR (in my test case a 5DII), but it delivers a better image right out of the camera. It does so for much less money too. The X100 is very cheap compared to a 60D/5DII plus 17-40/4L and 580EXII. I would support Thom's assertion that compact cameras are now at the same level as DSLRs in many ways. Where the X100 is weaker than a system camera is in system flexibility (prime lens), direct manipulation of the lens (primarily focus) and lack of an optical reflex viewfinder.