Faster is relative. But I'll explain it as best I know it but some research on your part will clarify anything I mangle.
Phase detection autofocus uses special techniques to use a small sensor array to "triangulate" so-to-speak on detail information. This process allows the autofocus system to not only know whether something is in or out of focus but also whether it's BEHIND or AHEAD of the current point of focus. This allows the AF system to immediately know which direction to turn the lens to achieve focus. In DSLR's the focus detection system sits over the prism looking down at the focus screen of the camera. It is not in the image sensor itself.
Contrast detection autofocus basically simply detects sharpness of the overall image in a certain area. But this technique doesn't tell the camera anything about where the out of focus item is compared to the lens position. The AF system makes a "guess" and based on whether the image gets sharper or softer it then decides whether it is going in the correct direction or not. So the actual sensing of focus is done from the image sensor itself.
The speed of either system depends on how fast the sensors can be read, results computed, and lens motor driven. The actual method has almost -- stress almost -- nothing to do with focus speed.
Where it matters is that little caveat about DEPTH and contrast detection's inability to perceive it. DSLR's with phase detect can nail focus in 3D much better, and track motion in 3D way, way better.
There are cameras now like the Nikon 1 that combine both systems on the image sensor itself. They work excellently.
One other thing to think about is focus accuracy. This are several factors involved including how well designed the motion systems controlling the lens motor are and so forth. Sometimes DSLR's can exhibit front or back focus issues which seem to be caused by mirror alignment, sensor mounting depth, or focus screen shimming. If any of these factors cause the focus screen to be at a different focal distance than the sensor, then the system will misfocus though often you might not even notice it except in very tight DOF situations.
With my Sony DSLRs phase detection is a real pain.
To be accurate it needs to be set up properly at the factory and Sony don't provide an easy way of making adjustments.
It's possible to tweak my Sony a little by pulling plastic bits off to access some screws but settings that work for the Kit Lens don't work with my Zeiss lens.
That's why I hate phase detection and have stopped using DSLRs now that speed is less of an issue.
No such issues with contrast detection and with the composite system like that on the X100S/ Nikon 1 you appear to get the advantages of both.