Players have been complaining that Xah'Rith is too difficult and makes normal progression through the act frustrating. Lylirra chimed in and noted that Blizzard agrees he is a bit too challenging and they may make changes in the future.
Blizzard Quote [source]
It's not a bug, but we're definitely paying close attention to player feedback about Xah'rith's current difficulty -- not just in Inferno, but in all difficulty levels, and at all MP levels too.
We agree that he may be a little bit too challenging at the moment, and are discussing the possibility of either reducing his outgoing damage, tweaking what affixes he spawns with, or moving his spawn location (right now, he's kind of hard to avoid if you're just questing normally). We'd like to hear more from the community on the topic first, though, before we make any decisions.
In the end, we want the keywardens to provide a fun challenge for players, and be a little more difficult than the surrounding wildlife (i.e. a fight you have to put some thought into), but we don't want that to happen at the expense of being able to progress through an Act.