I'm familiar w/ X-fit. Training five sessions a week is not uncommon frequency. The issue is intensity.
Do you follow the x-fit recommendation of regularly doing a week at reduced intensity? (What is the protocol--one easy week/month?) Do you take a month or two off a year and just do some moderate exercise? Every sensible athlete relaxes and recovers. X-fit often seems to expect people to go for a PR every day. You won't find an elite athlete anywhere on the planet who trains that way.
The elite athletes with whom I am familiar, those doing 2/day workouts, for example, do intense workouts maybe 2x or 3x a week. Their workouts may be long, but they are not pushing themselves to the brink daily. And they consume enormous amounts of food to fuel them, including adequate carbs to keep their glycogen stores up.