Workout frequency may be too high....as that is usually seen alot as well. Especially as you get older.....you time to recover takes longer.
I've found more optimal results (for energy and recovery) with one hard workout every 3-4 days....the rest is just play time to walk, hike, bike....whatever. As long as the workouts keep progressing you will make long term progress.
Recovery sometimes is more time based than eating more in a shorter base of time....unless you are like 18 and then you can probably get away with anything....but that will come to an end at some point (I learned that the hard way....burned out and injured).