Have you ever wonder why your buddy gains muscle and you don't? You both train the exact same pattern but at the end the result is not the same.
Yes, rest and nutrition take it's toll.
There's another factor that takes place here.
Genetics.
Basically there are three body stereotypes.
Keep in mind every person have a little bit of everybody type - no one is 100% clear.

• * Ectomorph body type - Those are the thin guys.
  They have small muscles, thin waist and small hips.
  They also have narrow shoulders.
  Those are the typical hard gainers.
  

Training of the ectomorph body type must be intense with a lot of compound exercises.
They must reduce all cardio to minimum in order to gain muscle.
They require larger rest time after each workout in order to build muscle.
The training should be short and highly intense.
Those guys have the fastest metabolism, which allows them to eat huge amounts of food and have no affect on their fat levels.
To gain muscle they must have medium carb intake and a lot of protein intake.
It is said they also benefit from taking good fats although I advise to take limited amounts of the last.

• * Endomorph body type - Those are the opposite type of people.
  They store fat pretty easy and burn it pretty slow.
  They are usually big fellow and got slow metabolism thus no problem gaining weight.
  

Their training should consist on lighter weights and more reps.
Their training also must be intense.
Also they should include cardio two times a week in their workouts.
This will ensure additional reduce of the fat levels.
Food should be lot fat and low calories.
Best is to eat often in small portions to speed up the metabolism.
The food should be mostly protein and the main meals should have some carbs in them.
Gaining muscle requires certain amount of carb intake.
If the goal is burning fat then the carbs should be reduced.
This however is another topic.

• * Mesomorph body type - The typical body builder type.
  Broad shoulders, narrow waist, fast metabolism, athletic type.
  They got big potential in building lean muscle fast.
  

They can train often due their fast recovery rates.
Also any kind of intense training is enough for them to have that process of muscle growth going.
They have very little risk of overtraining (yet they still must follow safe workouts).
Again when hitting the diet - they can eat almost everything.
Due their fast metabolism they build muscle and store little fat on most of the foods.
Yet medium protein and good carb diet will work best for this body type.
So let me say that again, typically no person is only one stereotype.
Most have a little bit of at least two types.
Know your type (which you should already do) and adjust your training and diet according that.
This will ensure maximum muscle gain for your situation.