I agree with Alaska Cub. The difference is in the exit pupil. This has a direct affect on twilight performance.
The Zeiss website has a brief read on this in their little online optics school.
http://www.zeiss.com/C1256BCF0020BE5F?The larger the objective lense is the larger the exit pupil. If the exit on the 40mm is larger than your eye pupil then the even larger exit pupil will not add much but on the higher powers it will allow more light to reach your eye and make it apppear brighter.
Here is the math on 20 power.
40mm / 20 power = 2mm
50mm / 20 power = 2.5mm
In very low lighting that .5mm is noticable. Could make a 10 minute of usable daylight difference. That's when it counts the most.
Turn the power down and it gets less important as they are bigger than your own pupil.
40/9=4.44mm
50/9=5.55mm
This is why on the lower powers most will say it makes no difference. The extra light can't be used by your eye though I will say it is more comfortable to use a scope with a very large exit pupil.