Since you are talking specifically about damage, we'll compare a damage skills ( Small Guns in 3 and Guns in New Vegas) to Commando in Fallout 4.
I looked into this, and I can sort of see where you are coming from. How is this any different than noting that if you don't put points into Small Guns or Guns early on, you can't do jack shit? What do you think though? I'd like to hear if anyone feels this way too. Least that's how I feel, keep in mind I play on console, not PC (offline because no internet for my PS4 & console mods disable achievements which I don't want) otherwise I'd try those fancy PC mods I see on the Nexus, you lucky PC sods with your cool guns & weather mods. and you'll be killing shit very frequently to level up (unless your going for a pacifist run but that's silly) so if I don't get those dmg perks early on you can't do jack shit. Idk, I just feel weak playing Fallout 4 again since all your damage is dependent on perks.
I especially liked New Vage's Wild Wasteland perk, it was silly & not necessarily but it was good for a laugh. I mean, in Fallout 4 everything is perk reliant, but in Fallout 3 & New Vegas you could put points into a skill & get better at a certain thing (small guns, speech, science est) then depending on your player level, Special Stats, and your skills then you could pick what perks suited you.
