Well yes, those are good suggestions, but having such combo like wisdom influencing building job there would be really lots of combinations possible (20x20... 400 different combinations! or maybe more?). It would be lot of work!
I'll change the stats decrease for sure though, but have bad events that decrease them (there are already some, probably will add more). But won't be a continuous decrease like now, except for the jobs decrease.
Right now you have a decrease of a certain skill if you do a job (like farmer, increase stamina, decrease charisma), plus every 1-2 months you get a decrease on all unused stats randomly (not big but of 1-2 points).
I will remove the latter decrease, and probably enlarge the skill range to 0-200.
Another option would simply be that after a certain value, like 60-70 the skills would be much harder to train. So could take you 1-2 in-game year to raise your strength to 50-60 but then, would take you 5 years to raise to 80-90. Probably that's a better solution