is there a place to store all your weapons and shit u are not using?
Buy a house, they usually go for 5k and you can save that up in no time.
I stored all my things in a barrel for a while but its not a good idea to do, luckily stuff you find earlier in the game becomes worthless after you level up. Theres a system in Elder Scrolls games where the level you are affects the type of equipment and items you find from enemies and in caves/mines/dungeons etc
Thats why you want to level up and explore those places (don't clear them yet) for later but if you go through a few it won't be a big deal. I believe it applies in this game as well.
You can also give your extra stuff to a companion if you have one, don't forget they will use the items you give them. (Weapons, armor, rings, amulets) Mine has better shit than I do lol