Question about stories in the written Torah

Almost every week while reading the parasha, I encounter questions that largely converge on the same question that is simply expressed in different examples. Almost every time there is a story in the Torah (and in the Bible in general, but in the Torah it is more disturbing that it exists), there are either missing details or excess details that are not clear why they are there, sometimes the entire story is unclear. The midrashim and the rishonim reconcile in all sorts of ways, which raise the following question: If the Torah really wanted us to understand this message or know the story in its entirety, why didn't it simply write it explicitly, but express it in some hint that requires hard work to extract the solution from it? This question is even more difficult for me in places where there are many different solutions to the same difficulty. What is the logic in the Torah expressing itself in a way that makes it very difficult for us to understand what it wants from us?
On the other hand, in places where the solution to the difficulty is not very important to us but only a superficial mediation, why did the Torah even bother to mention the unnecessary or missing detail? (An example of this is in our parasha 3:8, "And the Canaanite and the Perizzite were then living in the land" and Rashi there, this is a nice mediation, but why would the Torah bother to mention it at all?)