A lot of the time I ask myself what degree of sacrifice it is rational to make for faith (and therefore presumably reasonable to expect of us). After all, I’m not completely certain of the truth of religion, and even if I were, I’m not sure about its details, God’s will, and the rightness of my path at every step.
– I’ll touch something that probably won’t hurt me, but I won’t touch something that probably (only probably) won’t kill me –
Is it rational (and certain enough) to keep the Sabbath at the cost of a hand?
Or more extremely, is it rational not to worship idolatry at the cost of life itself?
(Is one to sacrifice moral values on the altar of religion?)
Or is the answer sometimes no, and in every situation one should weigh risks and chances?

A very good question, though it isn’t well defined. If you’re asking about Jewish law, it has its own answer. The question is whether it’s about Jewish law, or about you yourself—whether to obey Jewish law. It seems obvious that you’re supposed to weigh probabilities and risks. There is even a similar Tosafot about someone who stuck bread to the oven wall, which assumes that a person will not obey Jewish law if it would require him not to remove the bread and be punished by death for desecrating the Sabbath. The same applies to not extinguishing a fire on the Sabbath and not saving possessions, and all that because of a rabbinic prohibition (extinguishing when one does not need the charcoal). So I’ll violate a rabbinic prohibition, but at least I won’t be left poor, without a home and without all the property I accumulated over a lifetime.
What people usually do is work around the Jewish law. They do it with an unusual method and the like. Sometimes it seems to me that these rulings are meant to establish an educational norm, but they are not meant for practical implementation. Like the stubborn and rebellious son, etc.

Let me phrase a somewhat similar question: if faith is, say, 80 percent, can that be a reason to commit a moral wrong that is 100 percent wrong?
For example, could you kill an Amalekite?

You’re pushing on an open door. I wrote in my book exactly about this example—that apparently not.

By the way, not only because of doubts in faith. Even if I were certain in my faith, there are doubts in interpreting the halakhic instruction. It is an interpretation by human beings, and it could be mistaken.

So is it permissible to ask: if the Torah could never convince me to do such a thing, then what did the Torah mean? Or is that a question beyond the boundary… Or does the Torah speak from an absolute point of view and not care that I won’t be persuaded?

The Torah instructs you what you are required to do from the standpoint of Jewish law. It also expects you to carry it out. But you make your own calculation. There will be those who carry it out and those who won’t.