Rabbi Michi, do you have a source for this leniency? The distinction between affectionate touch and touch that is not affectionate?

See any summary of Jewish law. Even on Wikipedia:
https://he.wikipedia.org/wiki/%D7%A9%D7%9E%D7%99%D7%A8%D7%AA_%D7%A0%D7%92%D7%99%D7%A2%D7%94

"See any summary of Jewish law. Even on Wikipedia"
Unfortunately, that is really not correct. And thank goodness we do not issue halakhic rulings from Wikipedia.

Here is a halakhic summary from the Kipa website, which is not suspected of being Haredi. And the same appears in many halakhic summaries.

Question: Is a soldier allowed to shake hands back with a female soldier who extends her hand to him? After all, this is not a gesture of affection but simple politeness. Some want to bring proof from the Jerusalem Talmud (Sotah 3:5): "the evil inclination is not present at that moment," and therefore the priest was allowed to place his hand under the hand of the suspected adulteress at the time of the offering. If so, why can’t one give a small, brief handshake? Isn’t a soldier on duty considered occupied with his work, so that the touch should be permitted?

Answer:

A. First of all, there is no blanket permission for mere touch. According to the Beit Yosef, in Maimonides’ opinion even touch without affection is prohibited by Torah law, and one must be killed rather than transgress it (Yoreh De’ah 195). According to Nachmanides, affectionate touch is prohibited rabbinically, and according to the Shakh, touch that is not affectionate is not prohibited by Torah law even according to Maimonides (ibid.). Most later authorities agreed with the Shakh that touch that is not affectionate is not prohibited by Torah law. But some later authorities said that even so it is still prohibited rabbinically (Ezer Mikodesh, sec. 20), and according to all opinions it is an improper thing.

B. In any case, it is not clear that shaking a woman’s hand is not considered affectionate touch. Rabbi Moshe Feinstein mentions the argument that this is not a manner of affection or desire, but concludes that in practice it is hard to rely on this (responsa Igrot Moshe, Even HaEzer 1:56). The Ben Ish Chai likewise writes that this is affectionate touch (Od Yosef Chai, cited in Otzar HaPoskim, Even HaEzer sec. 20, letter 3). So too it seems from Sefer Hasidim, which prohibits clasping hands with a gentile woman (sec. 1090), even though this appears to be a custom of politeness; evidently it also has an element of friendliness. Rabbi Yaakov Yisrael Kanievsky, author of Kehillot Yaakov, said in the name of the Hazon Ish that giving one’s hand has the same status as hugging and kissing, and one must be killed rather than transgress it (cited in the book Taharat Am Yisrael by Rabbi Vagshal, p. 44). And so too in the responsa Mishneh Halakhot (6:123) and responsa Be’er Moshe (4:130).

C. In any event, Rabbi Yehoshua Menachem Ehrenberg was uncertain regarding returning a handshake, perhaps because it is not a manner of affection—although from the side of the woman extending her hand it is affectionate, perhaps from the side of the one returning it, it is only politeness. But he rejects this argument and concludes that one cannot be lenient about it, and that the woman too intends affection, so by complying he causes her to stumble and transgresses "do not place a stumbling block." He adds that one need not be concerned about embarrassing her, because by law one must separate a transgressor from sin even if that may involve embarrassment, as Maimonides writes that if there is no other way, one may shame the sinner in order to save him from sin (Laws of Character Traits 6:8). True, we do not have the power to intervene in such a case, but certainly one must not assist, even if she will be embarrassed because of it.

In general, he writes that this is not called embarrassing her; rather, she brought the loss upon herself and caused her own embarrassment. He concludes in practical Jewish law that one should not be lenient in any way whatsoever (responsa Devar Yehoshua 2, Even HaEzer 15). And the same applies to female relatives (see Maimonides, Laws of Forbidden Relations 21:6).

Question: In light of Rabbi Moshe Feinstein’s ruling that one may ride the subway even though it is hard to avoid physical contact, is it permitted to shake a woman’s hand back when she extends it?

Answer: What does that have to do with this? Rabbi Moshe Feinstein himself wrote that it is difficult to permit returning a handshake. Responsa Igrot Moshe, Even HaEzer 1:56; 4:32. (And once Rabbi Reuven Feinstein asked his father, Rabbi Moshe Feinstein: Is it permitted to sell leavened food to a non-Jewish woman? He answered: "There is no problem with selling leavened food to a non-Jewish woman, and in truth that was the custom in Uzda [his hometown], because a non-Jewish woman’s earnings do not belong to her husband. But as for clasping hands—either he should not do it, or some Jewish woman should do it." Masoret Moshe, p. 405. So it is clear that Rabbi Moshe Feinstein did not permit shaking a woman’s hand even when there is a need.)

Question: If one does not shake a woman’s hand back, she may be insulted; shouldn’t one take people’s feelings into account?

Answer: This is not embarrassing another person. On the contrary, she herself causes it. It is like a person who is hosted by his friend and is served food that is not kosher—if he refuses, the host will feel deeply embarrassed. Would that make it permissible? Or if a rabbi asks his student to marry his daughter and the student refuses, thereby insulting the rabbi—would the student therefore be obligated? Rather, one is forbidden to embarrass another person through an action that one himself initiates in positive action, but not when the other person is the initiator and embarrasses himself because the second person does not comply with his request through passive omission.

And there is a well-known incident involving Rabbi Ovadia Yosef, the Rishon LeZion, who received the Israel Prize and a reception was held in his honor. When Prime Minister Golda Meir extended her hand to him, he did not return it, and people remarked to him that he had created an unpleasant situation. He replied: I am sorry, but keeping Jewish law comes before everything. Later that evening an apology was sent to him. (The same happened when the President of the United States came to Israel and Rabbi Ovadia Yosef was serving as chief rabbi. All the rabbis of the country came to greet the president at the airport. He shook hands with everyone, and afterward his wife also shook hands. When she approached Rabbi Yosef and extended her hand, he took his hand and put it behind his back. She was shocked, and those accompanying her immediately explained that rabbis do not shake hands with women. Responsa Ma’ayan Omer 12:192, note.) Indeed, one should also take the feelings of those who observe Jewish law into account.

And once a reception was held for him by the Queen of England, and she extended her hand to him, but he stood silent and pressed his hands to his sides in full view of the royal family and the television cameras. That evening an apology arrived from the royal household for having embarrassed him, and they checked the protocol book and found it written there that the Queen should not extend her hand to a Jewish rabbi.

Question: Don’t the laws of modesty involve a kind of injury to women, a kind of hatred, a kind of exclusion? And isn’t there concern that if religious people "separate themselves from the public" and behave so differently, it will distance people from the Torah?

Answer: Heaven forbid. On the contrary, modesty preserves women’s dignity from human lust. This is what Rabbi Kook writes in Middot HaRe’iyah, under the entry "Modesty": "The trait of modesty causes much good in the world, and for that reason it merits pushing aside things that would have been good in themselves. But because, due to human inclination and human weakness, they would lead to breaches in modesty—which is the foundation of the spiritual and material world… the modest person recognizes that he does not distance himself and make fences out of hatred for the sex, but rather because of the beautiful general purpose." And so it is written in the book To the Perplexed of the Generation: "The distancing required by modesty from another man’s wife does not come, Heaven forbid, from jealousy or stinginess, but from purity of soul and holiness of character and deeds. For just as active closeness and signs of love are fitting and beautiful to make family life pleasant and complete, so when they flow from a stranger they can muddy family life and its purity. And while we distance ourselves from a woman who is not ours, we love her as a human being and strive for her good." There, p. 104.

Question: So what should one do if a female soldier extends her hand?

Answer: Of course, in all these cases one must act with great wisdom, in order to minimize the other person’s embarrassment as much as possible—for example, by making a polite bow to show respect, and by apologizing that one does not return a handshake. About all such cases it is said: "You graciously grant a person understanding."