What difference does it make whether I define payment for damages as a punishment or as compensation to the injured party? It just doesn’t make sense that a thief breaks in, takes expensive items, and we would rule that he is exempt from returning them because he was potentially subject to being killed?

This is a general question about the rule of "he receives the greater of the two penalties." Why does a person who commits an additional offense become exempt from punishment for the first offense? The question regarding a thief is a bit sharper, since it comes at someone else’s expense, but basically it’s the same question. If we understand the first, we’ll understand the second as well.
I once thought it might be possible to explain this rule by saying that the Torah does not want the more severe punishment to be cheapened. If a person murdered and stole, he incurs death and payment. If we obligate him to death and also collect 100 shekels from him, that cheapens the prohibition of murder and the death penalty. The Torah may want to highlight death as something total and infinite, beside which there is nothing, and to which there is no point in adding anything. Therefore it says that if a person is liable to death, he is exempt from everything else (we are no longer willing to relate to any other aspect of him. He no longer exists).
If that is the explanation, then perhaps one can say that in order to achieve this sharpening, the Torah is willing to do so even at the expense of other people’s money.

I’ll add here that in the Talmudic passage about a prostitute’s payment, where the Torah prohibited it even in a case where someone had relations with his mother, Rashi writes that even monetary liability that is certainly not a punishment, like payment of a prostitute’s fee, is overridden by the death liability. Based on that, one could perhaps also say the reverse—that there is no need for the death liability to stem from punishment either.
And perhaps one should distinguish between the more severe liability, which needs to be a punishment, and the lesser liability, which apparently does not need to be a punishment at all—as in the Rabbi’s last explanation.

Rabbi Michi,
A. Seemingly, whoever is supposed to fund public goals like these is the public. What did the poor injured party do wrong, that specifically he should be made to fund the public-awareness budget showing that death is severe? If they want to exempt the thief, then exempt him—and let the public treasury compensate the victim (and he too will participate in the payment as his share as a member of the public). Or maybe since from the outset everyone is equal regarding who will be the unlucky schlemiel from whom they seize this public-awareness budget, then it’s equal and fair.
B. I didn’t really understand how this ultimately explains the law of a pursuer who broke vessels, where after the act (when his pursuit failed) he is exempt from death and yet exempt from monetary liability as well. And the same with those liable to death who acted unintentionally. What kind of strange highlighting is this?

Regarding the pursuer who broke vessels, there was a typo, and it should say: exempt from death and nevertheless *exempt* from monetary liability.

A. Exactly the same way the price was imposed on a mamzer child in order to deter offenders.
B. He was exempt at the time he broke them, and that exemption does not later disappear.

And those liable to death who acted unintentionally—that is an emphasis on the severity of the prohibition. Anything that is a prohibition whose law is death, nothing should be added alongside it.

A. In the case of a mamzer, you can’t impose it on the public. And who says the goal there is only deterrence?
So here you’re accepting picking on some innocent, blameless individual in order to create deterrence for others or for various public goals like publicizing the severity of death. I thought one of your central arguments against grounding all punishment in “deterrence” was that if so, it doesn’t really matter that the one punished actually be the offender; it’s enough that the public think the offender is the one being punished.

The explanation is that the risk he took is his work, and the theft is the wage for that work.

When the view is that every taking is a kind of theft, since everything belongs to everyone (because everything is from You), then the ordinary person who works steals without risk but with a business agreement, while the thief steals with risk and without a business agreement.

This is not about punishment but about withholding compensation. A person pays prices for the public good. That is always the case. When they forbid me poultry with milk so that others won’t come to eat meat with milk (and I’m sure I wouldn’t eat it), and so on—that’s the same thing too.

But why should only he fund the public good and not everyone together?

By the way, it’s also a stretch why the two liabilities have to be simultaneous. If he desecrated the Sabbath yesterday and stole today, then he does pay and is executed.

Now I remembered an old responsum here https://mikyab.net/%D7%A9%D7%95%D7%AA/%D7%9C%D7%92%D7%91%D7%99-%D7%94%D7%9E%D7%A6%D7%91-%D7%94%D7%A4%D7%95%D7%9C%D7%99%D7%98%D7%99

Among other things, a story came up (very precious in my eyes, and a real anchor for me) about a rich man who wanted to evict a widow from his apartment because she had no way to pay rent, and people complained to him about his hard-heartedness and were astonished—how could it be, etc. And that rich man argued that he would be happy to pay the rental expense as his proportional share, like any public expense, but why should he specifically pay all of it? And to the question, “In your view, should the rich man refrain from evicting the family, while the community fund should not participate?” you answered in these words: “I don’t think he has such an obligation. Not at all. Let the community kindly participate in the expenses.”

But after the distinction between “punishment” (where one does not sacrifice an individual as a burnt offering on the altar of the public good) and “withholding compensation” (where according to your explanation of this rule, we do sacrifice the unlucky individual on the altar of the public good), maybe you would say that forcing specifically the rich man to keep the widow in his apartment is like taking money from him by force and is similar to punishment.
[And it requires clarification whether if someone receives a disability allowance from “the state,” maybe that one may indeed properly and fittingly and preferably be garnished in order to have money to hang up signs reminding people that death is a very undesirable thing, and therefore they should avoid things punishable by death and understand that those things are very severe. And maybe only an added allowance that was granted after he became disabled would be garnished for the public good, but not his basic allowance—something like a woman divorced after betrothal, according to the view that she receives her basic ketubah but not the supplement. And maybe if, God forbid, he is disabled in his legs, then one may garnish his allowance, but if disabled in his hands, since a hand has the status of agency, we grant him the role of an agent of the public to fund the above publicity and garnish it.]

Because in Jewish law there is no accessible public treasury, and that’s not how things work. If the cost falls on someone, then he bears it.
It has to be simultaneous, because otherwise there is no sharpening effect. When a person committed offense A and incurred death, and afterward committed offense B and incurred monetary liability, there is no reason not to obligate him in both. When it is the same act, resorting to the money is petty and cheapens the death. Someone liable to death cannot from that point on rob and cause damage to the entire universe.

I understand. But it seems to me that this means we are dealing with cheapening the severity of the act that incurs death, and therefore this rule depends on the act, and the community also does not compensate (if there is a fund), because with such a severe act we do not want to resort to additional marginal side issues. But if we are dealing with cheapening the death penalty in order to show that nothing is above it, then it is strange that this rule depends on the act being simultaneous, whereas one after the other he pays and is executed; and it is also strange that the public does not compensate from the communal fund, if one exists, and instead places the loss on the victim.