As I recall, the Hazon Ish in Makkot showed that there is a commandment to remain in the city of refuge even in cases where there is no danger to the killer.
But actually, even if there is a prohibition to leave, this is not a difficult question:
1. There is a scriptural decree here that he does not leave even for the sake of a commandment: “to which he fled” — there shall be his dwelling place (even for the sake of a commandment).
2. He is considered “on a distant journey,” and is therefore exempt from the Passover sacrifice.

In the plain sense of the Torah and the Talmud, a distant journey is considered coercion beyond one’s control (though one can analyze this further).
A scriptural decree is not an explanation — especially since there is no need to reach that, as explained above.

A. In any case, certainly the city of refuge is farther away than Modi’in and beyond, so he is on a distant journey and exempt by the plain meaning of the law of a distant journey (whether it is considered coercion or an independent exemption). Again, there is then no reason to assign him protection if in any event he is exempt from the commandment. (By the way, it seems to me that even the city of refuge, ancient Hebron, is slightly farther from the Temple Mount than ancient Modi’in.)
B. If there is a scriptural decree that “he shall flee there” — even for the sake of a commandment — then no further explanations are needed. And in any case, even if we say that the derivation is not “even for the sake of a commandment” but only an ordinary positive commandment, the positive commandment of the Passover sacrifice does not override the positive commandment of “to which he fled — there shall be his dwelling place,” especially since the positive commandment of Passover is fulfilled later, after the sacrifice, whereas the positive commandment of “and he shall dwell there” is nullified the moment he sets out on the road. Furthermore, according to everyone, even if we say that he has no commandment to remain in the city of refuge, the religious court certainly has a positive commandment to exile him there (Maimonides), and the positive commandment of offering the Passover sacrifice incumbent on him does not override a positive commandment incumbent on the court — needless to say, they should not assign him protection and thereby nullify their own commandment.

These answers explain why they do not assign him protection in order to fulfill the commandment, something that was not explained in the answer above.

Roni,
on A, the Rabbi already answered, and you did not refute his words. And I would add in support of the Rabbi’s point that even someone who is on a distant journey would not be exempt, because he could leave for the pilgrimage before the time. On the other hand, that does not exempt him from the Passover sacrifice in the second month.

Regarding B — there is karet, my brother, for someone who does not offer the Passover sacrifice. Is anything more severe than karet?! Choose your words carefully, my brother.

Moshe answered well that he can leave in time. Anyone who lives far away is on a distant journey. And so too his comment about the second Passover.

The moment you turned this into a positive commandment, the discussion was over. I already wrote that my remarks were made on the assumption that there is no such commandment and no prohibition to leave except because of the danger.
However, I will now add that even if it is a commandment, in the case of Passover there is karet involved (and just as a positive commandment does not override a prohibition punishable by karet, it is reasonable that a positive commandment involving karet would override an ordinary positive commandment).

And also, the fact that the religious court has a commandment to exile him does not necessarily override his Passover obligation. Let them exile him before and after.

Indeed, the answers I wrote are limited and fit only some of the positions.
What I wrote here was from the starting point of the view that there is no obligation to bring oneself closer and get out of the category of “a distant journey” (Tzelach). For one who disagrees with him, this point indeed does not stand.

Indeed, a positive commandment that carries karet is more severe and overrides an ordinary positive commandment, but as is well known, there is a dispute in Tosafot on Pesachim whether this applies when they do not occur simultaneously (so at least according to some opinions, this is resolved).

“And also, the fact that the religious court has a commandment to exile him does not necessarily override his Passover obligation. Let them exile him before and after” — true, not necessarily (only perhaps), but it seems to me that any answer to the question of why the religious court does not appoint guards for him so that he can go offer it and return will be in the realm of possibility, not necessity.

May you be sealed for a good year,

There is an interesting point here — I did not understand where you got this whole issue of protecting him when he goes out for the sake of a commandment carrying karet. If there were a fire there, or if that city were conquered by enemies, then in such a case too, if he wanted to flee, would it be easier for the blood-avenger to catch him? Or is he forbidden to leave because it says “he shall flee there” and “there shall be his home”?! By the same token, I could mention that if a fire broke out in the city, would he continue staying there, or would he flee the city as fast as possible?!

And for your consideration, there is the law of a pursuer, so who says that the accidental killer cannot himself pay for protection, and who says he cannot make do with his own weapon? After all, if the blood-avenger pursues him, he is permitted to kill him in self-defense, is he not?

Before a detailed summary: does everyone here agree that he goes out to offer the Passover sacrifice?

Moshe, it seems to me agreed and clear that he does not go out to offer the Passover sacrifice (and of course is not liable to karet for that). The discussion was only about the reason for this (distant journey / an explicit positive commandment / the court’s obligation to exile him / danger to life).

And as for your question: if there is a fire there consuming the city, reason dictates that it ceases to be a city of refuge, since it is no longer a city and no longer a refuge, nor is it possible to fulfill there “and he shall dwell there until the death of the High Priest.” Therefore, the religious court is obligated to exile him to another city and protect him during that time, and to fulfill: “and the congregation shall save the killer.”