There is a narrative commonly accepted both by theologians and by popular authors, according to which limbo is a hypothesis first invented by medieval theologians to reconcile the necessity of baptism for infants to attain grace and salvation with God's justice that does not punish people with the pains of hell except for their actual sins.
This narrative, however, seems to have a very strong Western bias and to inaccurately reflect the history. In the West, under the influence of Augustine, up until Abelard it was commonly held that unbaptized infants are punished in hell through with a milder punishment than those who committed actual sin.
But before Augustine (Tertullian being an exception), the view that unbaptized were punished with pain in hell on account of Adam's sin which passed down to them was not the common view. St. Ambrose says that the hereditary sin of Adam "cannot be a terror to me, since in the day of judgment we are not punished for another's sins, but for our own," and that whereas baptism takes away personal sins, the rite of washing the feet [a local custom] takes away hereditary sins.
When the question comes up, the Eastern Fathers do not generally allow that unbaptized infants will be positively punished in hell for original sin:
[Those who are not able to receive baptism], perhaps on account of infancy, or some perfectly involuntary circumstance through which they are prevented from receiving it, even if they wish… will be neither glorified nor punished by the righteous Judge, as unsealed [by baptism] and yet not wicked, but persons who have suffered rather than done wrong. For not every one who is not bad enough to be punished is good enough to be honored; just as not every one who is not good enough to be honored is bad enough to be punished. (St. Gregory Nazianzen, Oration 40)
The innocent babe has no such plague [of ignorance] before its soul’s eyes obscuring its measure of light, and so it continues to exist in that natural life; it does not need the soundness which comes from purgation, because it never admitted the plague into its soul at all… But the soul that has never felt the taste of virtue, while it may indeed remain perfectly free from the sufferings which flow from wickedness having never caught the disease of evil at all, does nevertheless in the first instances partake only so far in that life beyond (which consists, according to our previous definition, in the knowing and being in God) as this nursling can receive (Gregory of Nyssa, On Infants' Early Deaths).
Where do children of Jews or the unbaptized go who die lacking wickedness, five years old, or four years old? To damnation, or to Paradise?
Response: Since God has himself pronounced that the sins the of the fathers do not pass to the sons, and said through the prophets that children shall not perish for the sins of their fathers, it seems to me that they do not go into Gehenna. But it is not fitting to probe the judgments of God with one's hands. (Anastasius of Sinai, Quaestiones, q. 81, PG 90:709c)
What St. Gregory of Nazianzen says is, in fact, exactly what is later expressed by the term "limbo."
(Quaestiones, q. 81, PG 90:709c)