The Frasers first arrived in Scotland from France in the 12th century, or possibly even earlier, and settled in East Lothian then expanded into Tweeddale. The first Fraser in Tweedsmuir was probably Oliver Fraser, the “Olifurd” who witnessed documents in the reign of Malcolm IV (circa 1153-1165). He built Oliver Castle in Tweedsmuir. It is intimated that Olifurd gave his name to Oliver Castle but I believe the name, a corruption of Holyford, already existed in Tweedsmuir for some time and that he took this name Olifurd, which eventually beacame Oliver, from there for himself. Holyford would be the name of the ford across the river Tweed adjacent to the site of the present day Tweedsmuir Kirk.
The Hay family that is best remembered and recorded in Tweedsmuir is that of Hay of Yester. Gilbert Hay had married Mary Fraser who was the joint heiress of Simon Fraser of Oliver Castle - see Fraser of Oliver page. Gilbert inherited half the lands of the Barony of Oliver that were duly added to the already extensive lands held by the Hays of Yester.
The South of Scotland has a habit of producing great national leaders. Wallace was born in Ayrshire (possibly Renfrewshire), The Bruce at Turnberry (possibly Lochmaben) John Knox at Morham (possibly Haddington) but David Welsh was definitely born in Tweedsmuir. I can hear you saying now, “I know the first three but who is Welsh?”