Some people love borders. They collect the passport stamps. But equally thrilling, I reckon, is the border crossing that has gone. Years, perhaps centuries, of unbearable tension and violence, all the searches and passport checks, suddenly dissolved by a flood of goodwill. It doesn't always last, but there is something subtle and magical about those borders that have become invisible.

We are walking near the United Kingdom's only international land border, up Slieve Foy, the 588m mountain that overlooks Carlingford Lough in Ireland's County Louth. Local walking guides Peadar and Aude Laffon are pointing out the landmarks to my partner Sophie and me.

"There's Warrenpoint," says Peadar. "The IRA used to shoot across the water at the British army vehicles." The 10-mile-long sea lough is narrower there. Peadar's arm moves around, towards the south. "The big mountain back there is Slieve Gullion. That's in South Armagh."

We are on the Irish east coast, just south of the border, a border that still exists of course, and yet the threatening mists of violence have evaporated in recent years, leaving a beautiful and dramatic landscape: green-shouldered hills crested with crags, shading down into gentle valleys. It's a place washed with the light of the Irish Sea – little wonder the Celts called it after their sun god, Lu, hence Louth.

This is a land that has been soaking up human dramas for a long time. Every peak and valley carries a weight of song and story, a weight that can slowly move mountains, given sufficient time. Peadar points over the summit of Slieve Foy and down the eastern slopes to a strange gap in the mountain. "That was formed by Queen Maeve's army."

We sit on some rocks. There is always that tale-telling moment in Ireland, and you just know it's time to listen. "She'd come over from Connaught to try and capture the Brown Bull of Cooley on what became known as the Great Cattle Raid – the Táin." We were standing on the Cooley – a rounded 15km plug of Irish turf and granite – and the Táin is arguably the definitive Irish legend, a pagan odyssey of greed and self-sacrifice that has dug its narrative into the nap of the landscape. Between Sligo over to the west and the Cooley, almost every hill and dale carries the story forward a little more.

"But Cúchullainn, the Irish hero," said Aude, picking up the story, "he fought Maeve's army – alone of course, because the men of Ulster were struck down with birth pangs."

Aha! A side story ... and then we are into that explanation, which reminds Aude of another story, and another, and the Táin gets forgotten for now.

In England our Celtic roots were mostly extirpated by the Romans, but here they come looming up from the landscape. We walk on, over the summit of the mountain, and down to a sunny belvedere that looks north to the stark beauty of the mountains of Mourne. And Aude and Peadar keep the tales turning, adding more details and diversions as new vistas trigger new possibilities. These old legends are meant to be told on the hoof, I'm sure, keeping the children's feet marching on ancient seasonal migrations – rambling epics, quite literally, that were trodden into the scenery like wayward seeds, sprouting new growth all over the place.

Back down in Carlingford after our walk, we set up camp in Taaffe's Bar, gazing into the swirling dark magic of Guinness. A duo from Armagh are singing the landscape to life. Mountains of Mourne, Kilkenny, Galway bay, Fields of Athenry. Carlingford itself doesn't get a look-in. This was always a place that stood outside the Irish musical tradition, a Viking settlement that became an Anglo-Norman trading post in a sometimes hostile land, as remote in its way as any Hudson bay fur-trapper outpost. The people built a big wall around it and fortified their houses. King John visited in 1210 and the castle, built 20 years earlier, took his name.

All the ruins are still there, tucked into the huddle of houses: Taaffe's Bar is partly built into one of them. Down the street is another well-known bar – PJ O'Hare's – where the band is whipping up a storm and the girls are dancing on the tables. It's fun, but I want tradition – fiddles and bodhrán drums – so we head back to our hotel and ask owner Paul Carroll, a man who appears to know everyone and everything about the Cooley.

"You could try Lily Finnegan's," he says, and then generously drives us out to what is the quintessential Irish boozer, with a Saturday night traditional music session. The pub stands alone in the middle of fields, but the craic draws people in from surrounding villages.

Paul tells us how he gave up a career as a London photographer to come back and run the family hotel, Ghan House. Now he's a central personality of Carlingford life, known to everyone. He introduces me to Imelda O'Loan, a local historian, who explains how this area was already a borderland 2,200 years ago when the Celts moved in. "There are sacred places here that go back much further than that, such as Faughart – the whole history of belief is in that place. It's more ancient than Newgrange, the most famous megalithic site in Ireland – and that's more than 5,000 years old."

Taking Imelda's lead, we head off next day to take a look at this marvel. At first, however, we are a bit non-plussed. There is a small hill with a few overgrown ruins and gravestones, nothing at all on the grand scale we had expected. But then we find the well and follow the stream down to the healing stones below.

A few people are performing pilgrimages, praying for the intercession of Saint Brigid. If Christianity is here, however, it is a thin veneer over deeply pagan practices. Brigid herself was a Celtic goddess of fire who has travelled far and wide from her Eurasian roots – even turning up, so they say, in Haitian voodoo ceremonies.

Wandering around this strange place, we find various stones that reputedly heal various bodily woes: there is one for the knees, one for the head and so on. There is even a hoof stone, which Imelda had told me she once helped a farmer encourage a sick cow to put its foot into. I watch one lady laying her head on the relevant stone for a long time.

Around her, in the hedges and fences, are tied clouties or riddances – rags and ribbons in which the afflicted have left their woes. There are also worry beads, dolls, rush crosses, photographs, even a few footballer cards. Faughart may not have the imposing ruins expected in an Irish Delphi, but this place has a living atmosphere that the great oracle would envy, and its popularity is on the increase with all the recent troubles in the Irish Catholic church.

That afternoon we take some horses up on the southern slopes of Slieve Foy, riding with Niall Connolly, who runs an equestrian centre in Ravensdale, a few miles away. Every geographical feature seems to have resonant links with ancient heroes, but for Niall there are more recent associations. "When I was a boy the IRA men hatched a plot to steal a light aircraft and drop bombs on the British army barracks at Warrenpoint, over the lough." We are looking down at Niall's riding stables, once the family farm where he grew up.

"Their pilot was an old man, an ex-first world war pilot, but when the plane ran out of petrol, they landed there – in our field. And that man carefully carried all the bombs to safety – it saved our place."

"Another Irish hero?"

Niall chuckles. "Well, actually he was an Englishman who had settled here and joined the IRA." He turns in the saddle and points back at Slieve Foy. "Now that is an Irish hero. They say the mountain is the sleeping giant Finn McCool, who gained his amazing powers after eating a piece of flesh from the salmon of knowledge."

I nod. The salmon of knowledge?

"Unfortunately," continues Niall, "the powers did him no good when he ran into the witch of Beara on top of Slieve Gullion . . ."

But there we shall have to leave it, because that is another story.