The Butty Bus in Leverburgh, on the isle of Harris, serves a welcome egg and chips. There’s only space inside the bus for half a dozen customers and in summer the conversation swiftly turns to St Kilda, the remote group of islands 50 miles to the west. This harbour town is one of the main points from which to catch a boat for a day trip to St Kilda, but Hebridean weather being unpredictable and often stormy, boats can be cancelled, and visitors end up kicking their heels in Leverburgh.

Still, that’s not such a hardship with the exquisite 15th-century St Clement’s church and its carvings a few miles down the coast at Rodel. Or you can head north along the west coast’s stunning beaches and sip a cappuccino in the Temple Café as you watch eagles soar overhead.

I’d booked two trips, to be safe: one was cancelled but the other went ahead in choppy water and driving rain, a total of five hours, there and back, of exhilarating Atlantic. Most of my fellow passengers turned a ghastly shade of grey.

None of the extensive photography and filming of St Kilda can quite capture the drama of its topography. The group of islands and stacks includes some of the highest cliffs in the British Isles and is home to one of the biggest seabird colonies. Circling round the stacks, I craned my neck to look up at the sheer slabs of rock which towered over our boat with the screech of guillemots and gannets, the smell of ammonia and salt, and the roar of the surf. It’s astonishing to imagine that the St Kildans harvested these cliffs for the birds and eggs on which they survived.

The last inhabitants left in 1930, evacuated at their request, bringing to an end a remarkable story of human resilience. It’s the appreciation of this as much as the dramatic landscape which makes visiting St Kilda, a Unesco World Heritage Site, so memorable. I wandered through the poignant ruined cottages on the main island of Hiort, visited the old schoolroom and then headed for the main peak of Conachair, and along the cliffs. It was tough walking, but stunning even in a thick drizzle.

Back on Lewis, I came across one of the most inspiring and beautiful coastlines of all the islands I have visited in the Outer Hebrides. An hour or so by car from Stornoway I reached Uig, a sweep of golden sand with the mountains of Harris in the distance. If you continue along the single track road to Mangursta and Mealasta, you will stumble across a series of smaller bays. The machair [low-lying grassy plain] here is perfect for wild camping and we breakfasted on the cliffs in brilliant sunshine, watching gannets dive dramatically into the sea.

Jagged stacks rear spectacularly out of the Atlantic along the battered coastline. This dramatic spot on Lewis has been chosen as a site for the ambitious proposal to build a St Kilda Centre, dedicated to its history, wildlife and geology. It’s an unlikely location – building on St Kilda isn’t possible as bad weather would restrict access – so this would offer the best chance of seeing the islands, 44 miles to the west.

On my way home, I stopped by Uig Lodge, with its stunning view of the bay, for the essential souvenir to bring home: the most delicate smoked salmon, guaranteed to bring back memories of the bracing salty breeze and sound of Atlantic surf as it hits the palate.

Essentials

Trips to St Kilda are available from Harris on kildacruises.co.uk and from Lewis on Seatrek (seatrek.co.uk)

Love of Country: A Hebridean Journey by Madeleine Bunting is published by Granta on 6 October at £18.99. Order a copy for £15.57 at bookshop.theguardian.com