‘I’m not sure about this,” says the American woman, jammed into the jockey seat in front of me, as the rim of the boat, carrying 20 of us out into the Atlantic Ocean, slams down against the swell. “I think I’m a bit scared.”

She’s not the only one. A dolphin-spotting trip had seemed a wonderfully relaxing way to spend a morning out on the shimmering seas around Sagres, the Algarve’s westernmost town, believed for centuries to be the very edge of the world. Mar Ilimitado’s pre-trip briefing promised bottlenose dolphins, possibly minke whales and even hammerhead sharks, in conditions that were “unseasonably calm” for early autumn.

In spite of the still-warm sunshine, as we bounce out beyond the shelter of the cliffs and the swell starts to rise and fall in earnest, I wonder what “seasonably calm” might look like. Just as my stomach is starting to lurch a little, the spotter at the back of the boat lets out a shout. Almost immediately, a dolphin leaps from the water, executing a perfect arc before disappearing into the depths.

Everyone grabs their phones, then there’s another, and another, until the water ripples with fins, vague shapes whisking through the deep, emerging in sleek grey skins. It’s an incredibly joyful thing to watch, but Mar Ilimitado has strict rules on not spending more than 30 minutes with one pod. As we glide away, I think how lucky we were to be the only boat out at sea, rather than one of the many that head out each day from the bigger resorts of Carvoeiro and Albufeira.

Sagres itself is a slightly scrappy surfers’ town, wind-lashed and sunny right through autumn (an average of six hours a day in November), flanked by empty scrubland, soaring cliffs and the vast stretches of ocean. Under Moorish rule for 500 years, it’s from here that Henry the Navigator began the 15th-century expeditions that ushered in the Portuguese Golden Age of Exploration, the unassuming town a gateway to a whole undiscovered world.

And yet Sagres wears its history lightly. The huge fortress on the top of the cliffs offers nothing more than unremarkable walls and vast blue skies, although the lack of information boards and tour groups somehow adds to the end-of-the-world feel.

“People come to the Algarve expecting huge resorts and endless apartment blocks, but because the western end is a national park, there’s much less development,” says our guide, Paulo, over a lunch of garlicky mussels and salty local cheese with slabs of sourdough-like bread. “Surfers come, and hikers – Sagres is on the Fisherman’s Trail [a 140-mile coastal path from São Torpes in the Alentejo to Lagos on the Algarve] – but the whole area is quite under the radar.”

In late September, we’re on the cusp of the seasonal transition from sunkissed beach breaks to blustery walking weeks, but the sun is still warm enough for an hour or two on Burgau beach, a great swathe of golden sand, stretching out beneath a cluster of low-slung, whitewashed houses, quiet bars and narrow streets, woven into the hillside. We spot walkers, following the steep Fisherman’s Trail to the next beach at Salema, but can’t quite persuade ourselves off our loungers to join them.

Walking is an increasingly popular way to discover this stretch of coast and its hinterland, and this year the tourist board launched a website, algarvewalkingseason.com, with details of walking festivals throughout the winter. We decide to tackle the Seven Hanging Valleys Trail – one of the Algarve’s most popular – taking in Benagil beach with its spectacular sinkhole, and Praia da Marinha, famous for its dramatic grottoes and arches.

Only four miles each way, it proves pretty taxing and we turn back halfway, although the views are more than worth it. In summer, the beaches below teem with people, with car parks full by early morning and every patch of sand covered. In autumn, the route is still popular, but the crowds are fewer, with the scenery even more beautiful in the low, late sun.

But this area of the Algarve is about more than just its coast. Our villa, the elegant Casa Pardal, is set in the hills 10 minutes north of Portimão, a peaceful oasis surrounded by landscaped gardens, elegantly furnished with North African lanterns and carved wooden panels that are a legacy of the area’s Moorish past. From here, it’s an easy drive up to the mountain village of Monchique, said to be the greenest in the Algarve, where the narrow streets are dotted with art galleries and gift shops, and walking groups appear out of sidestreets clutching poles and empty water bottles.

There are times when we glimpse the other, better-known Algarve, like lunchtime in the picturesque village of Ferragudo, where every restaurant table seems to be taken by British retirees, flocked from their apartment blocks in Praia da Rocha. But mostly we avoid it, driving down to Portimão in the evenings to eat crispy-skinned bream at Taberna da Maré in the old fish market, and ambling around the pretty streets of Silves, once the capital of the Algarve, now a picturesque town dominated by the huge castle, one of most complete Moorish fortifications in the country.

None of it is quite what I expected; less developed, more authentic, a tourist region that has retained far more of its sense of self than parts of the neighbouring Spanish coastline. It’s my first time to the Algarve, but definitely not my last – although next time I’ll pack walking boots rather than swimwear – and I won’t let the Seven Hanging Valleys Trail beat me again.

Casa Pardal, sleeps eight and costs from £2,007 for seven nights (oliverstravels.com). For guided tours and hiking, faroldiscover.pt. A two-hour whale-watching trip costs €40pp (marilimitado.com). More information at visitalgarve.pt