It would be all too easy for Collioure to lose its identity. After all, it's a Mediterranean fishing village, a sort of cliché-on-sea by the Spanish border. It's true that the only trawlers moored at the quay nowadays are brightly painted museum pieces. But rather like one of the anchovies that used to be fished here, the village has a strong flavour not lost in the smothering salade Niçoise of tourism.
The main reason for this is that the first thing you see on the walled harbour is a ridiculously phallic pink church tower. I tried to watercolour it once and ended up hiding my sketchpad from passers-by. Not everyone has been so shy. Collioure was the birthplace of Fauvism, created when Matisse and Derain arrived here in 1905, having apparently forgotten their black paint and produced 242 pictures, making Collioure's church one of the most painted locations in France, on a par with the Moulin Rouge, Monet's lilies and Madame Renoir's thighs.
Today, the hilly streets are dotted with reproductions of Fauvist canvases in situ, and it's comforting to see that, apart from the fact that the sky isn't bright red, little has changed. The houses nearest the harbour are now restaurants and knick-knack shops, but wander up the hill and you're among lemon trees, stone cottages and grapevines. One welcome addition is a commando base. This might not sound attractive, but these are French commandoes, who behave more like a synchronised swimming team. They stroll through the village in their tight swimwear and go snorkelling in the bay, no doubt on alert for the day when France needs to repel an invasion by sea bass.
Undersea action is my main reason for coming back to Collioure, usually in spring or autumn when there are fewer crowds. Wade in from the pebbly beach and your mask reveals a veritable bouillabaisse of sealife. Edible-sized mullet, sea bream and various stripey local characters cruise around you as if you were one of them. I've even seen octopuses lurking among the rocks.
You do feel a twinge of guilt when you sit at a quayside restaurant and order local daurade (sea bream). But who wouldn't feel guilty at the sheer hedonism of lounging in the sun, drinking a perfect rosé bottled metres away, staring straight at the biggest phallic symbol on the French coast?
• Fly to Perpignan (half an hour from Collioure) from Stansted or Birmingham with Ryanair from £62 return, or take a train from London to Perpignan from £119.50 return through Rail Europe. Trains from Perpignan to Collioure cost from £3 return. Casa Pairal has doubles from €89, or Les Criques de Porteils campsite has pitches from €23.50 a day plus yurts and mobile homes for hire on a weekly basis.
Stephen Clarke's latest novel about life in France is The Merde Factor (Arrow, £7.99). To buy a copy for £6.39 with free UK p&p go to guardianbookshop.co.uk
