Hello! You want bananas?” The Surenthirans, on a family outing to the Hindu temple, make a colourful group on the sandy road: two bicycles, three adults, four children and one bag of bananas. Dad Harshiv is shirtless, as men must be to enter a temple, above a saffron sarong, and the rest are in shades of orange, indigo, turquoise.

We’re still full from lunch, but it turns out bananas are a pretext. The Tamil family want to meet a pair of westerners just as much as we want to chat to them. Bright-eyed and keen to practise her English, Shika, 11, is a “tsunami baby” – she was only one when the wave hit this low-lying area south of Trincomalee on Sri Lanka’s east coast. More than 500 people around here died as the wave plunged up to 2km inland. Shika and her mother were rescued by boat. Kogila, Harshiv’s sister-in-law, tells us her brother was killed: “Every family knew someone who died.”

We chat, swap addresses and, with promises of photos and postcards, wave the Surenthirans goodbye as they turn off towards their new-build home in Kalkudah village.

The 2004 tsunami caused death and misery across a huge area, but for Kalkudah and nearby Passikudah, it heaped insult on injury already caused by civil war. In the 1970s, this was a prosperous community, with modest hotels thriving on twin white beaches. But as fighting escalated, visitors stopped coming and the hotels were first abandoned, then blown up by the Tamil Tigers (LTTE), so the Sri Lankan army couldn’t use them.

Now, five years after the conflict ended, this is a tourist development zone, and Passikudah in particular bristles with luxury resorts. How much this will benefit families like Harshiv’s is doubtful, though: there are reports of fishing boats being ordered off more desirable stretches of sand.

At the start of our visit to Sri Lanka’s north and east – arranged by UK firm Bamboo Travel – it was war rather than tsunami that occupied our thoughts. Sri Lanka’s iconic A9 road closed in 1984, and was finally demined, rebuilt and reopened in 2013, linking Jaffna in the north with Kandy and all parts south. (A new train service also started last October from the capital, Colombo, to Jaffna.)

There’s still an army checkpoint in the former frontline village of Omanthai, where we had to show our Ministry of Defence permission to visit the north. The smiling officials take little notice of me – just a European tourist – but show a keen interest in my old uni mate Kalpana, who is British Indian, ushering her off to sign a paper promising, presumably, not to run arms to Tiger cells.

Sri Lanka generally doesn’t have India’s crowds, but north of Omanthai it’s even quieter. The area remains underpopulated, and car ownership is low, so the A9 is mostly empty for the last 100km, running north through a landscape dotted with sugar palms. The writing on signs and hoardings changes from curly Sinhalese to angular Tamil.

The Sri Lankan army and police are still very visible, and the road is lined with war memorials – history written, as ever, by the victors. In Kilinochchi, Tamil Eelam’s putative capital, lie the remains of a huge water tank destroyed by the LTTE, with the slogan: “Say no to destruction ever again.” Just over Elephant Pass causeway is a memorial to the self-sacrifice of a Sri Lankan army officer. We also spot a burned-out school, but the rebuilt railway, its new stations painted peachy pink, speaks of a more hopeful future.

Jaffna, once Sri Lanka’s second city, is on the up again after emigration and expulsion saw its population almost halve. But it has a way to go. Its 17th-century Dutch-built fort is being repaired with help from the Netherlands, but the contrast with the fort in the southern town of Galle, which has gift shops and boutique hotels at every turn, is stark. Jaffna’s fort is still bare, the once-magnificent Dutch church lies in shards at crazy angles, and cows outnumber people on a new promenade around its seaward sides.

There is more activity at the nearby fish market and at the city’s pride and joy: its library. Once one of Asia’s biggest, Jaffna library was destroyed by a mob in 1981; now rebuilt, it is a harmonious place where ceiling fans whirr and polished wood glows in the sunlight as men of all ages peruse newspapers. There were no women there – a sign of the area’s conservatism.

Hotels in Jaffna are few – no one came for decades. At the Subhas (doubles from £25 per night, subhasgroup.com), where we stay, only one wing is so far refurbished, but the welcome is friendly, and we’re pampered with room-service breakfasts of dosai or warm roti, with vegetable curry, spicy sambol, fruit and tea. Jaffna is known for food: it’s the home of cheaper-than-chips (but much more delicious) kottu – chopped roti stir-fried with vegetables and a profusion of flavourings. I recognise chilli, ginger, tamarind and curry leaf, but half a dozen more dance on my delighted tongue. Memorable, too, are curries of creamy cashews and okra.

The 47km causeway across the 10-island archipelago west of Jaffna is still partly ruined, so catching an early ferry to the remotest of these sunny islands means an early start. At 6.30am, as we drive between lagoons across Mandaitivu and Velanai islands, fishermen are dragging nets on foot or snorkelling to little prawn farms in the shallow water. War closed both this causeway road, from Jaffna to Pungudutivu island, and the passenger ferry we take to Delft (an island colonised by Portuguese but renamed by Dutch settlers), so these islanders were particularly isolated.

Now they’re making the most of reconnection with the world: colourfully dressed men, women and children pour off the free ferry at Pungudutivu’s Karaikattuvan jetty, ready to hit Jaffna. And the tiny wooden boat fills up again with mostly Sri Lankans keen to visit Delft’s wild beaches, peaceful interior and atmospheric ruins.

We’re met after the hour’s crossing to Delft by Father David, a leading light in attempts to develop the island ethically and sustainably. Most visitors pile into waiting tuk-tuks, but Father David has a rough-and-ready Jeep with benches in the back. The Jaffna-born Anglican priest runs a school and other projects on the island, with the aid of funding from overseas charities and his own tourism efforts.

There are no hotels on 8km by 6km Delft, but Father David offers a couple of simple en suite rooms in his home for about £20 a day for two, including sumptuous meals – fresh fish is cheap and abundant here. He and his wife, Rostelin, are also planning a beach cottage, and the Jeep is on offer for touring the “sights”.

Only those seriously into colonial archeology will find the ruins exciting: a derelict Dutch-built hospital, a ruined courthouse where a large cow now presides. But it’s fun to step inside the trunk of a huge baobab tree brought from Africa as a sapling 400 years ago, and to spot the wild horses, descendants of animals also brought by the Portuguese, that roam the island. However, the joy of Delft is its other-worldly feel. Kalpana and I walk to sandy Manal Kinaru beach, five minutes from Father David’s house: warm waves break on a shore of wild scrub and sugar palms, a few birds hop about, and there’s not another person nor a building in sight.

Back in town we fancy a cold beer, but as conservative Jaffna doesn’t really do bars, we’re looking forward to a more relaxed, westernised feel on the next stage of our tour. We’re driving 200km south-east, towards Trincomalee, where luxury resorts are springing up on the once out-of-bounds coast.

One of these, Jungle Beach, is reached down a windy lane through, well, jungle. A high thatched walkway leads to an outdoor restaurant and – yes! – bar. There are pools both ornamental and for swimming, and detached cottages with sandy paths to the beach. Honeymooning couples can dine at tables by the waves.

I enjoy my beer and a rather un-Sri Lankan dinner of fat prawns and salad leaves. The staff are super-attentive and it’s all pretty gorgeous – if that’s what you want. But, like a pair of unsatisfied toddlers, we’re now missing the “real” Sri Lanka.

We get a glorious fix next day near Valaichenai, further south. Glorious because that is the name of the restaurant where we lunch on rice, dhal, omelette and vegetables (about £3 for three people). Our Sinhalese driver, Wije, is initially wary of the Tamil owner, but Jivan is welcoming and keen to talk. He lived in London for several years until after the war, and says life here is still a struggle for Tamils. They have to make the long trip to Colombo for permits or licences, and a discriminatory points system makes it more difficult for their children to gain a university education.

He’s pleased, though, with his restaurant – it’s been open just three days – and with his two chiller cabinets, which have a surprising provenance. “East Lane, Wembley,” he says. “I worked at the Jet petrol station. The garage closed, they gave me the fridges and I shipped them here.”

Our last stop is Uga Bay resort, run by the same group as Jungle Beach and with a deep arch framing a vista of lawns, infinity pool and ocean. Our room is sumptuous. We swim and are served tea and cake then, restless, walk along the beach into the village. We feel lucky and pampered, but also that something of Sri Lanka has been taken away.

Then a young voice pipes up: “Hello! You want bananas?”

Way to go

The trip was provided by Bamboo Travel (020 7720 9285, bambootravel.co.uk) which offers tailor-made tours of Sri Lanka’s north and east. Prices start from £2,195pp for a 15-day tour, including international flights, transfers, accommodation, a driver/guide and some meals. Flights were provided by Sri Lankan Airlines (srilankan.com), which flies from Heathrow to Colombo from £505 return

• This article was amended on 10 February 2015 because the main photograph was wrongly captioned as the Kandaswamy temple in Jaffna, not Trincomalee.