Eastbourne seafront on a sunny day – palm trees, blue water and a whitewashed, gold-domed pier. The sea breeze brings a honey breath of broom flowers, yellow as highlighter pens, and the plangent sound of seagulls. The kids are cheering up after delayed trains and overcrowded London tubes. I’m hoping for a weekend of art and sea views.

My last trip to Sussex was in February, when I watched a murmuration of starlings swirling over Eastbourne pier at sunset. The area in front of the Congress Theatre’s new glass-fronted reception was then a building site. Now there are pristine paths and plazas with neat, purple banks of Russian sage and lavender, like an architect’s drawing come to life. It’s all part of the revamped Devonshire Quarter, a £54m upgrade to attract conferences.

Next door, Eastbourne’s free Towner gallery, has been in its current building for 10 years. To celebrate, the sinuously geometric chalk cliff of its facade has exploded into prismatic technicolour with a huge, jubilant mural by German artist Lothar Götz. The museum is part of a Coastal Culture Trail, linking three modern art galleries along 18 varied miles of East Sussex shore.

There’s a mind-expanding view of the new mural from the Devonshire Park Hotel, where we’re staying, two minutes from the sea. We stroll on the shingle past the moated Wish Tower, one of the Martello forts built as defences against Napoleon. A new seafront cafe is opening by the tower later this year. For now, we walk on in search of dinner, through fragrant, tamarisk-fronded seaside gardens. Of the “kids” (a teenager and a 21-year-old), one’s vegan and one’s fussy, but the Tuk Tuk restaurant serves Indian-style street food, including pani puri with chickpeas (£4.50) that hits the spot.

Next day, we head for the 1930s De la Warr pavilion (also free) in Bexhill-on-Sea. Designed by Erich Mendelsohn and Serge Chermayeff, its modernist escarpment of steel and concrete is a magnet for architecture fans. We arrive as the summer exhibitions open, including cartoon-like works by the Chicago Imagists and “a space for learning and unlearning”, where the kids play a cheerful game of table tennis. The balcony cafe alone is worth the trip – shining ocean, salty breezes and a vegan slate with beetroot falafel (£14).

You can travel from Eastbourne via Bexhill to Hastings by train, bus, bike or on foot. With showers forecast, we opt for the 99 bus (adult day rider £7.30), a cheap and leisurely way of connecting the three galleries. From the window, there are glimpses of ruined Pevensey castle across the marshy levels.

The weather clears for a hazy afternoon in Hastings. The trail’s third gallery first opened in 2012 and reopened this month as the newly independent Hastings Contemporary, with a bigger programme of exhibitions, including international artists (£9, under-16s free). Its glazed black bricks reflect the nearby boats and rows of tall wooden net shops. Inside, the cool white rooms host colourful exhibitions and a playful new series of sketches by gallery patron Quentin Blake. An impasto beach scene and row of blue boat pictures by Copenhagen-based artist Tal R echo the fishing fleet and pebbly strand outside.

Nearby are a pair of wonderfully ramshackle free museums, looking as though the tide has washed them up. The Fishermen’s Museum has sawfish blades and scallop dredgers, a crumpled doodlebug bomb and a suit sewn with silver winkles, all beached in a former church round a huge sailing lugger visitors can climb on. Next door, the larger Shipwreck Museum is a salvaged trove of rusted muskets and barnacle-crusted bottles.

The East Hill funicular railway opposite (adult £2.50) carries us up the steep cliffs to the panoramic Country Park, where we climb over a gorse-grown iron age hillfort and, eventually, down the palm-flanked steps of Old Humphrey Avenue to the half-timbered pubs and cottages of All Saints Street. It’s easy to see why Hastings old town was voted Britain’s best walking neighbourhood by the Ramblers in 2018.

A nap on the bus back to Eastbourne revives me for the Towner summer party. In the Eric Ravilious room (the gallery has lots of works by Ravilious, who lived locally) a watercolour of the winding river at Cuckmere Haven hangs next to a newly acquired one of Beachy Head lighthouse.

We explore these South Downs scenes on the way home to Essex the next day. We start with an open-topped bus ride from Eastbourne (adult day ticket £10) through fields of ox-eye daisies to Birling Gap. The on-board commentary tells us about wartime code-breaking and shipwrecked sailors escaping through chalk-cut tunnels in the cliff.

One advantage of car-free travel is that, so long as you pack light, you can set off on a linear adventure without worrying about where you parked. This theory is tested to its limits as we climb the wild-thyme-carpeted cliffs of the Seven Sisters. The kids manage the trek without puffing, even though they insisted on bringing big bags and laptops.

Heading for Brighton, we turn inland at Cuckmere Haven and follow the meandering river towards the red-roofed barns of Exceat village, with its Saltmarsh cafe and frequent buses. Behind us, the sound of surf on pebbles fades slowly into the distance.

• Train travel was provided by Southern (London Victoria to Eastbourne from £5 one-way). Accommodation was provided by the Devonshire Park Hotel (doubles from £110 B&B). For car-free directions to all three galleries see goodjourney.org.uk

Looking for a holiday with a difference? Browse Guardian Holidays to see a range of fantastic trips