Folkestone’s first Triennial, in 2008 hinged on the appeal of public art in locations visitors might not otherwise have explored. So the event, brainchild of local millionaire Roger de Haan, saw Tracey Emin’s tiny bronze castings of baby clothes scattered across town from railing to kerb, while Nathan Coley’s illuminated text, Heaven Is A Place Where Nothing Ever Happens, lit up scruffy Tontine Street.

It was hailed as a success, and highlighted what a scenic – if a little fatigued – place the town was. By the start of the second show in 2011, however, developments were really afoot: the HS1 high-speed rail link had arrived, shortening the journey time from London from 90 to 55 minutes; Mark Sargeant’s harbour restaurant, Rocksalt, had opened, giving tourists an upmarket dining option; and a new selection of works included Cornelia Parker’s life-size Folkestone Mermaid, at the tip of the Stade, the cobbled fishermen’s quarter.

So, with the third Triennial opening, how is Folkestone doing in 2014? Some might say the town’s fortunes rely too heavily on art, or on the new cafes, shops and restaurants on its hilly streets, but there are also significant civic developments this year. Most prominent is the reinvention by Muf Architects of the long-dilapidated Payers Park, a steeply sloping site in the heart of the Creative Quarter.

And overlooking the sea at the top of the Road of Remembrance, near Mark Wallinger’s powerful memorial Folk Stones (from the first Triennial), a shiny new centenary arch was unveiled earlier this month. Commemorating the millions who passed through the town on the way to the Western Front during the first world war, it’s an ultra-modern steel structure, balancing Folkestone’s past while keeping its eyes firmly on the future. It’s already wildly popular.

What to see at the Triennial

Wear comfy footwear, because seeing the 21 artworks, which include pieces by Yoko Ono, requires several miles of walking. The official route (which also lists works in situ from earlier Triennials) begins at the station: here the railway bridge has been transformed by local artists group Strange Cargo, with four colourful figures and inspiring plaques adorning the brickwork.

Other highlights? Jyll Bradley’s eerie Green/Light sculpture in the nearby disused gasworks; Emma Hart’s Lookout installation on two floors overlooking Tontine Street; Argentine artist Pablo Bronstein’s beach hut in the style of Nicholas Hawksmoor, with viewing point; and a polytunnel on the roof of a sixth-form college, repurposed by architects Something & Son. Climb the Zig Zag Path from the promenade to the Vinery, a walkway now restored to its 1930s heyday, to see Will Kwan’s chinoiserie screen.

Remember to look up. Detroit-based duo roofoftwo’s witty Whithervanes are five headless chicken sculptures on rooftops: responsive to Twitter, they will rotate to the level of “fear” generated by social media.

Most atmospheric is the former harbour railway station itself, where Paloma Weisz’s Rug People (from the last Triennial) still languishes on the disused tracks. It’s here that Tim Etchells’ mesmerising new neon work, Is Why The Place, lights up forlorn walls.

Where to eat

El Cortador (elcortadorfolkestone.com, 41 Old High Street) offers well-priced tapas (from £2 each), including creamy fish croquettes, and a generous plate of serrano ham and manchego cheese for just £4.50. Elsewhere in the Creative Quarter, opt for a contemporary meal at the Ampersand Design Cafe (ampersanddesigncafe.co.uk, 46 Tontine Street), Italian at Bloom’s (2-4 Old High Street) or veggie at cosy Beano’s Cafe (43 Old High Street).

Need sustenance on the Leas seafront promenade? Sisters Abi and Aimi Lauder, new arrivals from south London, have just opened chic bistro-cum-vintage shop Follies (133 Sandgate Road), with charcuterie and Lebanese lamb on the menu.

For fine dining, the aforementioned Rocksalt (1 Back Street, rocksaltfolkestone.co.uk) offers commanding harbour views, a lovely terrace and imaginative fish dishes. If the restaurant is full, try the upstairs bar for cute bowls of cockles or dressed crab. Next door, the Smokehouse (thesmokehousefolkestone.co.uk, 1 Back St) fries good fish and chips; cheaper options are seafood huts such as Bob’s on the Stade.

Cool down with La Casa del Bello Gelato (12 Cheriton Place), which has been churning out artisan ice-cream in flavours such as orange and carrot and panna cotta for a dozen years.

Where to drink

First, a coffee. Googies (googies.co.uk, 15 Rendezvous Street) is a cafe-cum-cocktail bar with arty vibe and first-rate beans. Its gourmet burgers and espresso martinis are pretty good, too. But to taste coffee from a real brewing master, Thomas Burrows at Manifest (46 Old High Street) is unbeatable: his smooth filter is made using Square Mile beans, and even tastier are his coffee colas, which he concocts using all manner of botanicals.

Fancy something a little stronger? Try the two micropubs: The Firkin, a former hairdresser’s (firkinalehouse.co.uk, 18 Cheriton Place), or new arrival Kipp’s (11 Old High Street), which serves potent ciders from Canterbury along with gentler ales. Nearby, you’ll find the town’s oldest boozer, the British Lion (8 The Bayle), which dates back to the 15th century, while The Pullman (9 Church Street) has just reopened with a stylish, candle-lit interior.

What else to do

There are three musts: historical quarter The Bayle, Sunny Sands, the beach at the end of the Stade, and the Lower Leas Coastal Park. Created in the 1870s, the park has been restored to its Victorian glory, with pine trees, flowering herbs, play areas, and views across the shingle beach to the harbour. Head back up to the Leas promenade via the Grade II-listed water-powered lift. The Creative Quarter is packed with tiny shops selling jewellery, homewares, contemporary art, mosaics, antiques and vintage finds. For entertainment try the Leas Cliff Hall (leascliffhall.co.uk, The Leas) Quarterhouse theatre (quarterhouse.co.uk, 49 Tontine Street) and the independent Silver Screen cinema in the Town Hall (http://folkestone.silverscreencinemas.com/) on Guildhall Street.

Where to stay

Rocksalt has four compact rooms (from £85 a night), while the boutique Hotel Relish is just behind the Leas (hotelrelish.co.uk, from £98, Augusta Gardens), with the sea-facing Sandgate Hotel a mile away (sandgatehotel.com, rooms from £50). Newest arrival is 10-12 Folkestone (10to12folkestone.co.uk, 10-12 Langhorne Gardens), a contemporary B&B near the Leas with rooms from £100 a night.

• Southeastern (southeasternrailway.co.uk) trains to Folkestone Central from London St Pancras take 55 minutes, returns from £37. The Triennial folkestonetriennial.org.uk runs until 2 November