How Kinect tech and two watermelons could make Australia’s trains run on time

In Japan they called them shiri oshi or ‘tushy pushers’, the white gloved railway platform attendants that with some strategic jostling and a well-timed shove, packed commuters onto crowded trains at speed.

They have, in the most part, now been replaced with platform screen doors, at which commuters queue ready to board.

Sliding doors are also in place at some London Underground stations. At others passengers are guided by floor markings which show where the train doors will be, with zones to keep clear so passengers have space to disembark.

(Some London commuters were angered by the floor paint when it was introduced, saying it took away their "competitive advantage".)

During peak hour at Town Hall and Wynyard stations in Sydney, marshals are employed as part of the ‘Fast Track Platform’ program to serve a similar purpose.

The aim in each case is to minimise what is known as the train’s ‘dwell time’ – the time it needs to stops at each station to allow passengers to board and alight.

Led by Dr Alen Alempijevic, researchers from the Centre for Autonomous Systems at UTS have collaborated with railway engineering group Downer Rail to come up with a more effective solution: far cheaper than installing sliding platform doors; more polite than a “step lively”; and less physical than a shove in the back.

Being put into a limited production run after pilots at a number of Australian stations, DwellTrack could even help you find a seat in peak hour. At its heart is the same technology that’s found in an Xbox 360 Kinect — and two watermelons.

Dwell time is the biggest variable affecting whether a commuter train is able to stay on schedule.

“When a train operator does their timetabling, they look at the time between trains and then they also have an allowance for how long the train can be at the platform, before it starts to delay the train behind it. That dwell time, networks have become more sensitive to, as more trains run through,” explains Stuart Warren, of UTS’ Rapido group which helps link industry to the university’s research.

The period cannot always be reliably predicted and the task is made even harder with the rising number of Australians using trains: in Sydney patronage is up 20 per cent in the past year alone.

Crowding at one end of a platform or at the bottom of stairs can create choke points, meaning more time is needed for riders to push their way through the throng to get on and off. A few seconds added to the dwell time might not seem very significant, but the knock on effects can be severe. The longer the dwell time at one station, the more commuters accumulate at the next station, adding to the dwell time there and so on.

Before too long, timetables are put out of whack causing more overcrowding, irate commuters and travel chaos.

“We saw what happened back in January when they introduced the new timetable. Fantastic that they put in loads more trains, but they become much more sensitive to dwell time, because the Sydney network is so entwined,” Warren added.

Watermelon heads

Demonstrated at the CeBIT technology event in Sydney last week, the DwellTrack system utilises a number of Ethernet connected depth cameras set-up along the length of a station platform.

Depth cameras – which are found in the now-defunct Kinect – work by firing an irregular pattern of dots from an infrared projector at a scene. An infrared camera then captures the pattern and sends it to a processor to work out depth from the displacement of the dots – which are more spread out close to the camera, and denser on objects further away.

Colour camera and depth camera footage

The system then applies computer vision techniques in order to identify people in the footage. In order to track them, it constructs a ‘Head-to-Shoulder Signature’, which UTS researcher affectionately refers to as ‘the two watermelons’.

“The technique works by fitting ellipsoids to 3D points, one to the head and one to the shoulders. It’s a way of going from a cluster of 3D points [to] trying to interpret that as a person, by fitting these two ellipsoids,” says UTS’s Alexander Virgona, whose research formed the genesis of DwellTrack.

“The orientation of the shoulder ellipsoid can show which way the person is facing. That’s relevant to our tracking problem, because people don’t move sideways typically,” he added.

As people are tracked along the platform, the system can detect if certain areas are becoming crowded. This information can be used to inform the platform marshals of where they need to move to, or rigged to green, amber, red lights so commuters arriving on the platform know where not to wait.

The DwellTrack dashboard

With Sydney Train’s last week making available feeds of carriage weight – indicative of how full a carriage is – via API, DwellTrack could work to move people on the platform away from crowded carriage doors.

“We can try and match them better. Get people standing in the right spot. With subtle visual cues to encourage people to move to certain parts of the platform. And we can game the system a bit, we can bias the bottom of staircases, make them go red a little sooner to discourage people from waiting there. If we know a wheelchair user is coming in on carriage four, we can turn the light red [on the corresponding platform space] to keep it clear,” Warren said.

“There’s the long term view of mining that data to see what’s happening and there’s also responding to what’s happening on the train platform right now,” he added.

More use cases down the track

DwellTrack has been backed by the government’s Rail Manufacturing Cooperative Research Centre, which was established in 2014. Data gathering exercises have taken place at Brisbane Central, Town Hall and Redfern stations, and further trials are planned.

UTS’ Rapido group has already manufactured a number of the depth cameras – they have to be heat sealed to clear railway safety standards – as part of a limited production run. The software behind it is also being commercialised, which Downer Rail could eventually offer to customers.

The person-tracking watermelons have potential beyond the station platform.

“Anywhere where you have crowds of people and you’re interested in understanding their movement autonomously, that’s going to be useful,” Virgona said.

While transport and retail environments are well covered by CCTV, the techniques used in the project have distinct advantages for privacy and spacial modelling.

“There’s far less person identifying information in [our footage] compared to a colour camera. You just see this grey silhouette basically – there’s no skin colour, no eye colour, no hair colour, you can’t tell a lot about the person personally from it. That’s potentially a good thing in a lot of contexts,” he said.

Copyright 2019 IDG Communications. ABN 14 001 592 650. All rights reserved. Reproduction in whole or in part in any form or medium without express written permission of IDG Communications is prohibited.