James Packer's $1.5 billion tower should be knocked back, the City of Sydney Council's chief planner said on Thursday, arguing the private structure would "dominate and overwhelm" the long-held vision for the Barangaroo site.