After searching many unrelated GitHub issues, fiddling with the aws-ecs tool and just attempting to inject likely looking strings into the command field; I eventually stumbled on a single example in the AWS documentation.

For Command override, type the command override to send. If your container definition does not specify an ENTRYPOINT, the format should be a comma-separated list of non-quoted strings.

The trick is to replace spaces (outside of quoted strings) with commas. For example, our earlier command becomes python,manage.py,migrate.

I suspect the reason for this is that in a Dockerfile the format is: CMD ["python", "manage.py", "migrate"]. In fact, this is what the command object (as in the example above) is translated to.