In that example the whole loading is driven from the command line, bypassing the need for writing a command in the pgloader command syntax entirely. As there's no command though, the extra information needed must be provided on the command line using the --type and --field and --with switches.

For documentation about the available syntaxes for the --field and --with switches, please refer to the CSV section later in the man page.

Note also that the PostgreSQL URI includes the target tablename.

Reading from STDIN

File based pgloader sources can be loaded from the standard input, as in the following example:

Some more options have to be used in that case, as the file contains a one-line header (most commonly that's column names, could be a copyright notice). Also, in that case, we specify all the fields right into a single --field option argument.

Again, the PostgreSQL target connection string must contain the tablename option and you have to ensure that the target table exists and may fit the data. Here's the SQL command used in that example in case you want to try it yourself:

Also notice that the same command will work against an archived version of the same data.

Streaming CSV data from an HTTP compressed file

Finally, it's important to note that pgloader first fetches the content from the HTTP URL it to a local file, then expand the archive when it's recognized to be one, and only then processes the locally expanded file.

In some cases, either because pgloader has no direct support for your archive format or maybe because expanding the archive is not feasible in your environment, you might want to stream the content straight from its remote location into PostgreSQL. Here's how to do that, using the old battle tested Unix Pipes trick:

Now the OS will take care of the streaming and buffering between the network and the commands and pgloader will take care of streaming the data down to PostgreSQL.

Migrating from SQLite

The following command will open the SQLite database, discover its tables definitions including indexes and foreign keys, migrate those definitions while casting the data type specifications to their PostgreSQL equivalent and then migrate the data over:

createdb newdb
pgloader ./test/sqlite/sqlite.db postgresql:///newdb

Migrating from MySQL

Just create a database where to host the MySQL data and definitions and have pgloader do the migration for you in a single command line: