18. Network and downloading

All methods in our engine take URL as the parameter,
not just a FileName. Although in almost all cases you can also pass
a filename (absolute or relative to the current directory),
if that's all you're interested in.

All loading and saving routines (for 3D models, images, sounds, and all
other resources) automatically deal with URLs. To actually load network
URLs (like http) you only need to set
CastleDownload.EnableNetwork to true.

To directly load or save files (as ObjectPascal TStream)
in your own code:

To load, use our simple
Download function.
It automatically handles all the details for you,
including downloading from network (if EnableNetwork),
and returns you a TStream that contains the resource indicated
by the URL. There is an example in the engine code
examples/tools/castle_download that uses this to implement a simple
command-line downloading tool (like wget).

Details about supported URL protocols are below.

To save, use trivial
URLSaveStream function. Right now, it can only save to a local file,
so it merely translates a URL to local filename and creates a TFileStream
for you. Still, it's a good idea to use it, to uniformly deal with
URLs throughout your application.

If you want to read or write text files from an URL, you may
also find useful classes
TTextReader and
TTextWriter.

Supported protocols

Support for network protocols: http.

Our engine automatically handles downloading data from the network.
Currently, only http is supported (through
FpHttpClient unit),
although we plan to add other network protocols (like ftp
and https) in the future.
All you have to do to enable network usage is set global
CastleDownload.EnableNetwork variable to true. By default it is false,
because for now the downloads are not user-friendly —
they are blocking (we wait for them to finish, there's no way
to cancel a download). This will be improved in the future, and eventually
downloading from network may be enabled by default.

All of the attributes in various files are URLs,
so you can refer from a local files to resources on the network.

For example, your game level data may be actually downloaded from the
network when loading level. To do this,
level.xml may use an http protocol when referring to
a scene. Like this:

Inside 3D models (like X3D, VRML and others), you can use network resources,
for example you can Inline them (add a 3D model from another file),
you can make Anchor to them,
you can refer to textures and sounds and scripts and everything else
from the network. Relative URLs are always resolved
with respect to the containing document.

file protocol.

Always (regardless of EnableNetwork value)
we support getting resources from file URLs.
These are just local filenames encoded as URL.
CastleURIUtils contains routines to operate on URLs (and more general URIs),
including converting between regular filenames and URLs with
file: protocol.

Use this to convert a FileName (relative or absolute)
to an absolute URL.

URL := FilenameToURISafe(FileName);

Use this to convert something that may be a FileName or URL to an URL.
This is safer than FilenameToURISafe(...), in that it will
never touch something that already is an URL.
On the other hand, there are some obscure cases
(when a relative filename starts with a component with colon inside)
when it may think that it has URL, while in fact it has a filename that
should be converted to file: URL.

URL := AbsoluteURI(FileNameOrURL);

Use this to convert URL back to a FileName.
When the URL is a file: protocol, it will decode back
the simple filename. Right now, URL without protcol is also
returned back as a simple filename. When the URL uses a different
protocol (like http), returns empty string.

If you read/write filenames from/to Lazarus classes,
for example if you use Lazarus TOpenDialog.FileName or
TSaveDialog.FileName, use the UTF-8 variants instead:
URIToFilenameSafeUTF8 and
FilenameToURISafeUTF8.
That is because Lazarus uses UTF-8 for all strings (as opposed to FPC RTL
that uses system encoding).

data protocol.

We can always load resources from data URIs.
The data URIs allow you to embed various resources
(like textures, sounds, other 3D models) inside a parent file,
for example instead of referring to the texture URL from 3D model —
you can embed the actual texture contents inside 3D model file.
This allows you to pack your data into a single file,
which is sometimes desired (it may be easier to distribute),
and data URIs are very handy for this. See
data: URI specification.

Our engine includes a tool to_data_uri (see inside
castle_game_engine/examples/tools/)
that can turn any file
into a data URI, and you can use such data URI everywhere where we expect URL.

Used to access Android asset files from the Android application.
Asset files live inside your application's .apk file, together with
your compiled game.
Typically you just copy/place the asset files to the
assets/ subdirectory of your Android project,
and then build the .apk using normal Android tools.
For example, file placed in .../assets/my_texture.png
can be accessed (from the Android app) using the URL
assets:/my_texture.png.

Notes about terminology: URI vs URL

URI
is a more general term. URI uniquely identifies a resource but does not
necessarily tell us how to load (download) or save (upload) it.
We have many routines in CastleURIUtils unit that process URIs (strings), they use the more general term URI.
They complement standard FPC URIParser routines.

URL
is a specific type of URI that also tells you how to load or save the resource.
For example http and file protocols define URLs.
Most of our routines that load or save use the term URL.

Things get a little more cloudy when you realize there's also
data URI scheme.
It's not precisely an URL (it's not an address of a resource),
but you can load it (since the URI itself contains the resource).
And we support it fully (our
Download method
loads it automatically). Admittedly, this means that our loading routines
should rather use the term URL or data URI, but that's just long
and (for those who don't use data URI) confusing, so for simplicity we
just keep (over-)using the term URL. Also, other standards (like CSS
and X3D and VRML) allow placing data URIs inside fields called url.

If you enjoy reading about Internet terminology,
note that we use in our engine also
URNs
(another subtype of URI). They are used by X3D external prototypes,
see X3D extensions introduction.