Intro to using curl with Fastly

Fastly recognizes the power of being able to manage your configurations via API. The platform was built API first, and most everything that is done in the web GUI, can be done via an API. This document will lay out some of the more useful API calls.

Purging:
Fastly provides types of cache purges, making it easy to purge a single URL, a group of URLs (utilizing a surrogate key) or an entire service. Additionally, there is the soft purge which allows purged (stale) content to remain until the newest version can be retrieved from origin. You can read more about soft purge at [here] (https://docs.fastly.com/guides/purging/soft-purges).

Note: Best practice recommends adding a whitelist for purging to ensure it objects are only purged from authorized sources. This can be found here and another resource here.

Managing Configurations:
Fastly archives all of your configurations and locks old configs, so they cannot be overwritten. It is important to understand how to check if the config is locked, how to clone, and how to activate a configuration.

Custom VCL:
Being able to build custom VCLs with Fastly gives the power to the user to programmatically make changes to their configuration. Fastly utilizes Vanish’s VCL language with custom enhancements.
Note: The ability to upload custom VCL is disabled when you sign up by default. It can be enabled by requesting it through support. support@fastly.com

Debugging:
Visibility into how your request is interacting with a cache node is important when trying to troubleshoot caching behavior. Fastly provides some tools through the html headers to help customers diagnose problems with connection behavior and configuration problems.

Turning on the debug flag for a page request:

curl -svo /dev/null [url] -H "Fastly-Debug: true"

The Details: Adds additional headers into the response to understand how the object is being handled by Fastly caches.

Fastly-Debug-Path: which cache servers serve the request

Fastly-Debug-TTL: the TTL of the object in the queried server(s)

Fastly-Debug-Digest: the hash digest of the object.

Run a test with shielding disabled:

curl -svo /dev/null [url] -H "Fastly-No-Shield: true"

The Details: Forces the cache node to interact directly with origin on a MISS. Shielding can introduce problems to VCL code that works perfectly when the request goes through a single node. Enabling this header helps troubleshoot VCL when shielding is involved.

Determine total time for a request to be completed:

curl -s -w "%{time_total}\n" -o /dev/null <url>

The Details: Gather the time it takes for a request to complete to gather some rough benchmark numbers. Put it in a for loop, and you can get look for performance patterns. Add the sleep command in the loop to pause between tests.

One last Tip:
When troubleshooting your configuration by hand, you will often find yourself trying to parse through a JSON response that is all on one line. This makes it difficult for a user to visually find the data they are looking for in a timely manner. There are two tools that come standard on OSX, json_pp or python –m json.tool. By piping your curl command to one of these tools, the output will be formatted for a human rather than a program. Happy curling.

0

Comments

You can actually expand the -w "%{time_total}\n" timing switch to include dns lookup time, connect time, tls negotiation, etc.. to help pin down where bottlenecks are. I tend to just create an alias or call a bash script to bundle all of the juicy curl bits into a single command.

e.g.:
curl -w "\nLookup time:\t%{time_namelookup}\nConnect time:\t%{time_connect}\nPreXfer time:\t%{time_pretransfer}\nStartXfer time:\t%{time_starttransfer}\n\nTotal time:\t%{time_total}\n" -svo /dev/null -HFastly-Debug:1 https://www.example.com/ --compress
Will give you all your headers but the timings will put things like the following at the bottom: