Preamble

NX-API is an enhancement to the Cisco Nexus 9000 Series CLI system, which supports XML output. NX-API also supports JSON output format for specific commands.

NX-API uses HTTP/HTTPS as its transport. CLIs are encoded into the HTTP/HTTPS POST body.

NX-API supports HTTPS. All communication to the device is encrypted when you use HTTPS.

NX-API is integrated into the authentication system on the device. Users must have appropriate accounts to access the device through NX-API. NX-API uses HTTP basic authentication. All requests must contain the username and password in the HTTP header. You should consider using HTTPS to secure your user’s login credentials.

Process

After generating a self-signed certificate off-box, copy the files to the switch supervisor(s) and configure the NX-API feature with HTTPS using the self-signed certs.

Generate a Certificate

Generate a self-signed certificate off-box, using OpenSSL on any linux host.

The -nodes argument is used because NX-OS cannot import a passphrase encrypted key file; ref: CSCva75989… This bug is still present in NX-OS version 7.0(3)I7(4).

Preamble

FileCloud is an enterprise grade file sharing, sync, and backup solution developed by CodeLathe that can be self-hosted (on-prem). They provide sync clients for Windows, Mac, and Linux.

The Linux FileCloud sync client can serve its purpose, but getting it working and accessible can be a bit cumbersome.

The Wound

As it currently sits, the Linux sync client installer isn’t quite as flushed out as the Windows sync client installer, primarily because they don’t have very many Linux users (reference).

For Linux, the documented installation process involves downloading and unzipping their latest archive, making a couple files executable, and then running one of them to start the sync app… which may work, providing all dependencies are met.

As user Ilsa Loving calls out on their community forums, the process leaves a bit to be desired.

Linux Sync client support for 18.10

The Linux client as it currently exists is hopelessly inadequate. On Ubuntu (18.10) it won’t launch. Furthermore, you have no idea what’s wrong unless you modify the filecloudsyncstart.sh file and remove the redirection to /dev/null. Apparently I do not have the necessary libpng12.so.0 file.

For anyone else having the problem and are using Ubuntu, here is the solution:https://www.linuxuprising.com/2018/05/fix-libpng12-0-missing-in-ubuntu-1804.html

In addition, there is no .desktop file, so unless the user knows how to create one on their own, it is not possible to launch the app from the application menu.

The best option would be to provide the linux sync client in the form of a snap, flatpak or appimage package, so that you can distribute the client in a non-distribution specific manner, using whatever libraries you want.

To elaborate on her points;

An end-user would have no idea if they’re missing a dependency since the filecloudsyncstart.sh script dumps output to /dev/null (prevents it from being displayed in terminal). So executing the command listed in the installation guide could lead to some head-scratching and googling things like "linux task manager" while trying to make sense of filtering top or ps aux via terminal, looking for a stale "filecloud" process.

A .desktop file is not included, which means an end-user will either have to create one or run the script manually any time they want to launch the sync client. This also adds complexity for end-users utilizing UI tools to automatically launch the sync client at startup/login.

Additionally the downloaded .zip file contains 93 files nested in the root of the archive; a plethora of libraries and images, and few certs and scripts. When the average end-user downloads the file and executes the command provided in the installation guide, all 93 of those files are likely going to be dumped straight into their ~/Downloads folder.

The Bandaid

I put a crude script on GitHub that will install the current sync client (18.2.0.1012). I created this on my current preferred flavor, Kubuntu 18.10 but it should work on any recent (.?[Uu]buntu) release.

Post-Install Thoughts

Cache & Config Visibility

The ~/FileCloudSyncData (cache folder) and ~/syncclientconfig.xml file are both created when the sync client is launched the first time; I’d like to see these files prepended with . by default so they aren’t visible in graphical file managers.

I was able to prepend the cache folder with a . (i.e. ~/.FileCloudSyncData) and then update the config file accordingly…

However I can’t prepend the config file with a . because the sync client won’t find it, causing it to populate a new one.

Initial Setup Wizard

This one is not constrained to Linux… once the sync client is launched on any platform and a user logs in, it immediately starts syncing everything to the default location. I’d like to see options to configure selective sync and sync location before any syncing occurs; similar to other solutions like MEGA and Dropbox, that provide a setup wizard to accommodate these changes.

Preamble

About a year ago /u/packetsar shared FreeZTP on reddit, just in time for a project that required upgrade/configuration of 400+ IOS-XE switches (3650/3850, several stacks). Over the following months, he was extremely receptive and accommodating with feature updates and tweaks; so kudos, pal.

This is a sanitized example of a switch template that I built out with a strong focus on keeping the process as simple and unbreakable as possible for the intended end-users, at the cost of a self-imposed excessively large learning curve. My lastthreeblog posts were a direct result of achieving this goal; something that a lot of docs, labbing, and scouring of community forums contributed to.

A Few Disclosures

I do not have a background in development, this was my first venture into any kind of programming/templating. This was largely an exercise in figuring out what I could accomplish in the template, without having to manually intervene, whether it be via python, console, or otherwise. There are probably some development best-practices that I unwittingly spat in the face of, or more optimal ways to accomplish some of these functions.

/u/packetsar has done an excellent job detailing all facets of FreeZTP, so questions/concerns regarding its inner-workings are better answered by the project’s GitHub readme.md.

FreeZTP config

I configured FreeZTP to use external keystore and template files. This kept the code and other sensitive template information somewhat concealed, for lack of a better term. This also kept ztp show config outputs concise and easier to read when validating FreeZTP settings.

cdp_desc_plat

This version of the applet gathers more information if AIR, Phone, or ATA is found in the CDP neighbor’s platform string ($_nd_cdp_platform), otherwise it does exactly what the cdp_desc applet does.

This is intended for use on access switches where devices such as wireless APs and IP Phones or ATAs are connected. Quite often some (or all) of these devices aren’t adequately renamed once adopted to the network; i.e. their hostnames are not set. Their default hostnames can be quite long, so to adhere with the aforementioned consideration (personal best-practice) the applet does the following for each platform type…

AIR (Cisco AP)

Applet gets the AP model number from $_nd_cdp_platform.

Applet gets the last four of the AP’s ethernet MAC address; i.e. checks the connected interface for a MAC address in the CAPWAP VLAN, this should be the AP.

Preamble

This was created for and used with FreeZTP, but can likely be used with any Jinja2 templating system.

When this snippet (located at the bottom of this post) is added to a Jinja2 template it will automatically build out an EEM applet that will set switch priorities and renumber all switches in the stack according to how they were allocated in the FreeZTP keystore.

All switches in the stack can be powered on simultaneously; i.e. there is no need to do the 2-minute-tango between powering up switches. After the election processes are complete, the stack will continue booting and then perform the smart-install procedure where it will receive this applet as part of the templated configuration.

In this use-case, FreeZTP’s idarray variables are being used to define stack member serial numbers. When the template is merged with the keystore, an action sequence is generated for each serial number (idarray_#) associated with the hostname (keystore_id). Switch serial numbers should be assigned to idarray_#‘s as they’re meant to be numbered in the stack (see the CSV example below).

Example

In this example, there are four switches allocated to the stack ASW-TR01-01;

FOC11111111 (idarray_1) should be switch 1 in the stack, with a priority of 15.

FOC22222222 (idarray_2) should be switch 2 in the stack, with a priority of 14.

FOC33333333 (idarray_3) should be switch 3 in the stack, with a priority of 13.

FOC44444444 (idarray_4) should be switch 4 in the stack, with a priority of 12.

CSV

Variables defined in the keystore (FreeZTP);

keystore_id = Hostname of the switch.

association = J2 template configured in FreeZTP.

idarray_# = Switch serial number(s).

idarray_# fields are populated with serial numbers in accordance with how they are to be numbered in the stack.

Switch Priorities

The default priority for all switches (out of the box) is 1; these priorities do not change during the election process.

Notice that all switches have a priority of 1 in the output below. The priority range is 1-15; the (online) switch with highest priority will be chosen as the active ‘supervisor’ switch during any election processes/failure events.

J2 variable sw_count is set during the merge process by counting serial numbers found in idarray.

Stack applies configuration and a syslog message is generated; sw_stack applet is triggered.

[EEM Applet sw_stack loaded to memory.]

Waits 120 seconds for the stack redundancy operations to complete.

Executes command show module | inc ^.[1-9](output as read by EEM for current example);

This output is what EEM stores as $stack for parsing. All line numbers correlate with the stack’s current switch allocation numbers; i.e. first line contains information for switch 1, second line contains information for switch 2, etc…

[J2 Loop] For each switch, searches the entire output for its serial number;

If not found, a syslog message will be generated and the applet will move onto the next switch.

[EEM Loop] If found, the applet searches the output line-by-line until it finds the serial number;

The last line of the output is the switch hostname (elevated prompt), which is ignored when searching for switch serial number(s); i.e. the number of lines that the applet will search is limited by the Jinja2 sw_count variable set in the template.

If the line number where the serial number was found matches the allocated switch number (idarray_#), only the priority will be set.

If the line number where the serial number was found does not match the allocated switch number, the priority will be set and the switch will be renumbered.

Syslog messages are generated outlining any errors and all changes made to priorities and numbers.

Applet deletes itself from running configuration, writes the startup-config, and generates a syslog message stating that the process is complete.

Considerations/Notes

Validated on Cisco 3850-12X48U-S switches with IOS-XE 3.7.4E installed out of the box; upgrading to IOS-XE 16.3.6.

Default TFTP blocksize is 512 on IOS-XE 3.7.4E, and 8192 on IOS-XE 16.3.6; adding this to the template significantly reduces the image transfer time.

The J2 template should not contain any configuration or syntax that is incompatible in IOS-XE version (3.7.4E). Any commands that are compatible with later IOS-XE versions only should be added to the post_ztp_2 applet.

Applet: post_ztp_1

Triggered by the switch receiving a DHCP address on Vlan1; notes regarding the maxrun and wait;

post_ztp_2 is triggered by a syslog command specific to IOS-XE 16.x (%IOSXE_REDUNDANCY-6-PEER).

This trigger may need to be changed!I’m unsure if this syslog message is present when provisioning a stand-alone switch as I never tested this scenario. I will update if I get an opportunity to test this on a stand-alone switch.

[EEM applet post_ztp_2 loaded into memory.]

Applet deletes itself from running configuration and writes the startup-config.

Required Config

Allocate a provisioning interface; i.e. the interface connected to the provisioning network.

Modify the variables in the template config snippet below to suit network/needs, then add the whole snippet to the J2 switch template. These four variables can be defined in the keystore or left in the template.

Variable

Description

tftp_addr

Address of TFTP server, typically FreeZTP.

image_bin

Name of the image file to download.

access_vlan

Vlan to configure on the provisioning interface (Gi1/0/48) after upgrade/reload is complete.

prov_int

Interface to be used for provisioning; e.g. Te1/0/48 (3850-12X48U-S interfaces 37-48 are TenGigabitEthernet.)

Preamble

A user’s status can be set as Bypass in the Duo Admin Panel (to bypass 2FA), but as an exercise we can also use Duo’s Auth Proxy along side a separate authentication server in an environment with Duo 2FA users and non-2FA users.

The end result is the same, end-users only have to type their credentials and they get a push, other flags such as phone are used as fallback in the event that the push isn’t working (e.g. poor cell reception, etc).

Example, in an AD environment;

One tunnel-group that uses the DUO Auth Proxy tied to Duo’s RADIUS application (not Cisco RADIUS VPN); for the LDAP integration configure appropriate values for search_dn and security_group_dn in the /opt/duoauthproxy/conf/authproxy.cfg file to specify an AD security group whose members are 2FA users.

Another tunnel-group that points to another authentication server server; e.g. NPS server with a RADIUS policy pointing to a different AD security group for non-2FA users.

Both tunnel-group aliases will show up in the Anyconnect client, but users will only be able to auth to the AD security group mapped to their respective tunnel-group; providing that the user account isn’t in both security groups.

This raises an issue in any environment where local DNS resolution is needed; i.e. active-directory (AD) domain environments. In a typical AD environment there are at least two domain controllers (DC) that fulfill DNS resolution for local domain endpoints. DNS on these DCs is then configured with Forwarders so public DNS resolution can be fulfilled.

Here are a couple ways to utilize Pi-hole and Cloudflared, I’ll call it Pi-flared, to reap the benefits of DoH for public DNS resolution and still use our DCs for local DNS resolution.

Option 1

AD DNS servers, or DCs, have the Pi-flared DNS server(s) configured as Forwarder(s).

Cons: Since all DNS queries are going to the DCs first, Pi-hole’s query logs will only show the requests as forwarded from the DCs; i.e. granular, per-user metrics will not be available, via Pi-hole anyways.

Option 2

(My preference) Clients are configured with the Pi-flared server(s) for DNS, the Pi-flared servers are then configured with domain-specific servers for local lookups.

Pros: Pi-hole’s query logs will now show requests from every host on the network.

Cons: Reconfiguration of static clients and DHCP scope or server options to define the Pi-flared DNS server(s) for DHCP clients. This will provide the granular, per-user metrics that I want to see in Pi-hole’s admin interface.

Process Order

The steps below require root privs, you will need sudo privileges if you’re logged in as a different user.

Install Ubuntu Mate on your Pi

Install Pi-hole

This command is straight from Pi-hole’s site, where they advise that piping to bash can be dangerous and suggest that you review the code and run the installer locally.

curl -sSL https://install.pi-hole.net | bash

Install Cloudflared

Full disclosure, the steps below are almost directly borrowed from Ben Dew’s DNS Over HTTPS post that I followed to get my Pi-hole/Cloudflared environment setup. I’m merely sharing my version of the deployment.

Preamble

Session logging can prove extremely useful when recollecting events during troubleshooting scenarios or even configuration-binges. SecureCRT’s logging settings allow the use of variables to define the filename/path. I’ve configured my default session to log every session whether I’m troubleshooting or just poking around.

I also have a custom windows environment variable that points %SessionLogs% to the directory of my choosing. This permits me to sync my sessions folder across multiple computers and VMs without having to worry about maintaining a similar directory structure on all of them.

Tip: For troubleshooting, I also like maintain a Notepad++ file or quick handwritten notes with timestamps for significant events, so I can cross-reference them with my timestamped SecureCRT logs.

Considerations

Using Notepad++ I can use CTRL-F and select Find All in Current Document to find events like !!!!!CONNECT, !!!!!DISCONN, <hostname>#, etc.

The %COMPUTERNAME% environment variable is used because my SecureCRT config folder is synced across my workstations/jumpboxes that all write to the same SessionLogs folder, which is also synced; this separates the log files based on the computer I was connecting from.

Preamble

Considering the 9000v is really only for labbing this isn’t a huge deal, but I was able but to default an interface from enable mode; config mode is not required. I don’t know if this is something I’d neccessarily consider a bug, but its something I’ll definitely keep in mind when working with these images.