Oracle Blog

A closer look at using Oracle Solaris

Thursday Aug 14, 2014

VXLAN, or Virtual eXtensible LAN, is essentially a tunneling mechanism used to provide isolated virtual Layer 2 (L2) segments that can span multiple physical L2 segments. Since it is a tunneling mechanism it uses IP (IPv4 or IPv6) as its underlying network which means we can have isolated virtual L2 segments over networks connected by IP. This allows Virtual Machines (VM) to be in the same L2 segment even if they are located on systems that are in different physical networks. Some of the benefits of VXLAN include:

Better use of resources, i.e. VMs can be provisioned on systems, that span different geographies, based on system load.

VMs can be moved across systems without having to reconfigure the underlying physical network.

Fewer MAC address collision issues, i.e. MAC address may collide as long as they are in different VXLAN segments.

Isolated L2 segments can be supported by existing mechanisms such as VLANs, but VLANs don't scale; the number of VLANs are limited to 4094 (0 and 1 are reserved), but VXLAN can provide upto 16 million isolated L2 networks.

Additional details, including protocol working, can be found in the VXLAN draft IETF RFC. Note that Solaris uses the IANA specified UDP port number of 4789 for VXLAN.

The following is a quick primer on administering VXLAN in Solaris 11.2 using the Solaris administrative utility dladm(1m). Solaris Elastic Virtual Switch (EVS) can be used to manage VXLAN deployment automatically in a cloud environment - this will be the subject of a future discussion.

The following illustrates how VXLANs are created on Solaris:

where IPx is an IP address (IPv4 or IPv6) and VNIs y and z are different VXLAN segments. VM1, VM2 and VM3 are guests with interfaces configured on VXLAN segments y and z. vxlan1 and vxlan2 are VXLAN links, represented by a new class called VXLAN.

Creating VXLANs

To begin with we need to create VXLAN links in the segments that we want to use for guests - let's assume we want to create segments 100 and 101. Additionally, we also want to create the VXLAN links on IP (remember VXLANs are overlay over IP networks), so we need the IP address over which we want to create the VXLAN links - let's assume our endpoint on this system is 10.10.10.1 (in the following example this IP address resides on net4).

# ipadm show-addr net4

ADDROBJ TYPE STATE ADDR

net4/v4 static ok 10.10.10.1/24

Create VXLAN segments 100 and 101 on this IP address.

# dladm create-vxlan -p addr=10.10.10.1,vni=100 vxlan1

# dladm create-vxlan -p addr=10.10.10.1,vni=101 vxlan2

Notes:

In the above example we explicitly provide the IP address, however, you could also:

provide a prefix and prefixlen to use an IP address that matches it, e.g:

# dladm create-vxlan -p addr=10.10.10.0/24,vni=100 vxlan1

provide an interface (say net4 in our case) to pick an active address on that interface, e.g:

# dladm create-vxlan -p interface=net4,vni=100 vxlan1

(you can't provide interface and addr together)

VXLAN links can be created on an IP address over any interface, including IPoIB link, except IPMP, loopback or VNI (Virtual Network Interface).

The IP address may belong to a VLAN segment.

Displaying VXLANs

Check if we have our VXLAN links:

# dladm show-vxlan

LINK ADDR VNI MGROUP

vxlan1 10.10.10.1 100 224.0.0.1

vxlan2 10.10.10.1 101 224.0.0.1

One thing we haven't talked about so far is the MGROUP. Recall from the RFC that VXLAN links use IP multicast for broadcast. So, we can assign a multicast address to each VXLAN segment that we create. If we don't specify a multicast address, we assign the all-host multicast address (or all nodes for IPv6) to the VXLAN segments. In the above case since we didn't specify the multicast address both vxlan1 and vxlan2 will use the all-host multicast address.

The VXLAN links created, vxlan1 and vxlan2, are just like other datalinks (physical, VNIC, VLAN, etc.) and can be displayed using

# dladm show-link

LINK CLASS MTU STATE OVER

...

vxlan1 vxlan 1440 up --

vxlan2 vxlan 1440 up --

The STATE reflects that state of the VXLAN links which is based on the status of the IP address (10.10.10.1 in this case). Note that the MTU is reduced because of the VXLAN encapsulation for each packet, on this VXLAN link.

Now that we have our VXLAN links, we can create Virtual Links (VNICs) over these VXLAN links. Note, the VXLAN links themselves not active links, i.e. you can't plumb IP address or create Flows on them, but they can be snooped.

# dladm create-vnic -l vxlan1 vnic1

# dladm create-vnic -l vxlan1 vnic2

# dladm create-vnic -l vxlan2 vnic3

# dladm create-vnic -l vxlan2 vnic4

# dladm show-vnic

LINK OVER SPEED MACADDRESS MACADDRTYPE VIDS

vnic1 vxlan1 10000 2:8:20:d9:df:5f random 0

vnic2 vxlan1 10000 2:8:20:72:9a:70 random 0

vnic3 vxlan2 10000 2:8:20:19:c7:14 random 0

vnic4 vxlan2 10000 2:8:20:88:98:6d random 0

You can see from the above that the process of creating a VNIC on a VXLAN link is no different from creating one any other link such as physical, aggregation, etherstub etc. This means that the VNICs created may belong to a VLAN and properties (such as maxbw and priority) can be set on them.

Once created, these VNICs can be assiged explicitly to Solaris zones. Alternatively, the VXLAN links can be set as the lower-link for configuring anet (automatic VNIC) links in Solaris Zones.

For Logical Domains on SPARC, the virtual switch (add-vsw) can be created on the VXLAN device which means the vnets created on the virtual switch will be part of the VXLAN segment.

Deleting VXLANs

A VXLAN can be deleted once all the VNICs over the VXLAN links have been deleted. Thus in our case:

# dladm delete-vnic vnic1

# dladm delete-vnic vnic2

# dladm delete-vnic vnic3

# dladm delete-vnic vnic4

# dladm delete-vxlan vxlan1

# dladm delete-vxlan vxlan2

Additional Notes:

VXLAN for Solaris Kernel zone and LDom guests are not supported with direct I/O.

Hardware capabilities such as checksum and LSO are not available for the encapsulated (inner) packet.

Some earlier implementations (e.g. Linux) might use a pre-IANA assigned port number. If so, such implementations might have to be configured to use the IANA port number to interoperate with Solaris VXLAN.

IP multicast must be available in the underlying network and if communicating across different IP subnets, multicast routing should be available as well.

Modifying properties (IP address, multicast address or VNI) on a VXLAN link is currently not supported; you'd have to delete the VXLAN and re-create it.

Tuesday Apr 29, 2014

The Service Management Facility guide is all new for the Oracle Solaris 11.2 release, with much more information including an example of creating a pair of services that start and stop an Oracle Database instance and an examination of the Puppet stencil service.

For more information about stencil services, see Solaris SMF Weblog, and see the svcio.1 and smf_stencil.4 man pages below.

Chapter 2, "Getting Information About Services"- Service states and contract processes- Service dependencies and dependents- New -L option to show service log files- Property values in layers, snapshots, and customizations

Chapter 5, "Using SMF to Control Your Application"- Creating a service to start or stop an Oracle Database instance- Using a stencil to create a configuration file

Appendix A, "SMF Best Practices and Troubleshooting"- Repairing an instance that is in maintenance- Diagnosing and repairing repository problems- How to investigate problems starting services at system boot

User Commands svcio(1)

NAME svcio - create text files based on service configuration properties

DESCRIPTION The svcio utility reads a template known as a stencil and emits text based on that file in conjunction with the pro- perties from a service instance. In the typical case, svcio is used to generate application-specific configuration files for services that are managed by, but are not able to read their configurations from, SMF.

If the stencil itself contains any errors, svcio will pro- vide a snippet of text along with a line number and the cause of the error. Unless the error would prevent further progress, each error is printed in the order it occurs in the file.

Error messages are printed to the standard error stream.

OPTIONS The following options are supported:

-a

Process all files configured for an instance.

Specifically, svcio will look at all property groups with the type "configfile" and determine which stencil to use and where to write the resulting file by examing the values of the properties "path" and "stencil" within that property group. For example, if property group "conf1" is of the appropriate type then svcio will use the value of "conf1/stencil" as the path of the stencil file and "conf1/path" as the path of the file to which to write the output. Additionally, the optional proper- ties "owner" and "group" can be used to set the owner and group of the output file respectively. If the pro- perty group name or property name contains a reserved character (see smf(5)) then it must be encoded.

-f FMRI-instance

The FMRI of a service instance to run the stencil against.

-g group

The group to associate the output files with

-i file

The path to the stencil file (default is stdin). This option cannot be used with -a.

-l

Rather than outputting a text file, simply list all pro- perties that would be referenced were a file to be out- put.

-L opts

Specify options to be passed to mount(2) when loopback mounting output files. If this option is not specified, output files will not be loopback mounted. The -R switch is required with this option. A regular file will be written to the specified output path, rooted at prefix. This file will be loopback mounted to the speci- fied output path, rooted at / or the value of the -R-fR option.

-m mode

Set the mode for any output file (default is 644).

-o file

The path to the output file (default is stdout). This option cannot be used with -a.

-O owner

Set the owner of the output files

-R prefix

Set the root prefix for all output files.

-p

Create nonexistent intermediate directories in the out- put file path rooted at the value of the -R option. Note: This option will not create directories that are missing in the path to the mount point.

Terminate svcio on the first error rather than continu- ing to the next stencil.

OPERANDS The following operands are supported:

FMRI

A fault management resource identifier (FMRI) that specifies one or more instances (see smf(5)). FMRIs can be abbreviated by specifying the instance name, or the trailing portion of the service name. For example, given the FMRI:

svc:/network/smtp:sendmail

The following are valid abbreviations:

sendmail :sendmail smtp smtp:sendmail network/smtp

The following are invalid abbreviations:

mail network network/smt

If the FMRI specifies a service, then the command applies to all instances of that service, except when used with the -D option.

Abbreviated forms of FMRIs are unstable, and should not be used in scripts or other permanent tools.

FMRI-instance

An FMRI that specifies an instance.

EXAMPLES Example 1 Processing All Configuration Files for an Instance

This example processes all configured configuration files for an instance:

example% svcio -a -f svc:/service:instance

Example 2 Removing All Configuration Files for an Instance

This example unlinks and unmounts all configured configura- tion files for an instance:

example% svcio -au -f svc:/service:instance

Example 3 Using an Unconfigured Stencil for an Instance

This example produces an output file based on a stencil that has not been configured:

NAME smf_stencil - defines the relationship between SMF proper- ties and a flat configuration file

DESCRIPTION A stencil file defines a mapping between SMF properties and flat text files. The Service Management Facility, described in smf(5), uses stencil files in conjunction with the svcio(1) utility to generate text-based configuration files from SMF properties by invoking svcio(1) before the start and refresh methods of a property configured service are run.

The language understood by svcio(1) is comprised of a small set of expressions that can be combined to concisely describe the structure of a configuration file and how to populate that file with data from the SMF repository. The expressions comprising the language are listed below:

I. $%{property_fmri[:<transform><transform_expression>]}

Retrieve and emit the value(s) associated with a property.

<transform> can be one of the following characters, which define how to handle <transform_expression>:

- emit <transform_expression> if the property is not defined

+ emit <transform_expression> if the property is defined

? <transform_expression> is of the form "<true>[/<false>]". If the boolean property is true, then emit <true>, otherwise emit <false>.

, emit <transform_expression> as a delimiter between values in multi-valued properties

^ <transform_expression> is of the form "<p>[/<s>]" where <p> is used as a prefix and <s> is used as a suffix when emitting property values

^* Same as '^', but nothing is emitted if the property is undefined or empty

' <transform_expression> takes the form "<pattern>/<replace>", where <pattern> is a shell pattern style glob (for details, see the File Name Generation section of sh(1)). The first substring to match <pattern> is replaced with <replace>

'' Same as ', but every substring that matches <pattern> is replaced with <replace>

II. $%/regular_expression/ { <sub_elements> }

Process <sub_elements> for each property FMRI and property group FMRI that matches regular_expression. As the pro- perty group and property is specified as an FMRI they must be encoded if they contain reserved characters (see smf(5)).

III. $%<number>

Retrieve a marked subexpression from a regular expression.

Retrieve a marked subexpression from a regular expression.

IV. $%define name /regular_expression/ { <sub_elements> }

Name a regular expression such that it can be used else- where in the stencil.

V. $%[regex_name[:<transform><transform_expression]]

Recall a previously defined regular expression (as in IV). In this case, the set of transform characters is limited to ^, ', and ''.

VI. $%define name arg 1 arg 2 ... argN { <sub_elements> }

Name a macro such that it can be used elsewhere in the stencil.

Note: In the text above, '[' and ']' denote the macro del- imiters rather than optional parameters as they do in I and V.

VII. $%<arg_name>

Retrieve the text associated with a macro argument.

VIII. $%[name foo bar ... baz]

Recall a previously defined macro (as in VI).

IX. $%<method_token>

Retrieve the value of an environment variable represented by a method token describe in smf_method(5).

X. Literal Text

Arbitrary text can be freely interspersed throughout the stencil without any denotative markers.

XI. ;comments

A line that starts with a ';', ignoring leading whi- tespace, is considered a comment and not processed along with the rest of the file.

Any of the special characters above can be escaped by preceding them with a blackslash (\) character. Addition- ally, the '\n' and '\t' sequences are expanded into endlines and tab characters respectively. Any non-special character preceded by '\' will emit only the character following the slash. Thus '\g' will be translated to 'g'.

This element will fetch the value (or values) of a pro- perty and emit a string subject to the transform, the transform string, and the values themselves. <transform> is a one- or two- character identifier that indicates how to modify a property value before emitting it, subject to <transform_string>, as explained above.

Note that nesting is allowed. Imagine we want to print the value of foo/b if foo/a is defined, but 'blueberry' if it is not. This could be accomplished via the following:

it is not. This could be accomplished via the following:

$%{foo/a:?$%{foo/b}/blueberry}

For the purposes of resolving FMRIs into values, a few shortcuts are allowed. Since svcio is always run against a specific instance, properties from that instance can be shortened to "pg/prop" rather than a fully qualified FMRI. To reference properties that are not part of the instance, the full "svc:/service:instance/:properties/pg/prop" is required.

II. $%/regular_expression/ { sub_elements> }

Example: $%/pg/(.*)/ {lorem ipsum}

This element defines a regular expression to match against the entire set of property FMRIs on a system. For each property FMRI that matches, the subelements are evaluated. When evaluating subelements, svcio(1) iterates over match- ing properties in lexicographical order. svcio(1) uses the POSIX extended regular expression set (see regex(5)), and supports saving subexpressions via parentheses. Finally, as a convenience svcio will surround the regular expression with ^ and $ characters. Should you want your expression to match the middle of strings, prepend and append '".*".

Since both properties associated with the operating instance as well as properties from other services or instances, regular expressions are only matched against a subset of FMRIs on the system. If a regular expression includes the substring ":properties", the expression is parsed for the service and/or instance where those proper- ties reside. Once those properties are fetched, the regu- lar expression is matched only against that set. If the regular expression does not contain that substring, the only properties matched are those associated with the operating instance.

Note that the end of a regular expression is denoted by '/ {' so it is not necessary to escape slash characters within the regular expression.

III. $%<number>

Example: $%3

This element emits the value from a stored subexpression in a preceding regular expression. Using this element outside the context of a regular expression is an error. A valid use would be as follows:

$%/foo/(.*)/ { $%1 = $%{foo/$%1} }

In the preceding example, every property in property group foo would be emitted as "<property_name> = <property_value>".

Since arbitrary subelements are allowed within a regular expression block, nested regular expressions have their subexpression indices adjusted relative to the index of the last subexpression of the containing expression. For example:

In the preceding example, every property group for an instance would be emitted in blocks as follows:

[property_group] prop1 = <prop1_value> prop2 = <prop2_value> ...

IV. $%define name /regular_expression/ { <sub_elements> }

Example: $%define getProp //(.*)/ {dolor sit amet}

This element follows the same basic rules as element II, but stores the element as a named regular expression that can be invoked later in the stencil file. Named regular expressions are not matched unless they are referenced as per element V, which immediately follows. Additionally, This element cannot be a child to any other.

V. $%[regex_name:<transform><transform_string>]

Example: $%[getProp:^restarter]

This inserts a previously defined regular expression, along with all its subelements into the stencil as though the definition were copy and pasted. Since the insertion is performed literally, there are some special rules that govern how the insertion is done in order to allow such an element to be meaningful at many levels of expression nesting. First of all, all subexpression indices are interally adjusted so that they do not collide with the outer regular expression context. Second, a subset of the transformations from element I are allowed. These transforms operate on relative FMRIs within the inserted element. Absolute FMRIs are left untouched. This allows a stencil author to do useful things like prepend to the FMRI in order to express logical property nesting. Here's an example:

$%define PROPERTY /(.*)/ { $%1 = $%{$%1} }

$%/([a-zA-Z_-]*)/ { [$%1] $%[PROPERTY:^$%1/] }

When the insertion is done, the expression will function as follows:

$%/([a-zA-Z_-]*)/ { [$%1] $%/$%1/(.*)/ { $%2 = $%{$%1/$%2} } }

This is equivalent to the example in element III.

It ends up this way because the rebasing during substitu-

This is equivalent to the example in element III.

It ends up this way because the rebasing during substitu- tion changes the $%1 to $%2, since $%1 occurs in the outer expression. And as a result of the prepend transform applied during substitution, the string "$%1/" is prepended to both the regular expression (since regular expressions match FMRIs) as well as to the element of type II, allowing it resolve to a full pg/property specifier. The subset of allowed transforms is ^,',''. Using other transforms is an error.

Macros provide simple text substitution with respect to the arguments defined for the macro. When called subse- quent to definition, the text of the sub-elements is emit- ted with the text of the arguments substituted where appropriate. See the elements below for more details.

VII. $%<argName>

Example: $%prop

This element emits the corresponding value passed into the macro that uses argName as an argument. For example:

After a macro has been defined, the sub-elements in con- tains can be substituted into other parts of the stencil by using the form above. When invoking a macro, spaces are used to delimit arguments. In order to use a space within the value of an argument, it is necessary to escape that space with a ''. For example, if we have the macro:

$%define theMacro variable value { $%variable = $%value }

We can then use this form to substitute that text else- where in the stencil. For example, we can call it as fol- lows:

$%[theMacro ciphers elGamal\ 3DES\ AES\ Blowfish]

And the resulting text in the output file would be:

ciphers = elGamal 3DES AES Blowfish

IX. $%<method_token>

Example: $%s

Each of the single-character method tokens described in smf_method(5) are available in stencils. In particular $%r, $%m, $%s, $%i, $%f, and $%% are understood and expanded. Due to the high chance of collision with macro variables (element VII), macro variables have precedence over method tokens when expansion occurs. This means that

variables (element VII), macro variables have precedence over method tokens when expansion occurs. This means that if the macro variable $%someVar is encountered, it will be expanded to the value of $%someVar rather than 'service- nameomeVar'. If output such as 'service-nameomeVar' is desired, simply escape a character in the macro variable as in $%s\omeVar.

Literal text can be freely interspersed within the stencil and is emitted without modification. The examples above make limited use of literal text. Text appearing inside a regular expression is emitted for each match, but is not emitted if there are no matches. Text appearing outside all the preceding expression types is emitted in all cases.

XI. Comments

Example: ;this is a comment ;so is this \;this text will appear in the output file so will this, even with the ';' character

To begin a comment, start the line with a ';' character (not including whitespace). The comment continues until the end of the line. If having comments in the resulting output file is desired, simply escape the ';' with a '' character.

Saturday Apr 19, 2014

You may have already heard that we're going to hold the Oracle Solaris 11.2 launch in New York City in a few days, and that there will also be a live webcast of the event.

One of the things that the webcast will feature that won't be part of the live event will be additional technical presentations where Solaris engineers will go into more detail about some of the new features that are being added. VP for Solaris core engineering Markus Flierl gives a quick rundown:

If this sounds interesting to you, you should register now. The event starts at 1 PM ET / 10 AM PT, with Mark Hurd and John Fowler. Markus then moves on to the more technical part of the in-person event, which will then be followed by the web-only deep-dive presentations.

During the live event, we'll have engineering folks and others on Twitter, tracking hashtag #solaris (apologies in advance to Stanislaw Lem fans).

Friday Jan 10, 2014

Glynn Foster notes that another OTN Virtual Sysadmin Day is coming up in just a couple of weeks, and talks about what's in store for the Oracle Solaris 11 track.

If you're not familiar with these, they're half-day, online, proctored hands-on labs, so you can learn more about various system administration technologies. They're also free--but you do need to register, and there's also some prep work to be done ahead of the event, so take a look at Glynn's blog post, and sign up today.

Tuesday Apr 24, 2012

You may have noticed over to the right, there's that red box mentioning the Solaris online forum coming up tomorrow.

There's still time to register; we'll have Markus Flierl, the head of Solaris core engineering, discussing what's been going on since the launch last November, plus two of his senior engineers, Dan Price and Bart Smaalders, who will be giving their point of view on not only what's cool in Solaris 11 today, but what they're working on for future updates.

Plus, we'll have a live Q&A running throughout the forum, so you can ask questions directly to various Solaris engineers.

Tuesday Oct 25, 2011

On November 9th, we're holding the Oracle Solaris 11 launch event
at Gotham Hall on Broadway in New York City. It should not only be a
lot of fun, but we're bringing our engineers, our execs, and some cool
software and hardware, so it's a chance to learn even more about what
we've been doing, and get a jump on the latest release of the #1
enterprise OS.

Register now --- space is limited, and you don't want to miss this event. It's been literally years in the making.

"The regal granite walls and delicate stained-glass
skylight are softened and warmed by the glow of the gold-leaf dome
providing an extraordinary experience that leaves you and your guests
breathless."

So that sounds pretty awesome right there. But I checked with our
event planners, and they said the contract specifically forbids crashing
through the ornate skylight. (Apparently, they've been asked before.)

Tuesday Mar 29, 2011

Leading up to the release of Solaris 11 later this year, the team has picked a compelling feature to "spotlight" each month of 2011. The spotlights include podcasts, screencasts, demos, white papers, cheat sheats, how-to guides, related blog posts and links to the official product documentation. In the very least, it's a great collection of all the material that we have on a given topic.

Friday Mar 25, 2011

Set aside 3 hours on April 14th to attend the Solaris Online Forum. The event runs from 9:00 AM to 12:15 AM US Pacific time on Thursday April 14th (click the links to find the corresponding day and time in your part of the world).

Fair warning my Solaris friends, in pains me to say that the company Oracle contracts to host the event, on24, doesn't include Solaris as a supported platform on their Test Your System page. However, I am told that as long as you're running Firefox 3.x with Flash 9.0.115+, you'll be OK.

Upgrade Note

The Release Notes do include instructions on how to upgrade to Solaris 11 Express from OpenSolaris. If you're still running OpenSolaris 2009.06 (build 111b), you'll first need to update to the never released (until now) 2010.05 (build 134). If you're already on a development build of OpenSolaris, you should be good to go. I also recommend using the command line when running image-update. I've had mixed results when trying to use to the Image Update GUI.

If you're unable to watch the live event, it is being recorded and I'll post a link to that when it's available.

Details on the Oracle Solaris Summit

This FREE all-day event will take deep-dives into each of the major
technologies in Oracle Solaris 11 Express that you'll need to
understand to deploy Oracle Solaris 11 in the enterprise. Each
discussion is led by a Solaris engineering or technical expert.

Tuesday, Nov 9, 2010 in Ballroom A4/A5, San Jose Convention Center. The summit starts at 9:00 AM Pacific Standard Time. You can see what time that is for you here.

Monday Sep 28, 2009

Not surprisingly, Sun is a big sponsor at this year's fast approaching Oracle OpenWorld . I used to think JavaOne was a big event, but OOW draws about 3 times the attendance. And with Oracle's renewed focus on Sun, you can bet Solaris has a big presence. For your convenience, we've put together a landing page with all things Solaris at Oracle OpenWorld. At a minimum, if you plan to be in the Bay Area the week of October 11th, consider the Discover package. For $75 ($125 on-site) it gets you access to the keynote sessions as well as the exhibit hall where me and my team will be hanging out all week showing off Solaris.

Friday Jul 17, 2009

Installing a zone in OpenSolaris requires a network connection and some
patience as a little over 70MB of data is downloaded. Fortunately, after
you've got the first zone installed, future zones can be cloned.[Read More]