Schema API

The Schema API allows you to use an HTTP API to manage many of the elements of your schema.

The Schema API utilizes the ManagedIndexSchemaFactory class, which is the default schema factory in modern Solr versions. See the section Schema Factory Definition in SolrConfig for more information about choosing a schema factory for your index.

This API provides read and write access to the Solr schema for each collection (or core, when using standalone Solr). Read access to all schema elements is supported. Fields, dynamic fields, field types and copyField rules may be added, removed or replaced. Future Solr releases will extend write access to allow more schema elements to be modified.

Note

Why is hand editing of the managed schema discouraged?

The file named "managed-schema" in the example configurations may include a note that recommends never hand-editing the file. Before the Schema API existed, such edits were the only way to make changes to the schema, and users may have a strong desire to continue making changes this way.

The reason that this is discouraged is because hand-edits of the schema may be lost if the Schema API described here is later used to make a change, unless the core or collection is reloaded or Solr is restarted before using the Schema API. If care is taken to always reload or restart after a manual edit, then there is no problem at all with doing those edits.

The API allows two output modes for all calls: JSON or XML. When requesting the complete schema, there is another output mode which is XML modeled after the managed-schema file itself, which is in XML format.

When modifying the schema with the API, a core reload will automatically occur in order for the changes to be available immediately for documents indexed thereafter. Previously indexed documents will not be automatically updated - they must be re-indexed if existing index data uses schema elements that you changed.

Important

Re-index after schema modifications!

If you modify your schema, you will likely need to re-index all documents. If you do not, you may lose access to documents, or not be able to interpret them properly, e.g., after replacing a field type.

Modifying your schema will never modify any documents that are already indexed. You must re-index documents in order to apply schema changes to them. Queries and updates made after the change may encounter errors that were not present before the change. Completely deleting the index and rebuilding it is usually the only option to fix such errors.

Modify the Schema

To add, remove or replace fields, dynamic field rules, copy field rules, or new field types, you can send a POST request to the /collection/schema/ endpoint with a sequence of commands in JSON format to perform the requested actions. The following commands are supported:

add-field: add a new field with parameters you provide.

delete-field: delete a field.

replace-field: replace an existing field with one that is differently configured.

add-dynamic-field: add a new dynamic field rule with parameters you provide.

delete-dynamic-field: delete a dynamic field rule.

replace-dynamic-field: replace an existing dynamic field rule with one that is differently configured.

add-field-type: add a new field type with parameters you provide.

delete-field-type: delete a field type.

replace-field-type: replace an existing field type with one that is differently configured.

add-copy-field: add a new copy field rule.

delete-copy-field: delete a copy field rule.

These commands can be issued in separate POST requests or in the same POST request. Commands are executed in the order in which they are specified.

In each case, the response will include the status and the time to process the request, but will not include the entire schema.

When modifying the schema with the API, a core reload will automatically occur in order for the changes to be available immediately for documents indexed thereafter. Previously indexed documents will not be automatically handled - they must be re-indexed if they used schema elements that you changed.

Add a New Field

The add-field command adds a new field definition to your schema. If a field with the same name exists an error is thrown.

All of the properties available when defining a field with manual schema.xml edits can be passed via the API. These request attributes are described in detail in the section Defining Fields.

For example, to define a new stored field named "sell_by", of type "pdate", you would POST the following request:

Replace a Field

The replace-field command replaces a field’s definition. Note that you must supply the full definition for a field - this command will not partially modify a field’s definition. If the field does not exist in the schema an error is thrown.

All of the properties available when defining a field with manual schema.xml edits can be passed via the API. These request attributes are described in detail in the section Defining Fields.

For example, to replace the definition of an existing field "sell_by", to make it be of type "date" and to not be stored, you would POST the following request:

Add a Dynamic Field Rule

The add-dynamic-field command adds a new dynamic field rule to your schema.

All of the properties available when editing schema.xml can be passed with the POST request. The section Dynamic Fields has details on all of the attributes that can be defined for a dynamic field rule.

For example, to create a new dynamic field rule where all incoming fields ending with "_s" would be stored and have field type "string", you can POST a request like this:

Delete a Dynamic Field Rule

The delete-dynamic-field command deletes a dynamic field rule from your schema. If the dynamic field rule does not exist in the schema, or if the schema contains a copy field rule with a target or destination that matches only this dynamic field rule, an error is thrown.

For example, to delete a dynamic field rule matching "*_s", you can POST a request like this:

Replace a Dynamic Field Rule

The replace-dynamic-field command replaces a dynamic field rule in your schema. Note that you must supply the full definition for a dynamic field rule - this command will not partially modify a dynamic field rule’s definition. If the dynamic field rule does not exist in the schema an error is thrown.

All of the properties available when editing schema.xml can be passed with the POST request. The section Dynamic Fields has details on all of the attributes that can be defined for a dynamic field rule.

For example, to replace the definition of the "*_s" dynamic field rule with one where the field type is "text_general" and it’s not stored, you can POST a request like this:

Add a New Field Type

The add-field-type command adds a new field type to your schema.

All of the field type properties available when editing schema.xml by hand are available for use in a POST request. The structure of the command is a JSON mapping of the standard field type definition, including the name, class, index and query analyzer definitions, etc. Details of all of the available options are described in the section Solr Field Types.

For example, to create a new field type named "myNewTxtField", you can POST a request as follows:

Delete a Field Type

The delete-field-type command removes a field type from your schema. If the field type does not exist in the schema, or if any field or dynamic field rule in the schema uses the field type, an error is thrown.

For example, to delete the field type named "myNewTxtField", you can make a POST request as follows:

Replace a Field Type

The replace-field-type command replaces a field type in your schema. Note that you must supply the full definition for a field type - this command will not partially modify a field type’s definition. If the field type does not exist in the schema an error is thrown.

All of the field type properties available when editing schema.xml by hand are available for use in a POST request. The structure of the command is a JSON mapping of the standard field type definition, including the name, class, index and query analyzer definitions, etc. Details of all of the available options are described in the section Solr Field Types.

For example, to replace the definition of a field type named "myNewTxtField", you can make a POST request as follows:

Multiple Commands in a Single POST

It is possible to perform one or more add requests in a single command. The API is transactional and all commands in a single call either succeed or fail together.

The commands are executed in the order in which they are specified. This means that if you want to create a new field type and in the same request use the field type on a new field, the section of the request that creates the field type must come before the section that creates the new field. Similarly, since a field must exist for it to be used in a copy field rule, a request to add a field must come before a request for the field to be used as either the source or the destination for a copy field rule.

The syntax for making multiple requests supports several approaches. First, the commands can simply be made serially, as in this request to create a new field type and then a field that uses that type:

Schema Changes among Replicas

When running in SolrCloud mode, changes made to the schema on one node will propagate to all replicas in the collection.

You can pass the updateTimeoutSecs parameter with your request to set the number of seconds to wait until all replicas confirm they applied the schema updates. This helps your client application be more robust in that you can be sure that all replicas have a given schema change within a defined amount of time.

If agreement is not reached by all replicas in the specified time, then the request fails and the error message will include information about which replicas had trouble. In most cases, the only option is to re-try the change after waiting a brief amount of time. If the problem persists, then you’ll likely need to investigate the server logs on the replicas that had trouble applying the changes.

If you do not supply an updateTimeoutSecs parameter, the default behavior is for the receiving node to return immediately after persisting the updates to ZooKeeper. All other replicas will apply the updates asynchronously. Consequently, without supplying a timeout, your client application cannot be sure that all replicas have applied the changes.

Retrieve Schema Information

The following endpoints allow you to read how your schema has been defined. You can GET the entire schema, or only portions of it as needed.

Defines the format of the response. The options are json or xml. If not specified, JSON will be returned by default.

fl

Comma- or space-separated list of one or more fields to return. If not specified, all fields will be returned by default.

includeDynamic

If true, and if the fl query parameter is specified or the fieldname path parameter is used, matching dynamic fields are included in the response and identified with the dynamicBase property.

If neither the fl query parameter nor the fieldname path parameter is specified, the includeDynamic query parameter is ignored.

If false, the default, matching dynamic fields will not be returned.

showDefaults

If true, all default field properties from each field’s field type will be included in the response (e.g., tokenized for solr.TextField). If false, the default, only explicitly specified field properties will be included.

List Fields Response

The output will include each field and any defined configuration for each field. The defined configuration can vary for each field, but will minimally include the field name, the type, if it is indexed and if it is stored.

If multiValued is defined as either true or false (most likely true), that will also be shown. See the section Defining Fields for more information about each parameter.

List Dynamic Fields

List Dynamic Field Parameters

The name of the dynamic field rule (if limiting request to a single dynamic field rule).

Query Parameters

The query parameters can be added to the API request after a '?'.

wt

Defines the format of the response. The options are json or xml. If not specified, JSON will be returned by default.

showDefaults

If true, all default field properties from each dynamic field’s field type will be included in the response (e.g., tokenized for solr.TextField). If false, the default, only explicitly specified field properties will be included.

List Dynamic Field Response

The output will include each dynamic field rule and the defined configuration for each rule. The defined configuration can vary for each rule, but will minimally include the dynamic field name, the type, if it is indexed and if it is stored. See the section Dynamic Fields for more information about each parameter.

List Field Types

List Field Type Parameters

The name of the field type (if limiting request to a single field type).

Query Parameters

The query parameters can be added to the API request after a '?'.

wt

Defines the format of the response. The options are json or xml. If not specified, JSON will be returned by default.

showDefaults

If true, all default field properties from each dynamic field’s field type will be included in the response (e.g., tokenized for solr.TextField). If false, the default, only explicitly specified field properties will be included.

List Field Type Response

The output will include each field type and any defined configuration for the type. The defined configuration can vary for each type, but will minimally include the field type name and the class. If query or index analyzers, tokenizers, or filters are defined, those will also be shown with other defined parameters. See the section Solr Field Types for more information about how to configure various types of fields.

List Field Type Examples

Get a list of all field types.

curl http://localhost:8983/solr/gettingstarted/schema/fieldtypes

The sample output below has been truncated to show a few different field types from different parts of the list.

List Copy Fields

List Copy Field Parameters

Defines the format of the response. The options are json or xml. If not specified, JSON will be returned by default.

source.fl

Comma- or space-separated list of one or more copyField source fields to include in the response - copyField directives with all other source fields will be excluded from the response. If not specified, all copyField-s will be included in the response.

dest.fl

Comma- or space-separated list of one or more copyField destination fields to include in the response. copyField directives with all other dest fields will be excluded. If not specified, all copyField-s will be included in the response.

List Copy Field Response

The output will include the source and dest (destination) of each copy field rule defined in schema.xml. For more information about copying fields, see the section Copying Fields.

List Copy Field Examples

Get a list of all copyFields.

curl http://localhost:8983/solr/gettingstarted/schema/copyfields

The sample output below has been truncated to the first few copy definitions.

Manage Resource Data

The Managed Resources REST API provides a mechanism for any Solr plugin to expose resources that should support CRUD (Create, Read, Update, Delete) operations. Depending on what Field Types and Analyzers are configured in your Schema, additional /schema/ REST API paths may exist. See the Managed Resources section for more information and examples.