There is an example of working with changesets in the introductory
documentation in the Ecto module. The functions cast/4 and
change/2 are the usual entry points for creating changesets.
The first one is used to cast and validate external parameters,
such as parameters sent through a form, API, command line, etc.
The second one is used to change data directly from your application.

The remaining functions in this module, such as validations,
constraints, association handling, are about manipulating
changesets. Let’s discuss some of this extra functionality.

External vs internal data

Changesets allow working with both kinds of data:

internal to the application - for example programmatically generated,
or coming from other subsystems. This use case is primarily covered
by the change/2 and put_change/3 functions.

external to the application - for example data provided by the user in
a form that needs to be type-converted and properly validated. This
use case is primarily covered by the cast/4 function.

Validations and constraints

Ecto changesets provide both validations and constraints which
are ultimately turned into errors in case something goes wrong.

The difference between them is that most validations can be
executed without a need to interact with the database and, therefore,
are always executed before attempting to insert or update the entry
in the database. Some validations may happen against the database but
they are inherently unsafe. Those validations start with a unsafe_
prefix, such as unsafe_validate_unique/3.

On the other hand, constraints rely on the database and are always safe.
As a consequence, validations are always checked before constraints.
Constraints won’t even be checked in case validations failed.

In the changeset/2 function above, we define three validations.
They check that name and email fields are present in the
changeset, the e-mail is of the specified format, and the age is
between 18 and 100 - as well as a unique constraint in the email
field.

Let’s suppose the e-mail is given but the age is invalid. The
changeset would have the following errors:

Validations and constraints define an explicit boundary when the check
happens. By moving constraints to the database, we also provide a safe,
correct and data-race free means of checking the user input.

Deferred constraints

Some databases support deferred constraints, i.e., constraints which are
checked at the end of the transaction rather than at the end of each statement.

Changesets do not support this type of constraints. When working with deferred
constraints, a violation while invoking Repo.insert/2 or Repo.update/2 won’t
return {:error, changeset}, but rather raise an error at the end of the
transaction.

Empty values

Many times, the data given on cast needs to be further pruned, specially
regarding empty values. For example, if you are gathering data to be
cast from the command line or through an HTML form or any other text-based
format, it is likely those means cannot express nil values. For
those reasons, changesets include the concept of empty values, which are
values that will be automatically converted to the field’s default value
on cast/4. Those values are stored in the changeset empty_values field
and default to [""].

Associations, embeds and on replace

Using changesets you can work with associations as well as with embedded
structs. There are two primary APIs:

cast_assoc/3 and cast_embed/3 - those functions are used when
working with external data. In particular, they allow you to change
associations and embeds alongside the parent struct, all at once.

put_assoc/4 and put_embed/4 - it allows you to replace the
association or embed as a whole. This can be used to move associated
data from one entry to another, to completely remove or replace
existing entries.

See the documentation for those functions for more information.

The :on_replace option

When using any of those APIs, you may run into situations where Ecto sees
data is being replaced. For example, imagine a Post has many Comments where
the comments have IDs 1, 2 and 3. If you call cast_assoc/3 passing only
the IDs 1 and 2, Ecto will consider 3 is being “replaced” and it will raise
by default. Such behaviour can be changed when defining the relation by
setting :on_replace option when defining your association/embed according
to the values below:

:mark_as_invalid - if attempting to remove the association or
embedded data via parent changeset - an error will be added to the parent
changeset, and it will be marked as invalid

:nilify - sets owner reference column to nil (available only for
associations). Use this on a belongs_to column to allow the association
to be cleared out so that it can be set to a new value. Will set action
on associated changesets to :replace

:update - updates the association, available only for has_one and belongs_to.
This option will update all the fields given to the changeset including the id
for the association

:delete - removes the association or related data from the database.
This option has to be used carefully. Will set action on associated changesets
to :replace

The :delete option in particular must be used carefully as it would allow
users to delete any associated data. If you need deletion, it is often preferred
to add a separate boolean virtual field to the changeset function that will allow
you to manually mark it for deletion, as in the example below:

This means that when working with changesets that are not meant to be
persisted to the database, such as schemaless changesets, you may need
to explicitly set the action to one specific value. Frameworks such as
Phoenix use the action value to define how HTML forms should act.

Instead of setting the action manually, you may use apply_action/2 that
emulates operations such as Repo.insert. apply_action/2 will return
{:ok, changes} if the changeset is valid or {:error, changeset}, with
the given action set in the changeset in case of errors.

The Ecto.Changeset struct

The public fields are:

valid? - Stores if the changeset is valid

data - The changeset source data, for example, a struct

params - The parameters as given on changeset creation

changes - The changes from parameters that were approved in casting

errors - All errors from validations

required - All required fields as a list of atoms

action - The action to be performed with the changeset

types - Cache of the data’s field types

empty_values - A list of values to be considered empty

repo - The repository applying the changeset (only set after a Repo function is called)

repo_opts - A keyword list of options given to the underlying repository operation

If the changes are valid, all changes are applied to the changeset data.
If the changes are invalid, no changes are applied, and an error tuple
is returned with the changeset containing the action that was attempted
to be applied.

This is similar to foreign_key_constraint/3 except that the
field is inferred from the association definition. This is useful
to guarantee that a child will only be created if the parent exists
in the database too. Therefore, it only applies to belongs_to
associations.

As the name says, a constraint is required in the database for
this function to work. Such constraint is often added as a
reference to the child table:

createtable(:comments)doadd:post_id,references(:posts)end

Now, when inserting a comment, it is possible to forbid any
comment to be added if the associated post does not exist:

Applies the given params as changes for the given data according to
the given set of permitted keys. Returns a changeset.

The given data may be either a changeset, a schema struct or a {data, types}
tuple. The second argument is a map of params that are cast according
to the type information from data. params is a map with string keys
or a map with atom keys containing potentially invalid data.

During casting, all permitted parameters whose values match the specified
type information will have their key name converted to an atom and stored
together with the value as a change in the :changes field of the changeset.
All parameters that are not explicitly permitted are ignored.

If casting of all fields is successful, the changeset is returned as valid.

Note that cast/4 validates the types in the params, but not in the given
data.

Options

:empty_values - a list of values to be considered as empty when casting.
Defaults to the changeset value, which defaults to [""]

Composing casts

cast/4 also accepts a changeset as its first argument. In such cases, all
the effects caused by the call to cast/4 (additional errors and changes)
are simply added to the ones already present in the argument changeset.
Parameters are merged (not deep-merged) and the ones passed to cast/4
take precedence over the ones already in the changeset.

User|>Repo.get!(id)|>Repo.preload(:addresses)# Only required when updating data|>Ecto.Changeset.cast(params,[])|>Ecto.Changeset.cast_assoc(:addresses,with:&MyApp.Address.changeset/2)

The parameters for the given association will be retrieved
from changeset.params. Those parameters are expected to be
a map with attributes, similar to the ones passed to cast/4.
Once parameters are retrieved, cast_assoc/3 will match those
parameters with the associations already in the changeset record.

Once cast_assoc/3 is called, Ecto will compare each parameter
with the user’s already preloaded addresses and act as follows:

If the parameter does not contain an ID, the parameter data
will be passed to MyApp.Address.changeset/2 with a new struct
and become an insert operation

If the parameter contains an ID and there is no associated child
with such ID, the parameter data will be passed to
MyApp.Address.changeset/2 with a new struct and become an insert
operation

If the parameter contains an ID and there is an associated child
with such ID, the parameter data will be passed to
MyApp.Address.changeset/2 with the existing struct and become an
update operation

If there is an associated child with an ID and its ID is not given
as parameter, the :on_replace callback for that association will
be invoked (see the “On replace” section on the module documentation)

Every time the MyApp.Address.changeset/2 function is invoked, it must
return a changeset. This changeset will be applied to your Repo with
the proper action accordingly.

Note developers are allowed to explicitly set the :action field of a
changeset to instruct Ecto how to act in certain situations. Let’s suppose
that, if one of the associations has only empty fields, you want to ignore
the entry altogether instead of showing an error. The changeset function could
be written like this:

defchangeset(struct,params)dostruct|>cast(params,[:title,:body])|>validate_required([:title,:body])|>casedo%{valid?:false,changes:changes}=changesetwhenchanges==%{}-># If the changeset is invalid and has no changes, it is# because all required fields are missing, so we ignore it.%{changeset|action::ignore}changeset->changesetendend

Partial changes for many-style associations

By preloading an association using a custom query you can confine the behavior
of cast_assoc/3. This opens up the possibility to work on a subset of the data,
instead of all associations in the database.

Taking the initial example of users having addresses imagine those addresses
are set up to belong to a country. If you want to allow users to bulk edit all
addresses that belong to a single country, you can do so by changing the preload
query:

The parameters for the given embed will be retrieved
from changeset.params. Those parameters are expected to be
a map with attributes, similar to the ones passed to cast/4.
Once parameters are retrieved, cast_embed/3 will match those
parameters with the embeds already in the changeset record.
See cast_assoc/3 for an example of working with casts and
associations which would also apply for embeds.

The changeset must have been previously cast using
cast/4 before this function is invoked.

Options

:with - the function to build the changeset from params.
Defaults to the changeset/2 function in the embed module

:required - if the embed is a required field

:required_message - the message on failure, defaults to “can’t be blank”

:invalid_message - the message on failure, defaults to “is invalid”

:force_update_on_change - force the parent record to be updated in the repository if
there is a change, defaults to true

changes is a map or keyword where the key is an atom representing a
field, association or embed and the value is a term. Note the value is
directly stored in the changeset with no validation whatsoever. For this
reason, this function is meant for working with data internal to the
application.

When changing embeds and associations, see put_assoc/4 for a complete
reference on the accepted values.

The check constraint works by relying on the database to check
if the check constraint has been violated or not and, if so,
Ecto converts it into a changeset error.

In order to use the check constraint, the first step is
to define the check constraint in a migration:

createconstraint("users",:price_must_be_positive,check:"price > 0")

Now that a constraint exists, when modifying users, we could
annotate the changeset with a check constraint so Ecto knows
how to convert it into an error message:

cast(user,params,[:price])|>check_constraint(:price_must_be_positive)

Now, when invoking Repo.insert/2 or Repo.update/2, if the
price is not positive, it will be converted into an error and
{:error, changeset} returned by the repository. Note that the error
will occur only after hitting the database so it will not be visible
until all other validations pass.

Options

:match - how the changeset constraint name is matched against the
repo constraint, may be :exact or :suffix. Defaults to :exact.
:suffix matches any repo constraint which ends_with?:name
to this changeset constraint.

The exclusion constraint works by relying on the database to check
if the exclusion constraint has been violated or not and, if so,
Ecto converts it into a changeset error.

Options

:message - the message in case the constraint check fails,
defaults to “violates an exclusion constraint”

:name - the constraint name. By default, the constraint
name is inferred from the table + field. May be required
explicitly for complex cases

:match - how the changeset constraint name is matched against the
repo constraint, may be :exact or :suffix. Defaults to :exact.
:suffix matches any repo constraint which ends_with?:name
to this changeset constraint.

The foreign key constraint works by relying on the database to
check if the associated data exists or not. This is useful to
guarantee that a child will only be created if the parent exists
in the database too.

In order to use the foreign key constraint the first step is
to define the foreign key in a migration. This is often done
with references. For example, imagine you are creating a
comments table that belongs to posts. One would have:

createtable(:comments)doadd:post_id,references(:posts)end

By default, Ecto will generate a foreign key constraint with
name “comments_post_id_fkey” (the name is configurable).

Now that a constraint exists, when creating comments, we could
annotate the changeset with foreign key constraint so Ecto knows
how to convert it into an error message:

cast(comment,params,[:post_id])|>foreign_key_constraint(:post_id)

Now, when invoking Repo.insert/2 or Repo.update/2, if the
associated post does not exist, it will be converted into an
error and {:error, changeset} returned by the repository.

Options

:message - the message in case the constraint check fails,
defaults to “does not exist”

:name - the constraint name. By default, the constraint
name is inferred from the table + field. May be required
explicitly for complex cases

While get_change/3 only looks at the current changes
to retrieve a value, this function looks at the changes and
then falls back on the data, finally returning default if
no value is available.

For relations, these functions will return the changeset data
with changes applied. To retrieve raw changesets, please use get_change/3.

iex> post=%Post{title:"A title",body:"My body is a cage"}iex> changeset=change(post,%{title:"A new title"})iex> get_field(changeset,:title)"A new title"iex> get_field(changeset,:not_a_field,"Told you, not a field!")"Told you, not a field!"

This function merges two changesets provided they have been applied to the
same data (their :data field is equal); if the data differs, an
ArgumentError exception is raised. If one of the changesets has a :repo
field which is not nil, then the value of that field is used as the :repo
field of the resulting changeset; if both changesets have a non-nil and
different :repo field, an ArgumentError exception is raised.

The other fields are merged with the following criteria:

params - params are merged (not deep-merged) giving precedence to the
params of changeset2 in case of a conflict. If both changesets have their
:params fields set to nil, the resulting changeset will have its params
set to nil too.

This is similar to foreign_key_constraint/3 except that the
field is inferred from the association definition. This is useful
to guarantee that parent can only be deleted (or have its primary
key changed) if no child exists in the database. Therefore, it only
applies to has_* associations.

As the name says, a constraint is required in the database for
this function to work. Such constraint is often added as a
reference to the child table:

createtable(:comments)doadd:post_id,references(:posts)end

Now, when deleting the post, it is possible to forbid any post to
be deleted if they still have comments attached to it:

Optimistic
locking (or
optimistic concurrency control) is a technique that allows concurrent edits
on a single record. While pessimistic locking works by locking a resource for
an entire transaction, optimistic locking only checks if the resource changed
before updating it.

This is done by regularly fetching the record from the database, then checking
whether another user has made changes to the record only when updating the
record. This behaviour is ideal in situations where the chances of concurrent
updates to the same record are low; if they’re not, pessimistic locking or
other concurrency patterns may be more suited.

Usage

Optimistic locking works by keeping a “version” counter for each record; this
counter gets incremented each time a modification is made to a record. Hence,
in order to use optimistic locking, a field must exist in your schema for
versioning purpose. Such field is usually an integer but other types are
supported.

Examples

Assuming we have a Post schema (stored in the posts table), the first step
is to add a version column to the posts table:

altertable(:posts)doadd:lock_version,:integer,default:1end

The column name is arbitrary and doesn’t need to be :lock_version. Now add
a field to the schema too:

Puts the given association entry or entries as a change in the changeset.

This function is used to work with associations as a whole. For example,
if a Post has many Comments, it allows you to add, remove or change all
comments at once. If your goal is to simply add a new comment to a post,
then it is preferred to do so manually, as we will describe later in the
“Example: Adding a comment to a post” section.

This function requires the associated data to have been preloaded, except
when the parent changeset has been newly build and not yet persisted.
Missing data will invoke the :on_replace behaviour defined on the
association.

For associations with cardinality one, nil can be used to remove the existing
entry. For associations with many entries, an empty list may be given instead.

If the association has no changes, it will be skipped. If the association is
invalid, the changeset will be marked as invalid. If the given value is not any
of values below, it will raise.

The associated data may be given in different formats:

a map or a keyword list representing changes to be applied to the
associated data. A map or keyword list can be given to update the
associated data as long as they have matching primary keys.
For example, put_assoc(changeset, :comments, [%{id: 1, title: "changed"}])
will locate the comment with :id of 1 and update its title.
If no comment with such id exists, one is created on the fly.
Since only a single comment was given, any other associated comment
will be replaced. On all cases, it is expected the keys to be atoms.
This API is mostly used in scripts and tests, to make it straight-
forward to create schemas with associations at once, such as:

changesets or structs - when a changeset or struct is given, they
are treated as the canonical data and the associated data currently
stored in the association is ignored. For instance, the operation
put_assoc(changeset, :comments, [%Comment{id: 1, title: "changed"}])
will send the Comment as is to the database, ignoring any comment
currently associated, even if a matching ID is found. If the comment
is already persisted to the database, then put_assoc/4 only takes
care of guaranteeing that the comments and the parent data are associated.
This extremely useful when associating existing data, as we will see
in the “Example: Adding tags to a post” section.

Note, however, that put_assoc/4 always expects all data currently associated to
be given. In both examples above, if the changeset has any other comment besides
the comment with id equal to 1, all of them will be considered as replaced,
invoking the relevant :on_replace callback which may potentially remove the
data. In other words, if only a comment with a id equal to 1 is given, it will
be the only one kept. Therefore, put_assoc/4 always works with the whole data,
which may be undesired in some cases. Let’s see an example.

Example: Adding a comment to a post

Imagine a relationship where Post has many comments and you want to add a
new comment to an existing post. While it is possible to use put_assoc/4
for this, it would be unecessarily complex. Let’s see an example.

The reason why the example above is wrong is because put_assoc/4 always
works with the full data. So the example above will effectively erase
all previous comments and only keep the comment you are currently adding.
Instead, you could try:

In this example, we prepend the new comment to the list of existing comments.
Ecto will diff the list of comments currently in post with the list of comments
given, and correctly insert the new comment to the database. Note, however,
Ecto is doing a lot of work just to figure out something we knew since the
beginning, which is that there is only one new comment.

In cases like above, when you want to work only on a single entry, it is
much easier to simply work on the associated directly. For example, we
could instead set the post association in the comment:

In other words, when you find yourself wanting to work only with a subset
of the data, then using put_assoc/4 is most likely unnecessary. Instead,
you want to work on the other side of the association.

Example: Adding tags to a post

Imagine you are receiving a set of tags you want to associate to a post.
Let’s imagine that those tags exist upfront and are all persisted to the
database. Imagine we get the data in this format:

params=%{"title"=>"new post","tags"=>["learner"]}

Now, since the tags already exist, we will bring all of them from the
database and put them directly in the post:

tags=Repo.all(fromtinTag,where:t.namein^params["tags"])post|>Repo.preload(:tags)|>Ecto.Changeset.cast(params,[:title])# No need to allow :tags as we put them directly|>Ecto.Changeset.put_assoc(:tags,tags)# Explicitly set the tags

Since in this case we always require the user to pass all tags
directly, using put_assoc/4 is a great fit. It will automatically
remove any tag not given and properly associate all of the given
tags with the post.

Furthermore, since the tag information is given as structs read directly
from the database, Ecto will treat the data as correct and only do the
minimum necessary to guarantee that posts and tags are associated,
without trying to update or diff any of the fields in the tag struct.

Although it accepts an opts argument, there are no options currently
supported by put_assoc/4.

key is an atom that represents any field, embed or
association in the changeset. Note the value is directly
stored in the changeset with no validation whatsoever.
For this reason, this function is meant for working with
data internal to the application.

If the change is already present, it is overridden with
the new value. If the change has the same value as in the
changeset data, it is not added to the list of changes.

When changing embeds and associations, see put_assoc/4
for a complete reference on the accepted values.

Optionally function can accept three arguments: changeset, field
and error tuple {msg, opts}. It is useful whenever you want to extract
validations rules from changeset.validations to build detailed error
description.

The unique constraint works by relying on the database to check
if the unique constraint has been violated or not and, if so,
Ecto converts it into a changeset error.

In order to use the uniqueness constraint, the first step is
to define the unique index in a migration:

createunique_index(:users,[:email])

Now that a constraint exists, when modifying users, we could
annotate the changeset with a unique constraint so Ecto knows
how to convert it into an error message:

cast(user,params,[:email])|>unique_constraint(:email)

Now, when invoking Repo.insert/2 or Repo.update/2, if the
email already exists, it will be converted into an error and
{:error, changeset} returned by the repository. Note that the error
will occur only after hitting the database so it will not be visible
until all other validations pass.

Options

:message - the message in case the constraint check fails,
defaults to “has already been taken”

:name - the constraint name. By default, the constraint
name is inferred from the table + field. May be required
explicitly for complex cases

:match - how the changeset constraint name is matched against the
repo constraint, may be :exact or :suffix. Defaults to :exact.
:suffix matches any repo constraint which ends_with?:name
to this changeset constraint.

Complex constraints

Because the constraint logic is in the database, we can leverage
all the database functionality when defining them. For example,
let’s suppose the e-mails are scoped by company id. We would write
in a migration:

createunique_index(:users,[:email,:company_id])

Because such indexes have usually more complex names, we need
to explicitly tell the changeset which constraint name to use (here we’re
using the naming convention that unique_index uses):

Notice that the first param is just one of the unique index fields, this will
be used as the error key to the changeset errors keyword list. For example,
the above unique_constraint/3 would generate something like:

Alternatively, you can give both unique_index and unique_constraint
the same name:

# In the migrationcreateunique_index(:users,[:email,:company_id],name::users_email_company_id_index)# In the changeset functioncast(user,params,[:email])|>unique_constraint(:email,name::users_email_company_id_index)

Case sensitivity

Unfortunately, different databases provide different guarantees
when it comes to case-sensitiveness. For example, in MySQL, comparisons
are case-insensitive by default. In Postgres, users can define case
insensitive column by using the :citext type/extension. In your migration:

execute"CREATE EXTENSION IF NOT EXISTS citext"createtable(:users)do...add:email,:citext...end

If for some reason your database does not support case insensitive columns,
you can explicitly downcase values before inserting/updating them:

Validates that no existing record with a different primary key
has the same values for these fields.

This function exists to provide quick feedback to users of your
application. It should not be relied on for any data guarantee as it
has race conditions and is inherently unsafe. For example, if this
check happens twice in the same time interval (because the user
submitted a form twice), both checks may pass and you may end-up with
duplicate entries in the database. Therefore, a unique_constraint/3
should also be used to ensure your data won’t get corrupted.

However, because constraints are only checked if all validations
succeed, this function can be used as an early check to provide
early feedback to users, since most conflicting data will have been
inserted prior to the current validation phase.

Examples

unsafe_validate_unique(changeset,[:email],repo)unsafe_validate_unique(changeset,[:city_name,:state_name],repo)unsafe_validate_unique(changeset,[:city_name,:state_name],repo,message:"city must be unique within state")unsafe_validate_unique(changeset,[:city_name,:state_name],repo,prefix:"public")

The given function is invoked with the change value only if there
is a change for the given key. Note that the value of the change
can still be nil (unless the field was marked as required on validate_required/3).

It invokes the validator function to perform the validation
only if a change for the given field exists and the change
value is not nil. The function must return a list of errors
(with an empty list meaning no errors).

In case there’s at least one error, the list of errors will be appended to the
:errors field of the changeset and the :valid? flag will be set to
false.

Similar to validate_change/3 but stores the validation metadata
into the changeset validators. The validator metadata is often used
as a reflection mechanism, to automatically generate code based on
the available validations.

Validates that the given field matches the confirmation
parameter of that field.

By calling validate_confirmation(changeset, :email), this
validation will check if both “email” and “email_confirmation”
in the parameter map matches.

Note that if the confirmation field is nil or missing, by default this does
not add a validation error. You can specify that the confirmation field is
required in the options (see below). Note “email_confirmation” does not need
to be added as a virtual field in your schema.

Options

:message - the message on failure, defaults to “does not match confirmation”

Note that the length of a string is counted in graphemes by default. If using
this validation to match a character limit of a database backend,
it’s likely that the limit ignores graphemes and limits the number
of unicode characters. Then consider using the :count option to
limit the number of codepoints (:codepoints), or limit the number of bytes (:bytes).

Options

:is - the length must be exactly this value

:min - the length must be greater than or equal to this value

:max - the length must be less than or equal to this value

:count - what length to count for string, :graphemes (default), :codepoints or :bytes

:message - the message on failure, depending on the validation, is one of:

You can pass a single field name or a list of field names that
are required.

If the value of a field is nil or a string made only of whitespace,
the changeset is marked as invalid, the field is removed from the
changeset’s changes, and an error is added. An error won’t be added if
the field already has an error.

If a field is given to validate_required/3 but it has not been passed
as parameter during cast/3 (i.e. it has not been changed), then
validate_required/3 will check for its current value in the data.
If the data contains an non-empty value for the field, then no error is
added. This allows developers to use validate_required/3 to perform
partial updates. For example, on insert all fields would be required,
because their default values on the data are all nil, but on update,
if you don’t want to change a field that has been previously set,
you are not required to pass it as a paramater, since validate_required/3
won’t add an error for missing changes as long as the value in the
data given to the changeset is not empty.

Do not use this function to validate associations are required,
instead pass the :required option to cast_assoc/3.

Opposite to other validations, calling this function does not store
the validation under the changeset.validations key. Instead, it
stores all required fields under changeset.required.

Options

:message - the message on failure, defaults to “can’t be blank”

:trim - a boolean that sets whether whitespaces are removed before
running the validation on binaries/strings, defaults to true