Visitors to a website may want to perform certain actions related to Entities (specific places or people or things) that are displayed to them on the Web.

For example, at a page for a restaurant (an entity), a person viewing the site may want to create a reservation or get driving directions to the restaurant from their current location. Doing those things may require a person to take a number of steps, such as selecting the name of the restaurant and copying it, pasting that information into a search box, and submitting it as a search query, selecting the site from search results, determining if making a reservation is possible on the site, and then providing information necessary to make a reservation; getting driving directions may also require multiple steps.

Using a touch screen device may potentially be even more difficult because the site would possibly then be limited to touch input.

A patent granted to Google this week describes a way to easily identify an entity such as a restaurant on a touch device, and select it online and take some action associated with that entity based upon the context of a site the entity is found upon. Actions such as booking a reservation at a restaurant found on a website, or procuring driving directions to that site, or other actions could be easily selected by the user of a site.

The patent is:

Semantic selection and purpose facilitation

Inventors: Paul Nordstrom, Casey Stuart Whitelaw,

Assignee: Google

US Patent 9,305,108

Granted April 5, 2016

Filed: October 5, 2012

Abstract

Computer-implemented methods for proposing actions to a user to select based on the users predicted purpose for selecting content are provided. In one aspect, a method includes receiving an identifier of a referent entity associated with user-selectable content, identifying, based on a prediction of a purpose in selecting the content, at least one action to be executed that is associated with the entity, and providing, for display, at least one identifier of the at least one action to the device for selection by a user. Systems, graphical user interfaces, and machine-readable media are also provided.

How an entity and actions might be selected by a site visitor

A person searches for a site using text such as sushi restaurants in Mountain View. That person then circles the text we love Ramen Sushi out of all of the places weve been to on the web page they found with that search, by circling the text using a touch input. Based on the content they chose and the context of their selection of that text, The system decides that viewer of the page has selected Ramen Sushi, and it proposes that entity to the user. The user can confirm that, and is then given a number of actions to perform on the entity based on a context of that selection.

Someone circles an entity on a touch screen to perform actions on it.

The context can include:

The current location of the deviceA past location of the deviceThe type of the device A previous action associated with the entity taken by the user or another userA search queryInformation on another user associated with the userThe file from which the user-selectable content was selectedThe remaining content from which the user-selectable content was selected

Entities are contained in an entity database, which may contain attributes or properties associated with the entity, and those can be pre-defined, and can have associated descriptors such as location, restaurant, and phone number. An entity that is a person such as George Washington can have an associated descriptor notable person.

The patent tells us that entities that are listed in the entity database can be associated with one or many user purposes and/or actions based on an associated descriptor.

A purpose is something that a user would want to do or find out with respect to an entity that selected. These actions are shown in a menu to the user as choices of actions to take regarding selected entities. These purposes may be referred to as a task. The patent provides a number of examples, that include:

play (e.g. for games and sports), rate or evaluate, travel to, contact, communicate, share, record, remember, dine, consume, experience or enjoy (e.g. art, music), reserve (tickets, etc.), compare, learn, study, understand, purchase, repair, fix, teach, cook, and make. For the example purpose dine, an example sub-purpose can be eat dinner, from which example sub-purposes can be make reservation, get directions, and find parking.

The patent tells us that users can select multiple entities of the same type at the same time to compare them.

Entities, purposes, and actions can be added to the entity database either manually or automatically with a user (or even an owner of the entity) adding information. The patent provides some examples of how information might be added to the entity database, but it seems to be fairly wide open under the patent.

The patent doesnt mention Schema vocabulary, which would be one way for a site owner to add entity information to an entity database.

Entities may be products, and actions presented to a user could include providing a review of the product, identifying a seller of the product, providing a price for the product, or providing an offer (e.g., discount or coupon) associated with the product. If the entity is a service, such as watching a movie or a plumber for hire, the actions that may be presented to the user could include providing a review of the service, identifying an availability of the service (e.g., show times), identifying a location where the service is being provided (e.g., an address of the plumber), or providing an option to purchase the service (e.g., purchasing tickets for the movie or rates offered by the plumber).

Take Aways

The entity database described in this patent could be a very big one, containing multiple businesses (like those from Google Maps), multiple products, multiple people (like those found at a knowledge base like Wikipedia), and multiple potential actions and tasks associated with those entities.

This seems to be a fairly aspirational patent, which might require a lot of steps being put into place before it is implemented. It does present a vision of how entities on the web could eventually be acted upon by people who see them in web pages.

This could be something that Google may intend to do, and some of the pieces for it are in place, such as a knowledge graph filled with entities, and a schema system that is extendable. Its interesting seeing a patent that lays out a framework like this one does. Is this a future path that Google will follow? We may need to wait to see.