OpenStack scheduler has nice options like filters which allow resource selection based on simple constraints e.g. Don't put instances in a set on the same host. However for complex constraints, building a filter would be as complex as building a real constraint solver e.g. place VMs while minimizing average (VM-storage-bandwidth) with complex constraints. On the other hand, complex solvers are available in open source (PULP, CVXOPT, COIN_OR). Hence a natural solution to the above problem is to design a pluggable scheduler that leverages existing solvers. We believe that this will open new avenues for complex constraint (and objectives) based resource placement in large OpenStack deployments.

Initial performance benchmarks have shown this driver to perform better than the default Filter Scheduler.

This new driver will provide a better alternative to the default scheduler driver.

Related branches

Related bugs

Sprints

Whiteboard

Added the nova-spec: https://review.openstack.org/96543 for review. A lot of code patches were submitted as part of the Icehouse cycle, but missed the deadline last time. Would appreciate your review. --Yathi.

Deferred to icehouse-3 as the blueprint was not approved by the icehouse-2 blueprint approval deadline. --russellb

Addressed by: https://review.openstack.org/#/c/46588/
Add a pluggable solver scheduler
This provides a solver scheduler, where complex constraints can be specified using pluggable solvers. An illustrative implementation of a solver is included and is based on PULP. Here the constraints are based on the disk space and memory requirements, and optimization is done using the host's free ram as a cost metric.

I'm not sure we want to add this right now since we're in the middle of making plans to split the scheduler out into a new repo. --russellb

russellb, why should the splitting out the scheduler effort block this? -- jogo

jogo, Can you please review this blueprint and get it approved for the next icehouse milestone target? The latest code patch now passes the JENKINS build and provides the base solver_scheduler driver and a reference implementation of a solver that is based on PULP. This patch is now ready for review. --Thanks, Yathi.

Apologies, this missed the deadline for Feature Freeze. Please rebase patches as soon as Juno opens, and we will try to get this in during that period. --johnthetubaguy (5th March 2014)

However seems like some of the approaches need to be discussed more, and we need a much clearer blueprint, with steps on what is being done, so we can try and stop blocking these changes after so much work has been done. Un-approving this one. Need more details on what configuration values will be added or altered, and how exisiting filters and constraints will be consolidated. --johnthetubaguy

-------
[Yathi - March 5 2014]
We would like to make a request for FFE for the Solver Scheduler work. A lot of work has gone into it since Sep’13, and the first patch has gone through several iteration after some reviews. The first patch - https://review.openstack.org/#/c/46588/ introduces the main solver scheduler driver, and a reference solver implementation, and the subsequent patches that are already added provide the pluggable solver, and individual support for adding constraints, costs, etc.

John,
Regarding the concerns you raised above to make this BP "pending Approval", the plan is not many new configuration values will be added, just the ones to specify the solver to use, and the pluggable constraints, and costs to use, with the weights for the costs. (these are mainly part of the second patch - https://review.openstack.org/#/c/70654/ Something similar to FilterScheduler, where the filters to be used are specified)

The plan is to gradually support the concepts for the existing filters as the constraints that are accepted by our Solver Scheduler. Depending on the constraints and the costs chosen, the final scheduling will be done by solving the problem as an optimization problem.