The Cognitive Foundry is a modular Java software library for the research and development of cognitive systems. It is primarily designed for research and development to be easy to plug into applications to provide adaptive behaviors.

The main part of the Foundry is the Machine Learning package, which contains reusable components and algorithms for machine learning and statistics. It contains many algorithms for supervised and unsupervised learning as well as statistical modeling. It is interface-centric and uses generics to make it easy to customize to the needs of individual applications.

The Cognitive Foundry's development is led by Sandia National Laboratories and is released under the open source BSD License. It requires Java 1.6.

Changes to previous version:

Common Core:

Added NumericMap interface, which provides a mapping of keys to numeric
values.

Added ScalarMap interface, which extends NumericMap to provide a mapping
of objects to scalar values represented as doubled.

Added AbstractScalarMap and AbstractMutableDoubleMap to provide abstract,
partial implementations of the ScalarMap interface.

Added VectorSpace interface, where a VectorSpace is a type of Ring that
you can perform Vector-like operations on such as norm, distances, etc.

Added AbstractVectorSpace, which provides an abstract, partial
implementation of the VectorSpace interface.

Updated Vector, AbstractVector, VectorEntry to build on new VectorSpace
interface and AbstractVectorSpace class.

Added InfiniteVector interface, which has a potentially infinite number
of indices, but contains only a countable number in any given instance.

Added DefaultInfiniteVector, an implementation of the InfiniteVector
interface backed by a LinkedHashMap.

Rewrote FiniteCapacityBuffer from the ground up, now with backing from a
fixed-size array to minimize memory allocation.

Renamed IntegerCollection to IntegerSpan.

Learning Core:

Updated ReceiverOperatingCharacteristic to improve calculation

Added PriorWeightedNodeLearner interface, which provides for configuring the
prior weights on the learning algorithm that searches for a decision
function inside a decision tree.

Updated AbstractDecisionTreeNode to fix off by one error in computing node's
depth.

Made LogNumber into a signed version of a log number and moved the prior
unsigned implementation into UnsignedLogNumber.

Added EuclideanRing interface that provides methods for times,
timesEquals, divide, and divideEquals. Also added Field interface that
provides methods for inverse and inverseEquals. These interfaces are now
implemented by the appropriate number classes such as ComplexNumber,
MutableInteger, MutableLong, MutableDouble, LogNumber, and
UnsignedLogNumber.

Added interface for Indexer and DefaultIndexer implementation for
creating a zero-based indexing of values.

Added interfaces for MatrixFactoryContainer and
DivergenceFunctionContainer.

Added ReversibleEvaluator, which various identity functions implement as
well as a new utility class ForwardReverseEvaluatorPair to create a
reversible evaluator from a pair of other evaluators.

Added method to create an ArrayList from a pair of values in
CollectionUtil.

ArgumentChecker now properly throws assertion errors for NaN values.
Also added checks for long types.

Fixed handling of Infinity in subtraction for LogMath.

Fixed issue with angle method that would cause a NaN if cosine had a
rounding error.

Added new createMatrix methods to MatrixFactory that initializes the
Matrix with the given value.

Added copy, reverse, and isEmpty methods for several array types to
ArrayUtil.

Added utility methods for creating a HashMap, LinkedHashMap, HashSet, or
LinkedHashSet with an expected size to CollectionUtil.

Added getFirst and getLast methods for List types to CollectionUtil.

Removed some calls to System.out and Exception.printStackTrace.

Common Data:

Added create method for IdentityDataConverter.

ReversibleDataConverter now is an extension of ReversibleEvaluator.

Learning Core:

Added general learner transformation capability to make it easier to adapt
and compose algorithms. InputOutputTransformedBatchLearner provides this
capability for supervised learning algorithms by composing together a
triplet. CompositeBatchLearnerPair does it for a pair of algorithms.

Added a constant and identity learners.

Added Chebyshev, Identity, and Minkowski distance metrics.

Added methods to DatasetUtil to get the output values for a dataset and
to compute the sum of weights.

Made generics more permissive for supervised cost functions.

Added ClusterDistanceEvaluator for taking a clustering that encodes the
distance from an input value to all clusters and returns the result as a
vector.

Fixed potential round-off issue in decision tree splitter.

Added random subspace technique, implemented in RandomSubspace.

Separated functionality from LinearFunction into IdentityScalarFunction.
LinearFunction by default is the same, but has parameters that can change
the slope and offset of the function.

Default squashing function for GeneralizedLinearModel and
DifferentiableGeneralizedLinearModel is now a linear function instead of
an atan function.

Added a weighted estimator for the Poisson distribution.

Added Regressor interface for evaluators that are the output of
(single-output) regression learning algorithms. Existing such evaluators
have been updated to implement this interface.

Added support for regression ensembles including additive and averaging
ensembles with and without weights. Added a learner for regression bagging
in BaggingRegressionLearner.

Added checkedAdd and checkedMultiply functions to MathUtil, providing a
means for conducting Integer addition and multiplication with explicit
checking for overflow and underflow, and throwing an ArithmeticException
if they occur. Java fails silently in integer over(under)flow situations.

Added explicit integer overflow checks to DenseMatrix. The underlying MTJ
library stores dense matrices as a single dimensional arrays of integers,
which in Java are 32-bit. When creating a matrix with numRows rows and
numColumns columns, if numRows * numColumns is more than 2^31 - 1, a
silent integer overflow would occur, resulting in later
ArrayIndexOutOfBoundsExceptions when attempting to access matrix elements
that didn't get allocated.

Added new methods to DiagonalMatrix interface for multiplying diagonal
matrices together and for inverting a DiagonalMatrix.

Optimized operations on diagonal matrices in DiagonalMatrixMTJ.

Added checks to norm method in AbstractVectorSpace and DefaultInfiniteVector
for power set to NaN, throwing an ArithmeticException if encountered.

Learning Core:

Optimized matrix multiplies in LogisticRegression to avoid creating dense
matrices unnecessarily and to reduce computation time using improved
DiagonalMatrix interfaces.

Added KernelizableBinaryCategorizerOnlineLearner interface for an online
linear binary categorizer that can also be used with a kernel. Also
provided abstract class AbstractKernelizableBinaryCategorizerOnlineLearner
and AbstractLinearCombinationOnlineLearner for common functionality.

Moved KernelPerceptron and KernelAdatron to the new
learning.perceptron.kernel package.

Upgraded to mtj-9.9.14 and added the netlib-java-0.9.3 library, which MTJ
now depends on.

Common Core:

ParallelUtil: Added executeInParallel(tasks, algorithm)

CollectionUtil: Added removeElement.

Added MutableDouble class, which is like Double but with a mutable value.
It also implements Ring and Vectorizable.

UnivariateStatisticsUtil: Improved stability of computeMean with large
values.

VectorReader: Changed to use a Collection of tokens instead of requiring
an ArrayList.

AbstractSingularValueDecomposition: Fixed a bug for certain types of
rectangular matrices.

DiagonalMatrixMTJ: Made inverse faster.

ArgumentChecker: Added assertIsInRangeExclusive and fixed a formatting
issue with some exception messages.

Added Identified interface.

Learning Core:

Added BatchAndIncrementalLearner interface, which is for algorithms that
can be used in batch and incremental modes. Note that it defines that the
BatchLearner interface uses a Collection of data for consistency with
existing batch (non-incremental) algorithms. However, it does also include
another learn method that takes an Iterable.
AbstractBatchAndIncrementalLearner now implements it.

VectorNaiveBayesCategorizer: Now has generic for the distribution type
representing each feature, including OnlingBaggingCategorizerLearner,
AbstractOnlineLinearBinaryCategorizerLearner,
OnlinePassiveAggressivePerceptron, OnlinePerceptron, OnlineVotedPerceptron,
and Winnow.

BaggingCategorizerLearner: Added a protected fillBag method, which
can be overridden to implement a different sampling approach.

Added BatchMultiPerceptron, which is an implementation of a multi-class
Perceptron that keeps one Perceptron per class.

Added MultiCategoryAdaBoost, which is an implementation of the AdaBoost.M1
algorithm.

DecisionTree: Added findTerminalNode methods.

CrossFoldCreator: Added constructor that takes just a number of folds.

KernelBinaryCategorizer is now an interface with a default implementation
in DefaultKernelBinaryCategorizer. KernelPerceptron, KernelAdatron,
SequentialMinimalOptimization, and SuccessiveOverrelaxation now all
use the new interface or default class.

Added LinearMultiCategorizer class, which keeps a LinearBinaryCategorizer
for each class and

KDTree: Fixed two bugs that caused the KDTree to not always return the
nearest points in rare cases.

Learning Core:

PartitionalClusterer: Added an implementation of a partitional clustering
algorithm.

IncrementalClusterCreator: New interface for a cluster creator that can
incrementally update clusters. DefaultIncrementalClusterCreator is a
default implementation of the interface that just updates the cluster
memberships.

VectorMeanCentroidClusterCreator: Modified to implement the new
IncrementalClusterCreator interface.

BinaryBaggingLearner: Generalized generics in constructor.

OnlineBaggingCategorizerLearner: Added an implementation of an online
version of the bagging algorithm for building ensembles.

VotingCategorizerEnsemble: Added an unweighted voting categorization
ensemble as a counterpart to WeightedVotingCategorizerEnsemble.

OnlinePassiveAggressivePerceptron: Added an implementation of the
Passive-Aggressive algorithm for binary classification. Also contains
PA-I (LinearSoftMargin) and PA-II (QuadraticSoftMargin) variants.

Added DiscriminantCategorizer and BinaryDiscriminantCategorizer interfaces.
Almost all Categorizers now implement one of these interfaces; some
existing methods were renamed to match these standard interfaces.

Added DistanceSamplingClusterInitializer, which is an implementation of
the k-means++ initialization algorithm.

HiddenMarkovModel and BaumWelchAlgorithm had some performance improvements.

Added KernelPrincipalComponentsAnalysis, which is an implementation of
kernel PCA using eigen-decomposition.

DatasetUtil: Added countOutputValues.

TargetEstimatePair and WeightedTargetEstimatePair are now interfaces;
existing implementations moved to DefaultTargetEstimatePair and
DefaultWeightedTargetEstimatePair.

ConstantEvaluator: Now extends AbstractCloneableSerializable.

ThresholdBinaryCategorizer is now an interface. Existing implementation
moved to ScalarThresholdBinaryCategorizer.

AbstractThresholdBinaryCategorizer now longer contains the threshold
directly, instead it assumes the threshold is applied in evaluateAsDouble
and categorizes based on zero. This is to remove the confusion of having
both a bias and a threshold in some classes and to clarify the meaning of
evaluateAsDouble so that it matches the new BinaryDiscriminantCategorizer
interface.

Made new ConfusionMatrix and BinaryConfusionMatrix interfaces and put them
in the gov.sandia.cognition.learning.performance.categorization. Refactored
existing confusion matrix code to implement these interfaces; the existing
class ending up in DefaultBinaryConfusionMatrix. New confusion matrix
interface uses generics to be useful for more than just binary
categorization.