This paper is concerned with ill-formed language. We distinguish two types of ill-formed language: string errors which can be explained by modifications to the string generated by the grammar, and grammar errors which can be explained by a modification of the grammar which generates the string. A chart-based method for dealing with string errors is presented. We discuss the adequacy of different grammatical formalisms for dealing with grammar errors, and show that context-free grammars are not expressive enough, but that constraint-based grammars are more suitable, if absolute and non-absolute constraints are distinguished. We augment non-absolute constraints in these grammars with probabilities. The relationship between the grammaticality of a string and its probability is discussed.