History: the very first version was created by John Backus, and shortly
after improved by Peter Naur, and it was this improved version that was
publicly used for the first time, to define Algol 60.
[Naur P (ed.), 1963, Revised report on the algorithmic
language Algol 60, Comm. ACM 6:1 pp1-17]Compilers: Principles, Techniques, and Tools,
by Aho, Sethi and Ullman says:
"The proposal that BNF, which began as an abbreviation of Backus Normal
Form, be read as Backus-Naur Form, to recognise Naur's contributions ...
is contained in a letter by Knuth
[Knuth D.E., 1964, Backus Normal Form vs. Backus Naur Form,
Comm. ACM 7:12 pp735-736]"Knuth's letter
(which you may be able to access via
ACM)
is interesting to read, as it indicates exactly what he
thought the importance concepts in BNF were.
He also points out that BNF is not a "Normal Form",
which would imply there were some restrictions on how a grammar could be
written down, as in e.g.
Chomsky Normal Form
or
Greibach Normal Form.
There is a
message
archived from the comp.compilers newsgroup that gives a different view of
Naur's contribution.

which can be read as something like:
"a number is a digit, or any number followed by an extra digit"
(which is a contorted but precise way of saying that a number
consists of one or more digits)
"a digit is any one of the characters 0, 1, 2, ... 9"

There are many variants, all equivalent to the original definition, such as:

Every variant has to distinguish between the names of grammar rules, such as
number,
and actual characters that can appear in the text of a program, such as
0123456789.
Usually, one or the other is quoted, e.g.
<number>
or
'0123456789' (or "0123456789")
so the other can be left unquoted.
Sometimes both are quoted, as in the last example above.

However, if we decide to quote the names of rules and leave the actual
characters unquoted (as in the first example above) a
problem can arise if the metacharacters (i.e. the characters such as
' < > . | used to punctuate the rules) can appear as
actual characters in the programming language. Therefore, the
most widely used variants of BNF usually quote the actual characters e.g.:

Unlike BNF, this kind of notation does not seem to have a commonly agreed-on
name. "Syntax diagrams" are also known as "Railway Tracks" or "Railroad
Diagrams".
Whatever they are called, they do not allow us to write anything that can't be
written in BNF, they just make the grammar easier to understand.

Like syntax diagrams, EBNF does not allow us to write anything that can't
be written in BNF, it just makes the grammar easier to understand.

History: many extensions to BNF were used to define computer languages
after Algol 60, all slightly different. Niklaus Wirth proposed a single
formulation in:
"What Can We Do About the Unnecessary Diversity of Notation
for Syntactic Definitions, November 1977, Comm. ACM 20:11 pp. 822-823"
However, this did not actually mention "EBNF".
You may be able to access Wirth's article via
ACM.
Alternatively, you can see most of the details of his proposal
here
- look for section 5.9.1

I don't know of a universally accepted set of extensions to BNF,
but here are some attempts:

ISO
has a standard for EBNF notation, which you can read about
here
or
here
or
here
(as I understand it, these are/point to the final draft version
rather than the actual standard itself)
and
here
(includes information about BNF and the ISO and BSI versions of EBNF
- the BSI standard has been withdrawn, presumably because the ISO
standard replaces it).

(It is unfortunate that the limited number of different kinds of brackets
seems to limit the number of concepts that can be added to this style - e.g.
we can't easily express "1 or more times". The whole point of EBNF is to make
writing grammars simpler!)

(This seems to me to be, technically, the most complete and therefore useful
style, although perhaps the most ugly!)

Note that e.g.
x* means the same as (x+)? ,
and x+ the same as xx* ,
and x? the same as (x|) ,
so it is possible, but not as simple, to use a subset of these facilities.
Given that the only reason for using EBNF instead of BNF is to simplify and
clarify language descriptions, it would seem sensible to provide all these
facilities, and maybe others as well (e.g. see the note about list separators below).

One further point - if we really want to minimise the amount of work
we do and make the grammar as clear as possible, then we need to deal
with the problem of list separators:

We can recognise lists where every item is identical,
like "a ; b ; c ;" using:
( name ';' )+[here ';' acts as a list-item terminator]
but the only way to recognise lists where the items are separated from
each other, like "a , b , c" is:
name ( ',' name )*
or equivalently:
( name ',' )* name[here ',' acts as a list-item separator, and a
different marker, such as ';', would normally be used to
terminate the list]
so we end up with something that is harder to use (we need to deal with
names in two different places rather than one), longer to type
and - most important - less easy to understand.
You can see 9 examples in
ANSI C syntax from K&R using an EBNF.

We really need a notation where we can say:
a list of one or more names separated by ','
I don't know of a "natural" notation for this, but we need to be able to
write something like:
( name # ',' )+
Unfortunately, most EBNFs do not include a notation for this concept.
As an example of this occuring in "real life", here is an
ANSI C syntax using # for list separators.