Secondary Navigation

Committees for 08-09 disk

Currently 176 games into a 07- 08 Replay using the actual dressed lineups for each NHL team and I have to say last year s disk was extremely well done.

Message 1 of 19
, Apr 6, 2009

Currently 176 games into a '07-'08 Replay using the actual dressed lineups foreach NHL team and I have to say last year's disk was extremely well done. Scores, penalties called, etc. are very realistic. Only noticed a few tweakshere and there that could be made to the disk. Considering the number ofplayers on the disk, it's a testament to the great work Dave, Herb & the committees put into the disk.

With a week remaining in the '08-'09 NHL schedule, I'm wondering if thecommittees for each team are ready to begin the rating process startingnext week. That was our one fault last year, in not having the committees in place and active right after the NHL season ended. Hopefully, that won't bethe case this year and we'll produce another fantastic disk in a moretimely manner.

sylvain

I was part of Anaheim comittee and I have never received any news from the other 2 guys... I know Philadelphia (my other comittee) still corresponding ...

Message 2 of 19
, Apr 6, 2009

I was part of Anaheim comittee and I have never
received any news from the other 2 guys...

Currently 176 games into a '07-'08 Replay using the
actual dressed lineups foreach NHL team and I have to say last
year's disk was extremely well done. Scores, penalties called, etc.
are very realistic. Only noticed a few tweakshere and there that
could be made to the disk. Considering the number ofplayers on the
disk, it's a testament to the great work Dave, Herb & the
committees put into the disk.

With a week remaining in the
'08-'09 NHL schedule, I'm wondering if thecommittees for each team
are ready to begin the rating process startingnext week. That was
our one fault last year, in not having the committees in place
and active right after the NHL season ended. Hopefully, that won't
bethe case this year and we'll produce another fantastic disk in a
moretimely manner.

Herb Garbutt

Hey guys, We ve decided to go with a slightly different approach this season but the ratings groups will still be involved. Here s what we re doing. I am going

Message 3 of 19
, Apr 7, 2009

Message

Hey
guys,

We've
decided to go with a slightly different approach this season but the ratings
groups will still be involved.

Here's
what we're doing. I am going to rate the entire league and then send those
ratings to the groups for discussion. I know this probably worries some of you,
but to (hopefully) ease your concerns, here is why we believe this process will
work better.

1/
Consistency: Last year we had 30 different groups each using their own
criteria. Some groups mandated that there had to be a certain number of 1s,
2s, 3s etc., some gave out very few 2s and 1s, some gave fringe players 2s
across the board, other gave them 1s. That is not to say anyone was wrong.
They were right using their criteria. But with so many different criteria, there
wasn't much consistency from team to team. By having one rater, throughout, we
can eliminate these issues and then use the groups to correct/tweak
ratings.

2/
Timing. The committees were actually in place by about February last year. Some
started right away, some waited until the end of the season, some didn't start
rating until July, some just never happened. I was also encouraging groups to
start ratings as far back as February so they could evaluate while games were
still being played. Unfortunately, by the end of the playoffs probably 2/3
of the groups had not started ratings.

We're
hoping that worst case, if groups are slow to respond, at least we'll have my
baseline to work from and Dave can start testing much, much earlier. I'm hoping
end of June, beginning of July, which will lead to us getting the disk much
earlier and Dave actually being able to enjoy some of his
summer.

In
doing my ratings, I am using last year's ratings as a baseline. If there is no
evidence to support a particular rating, or I don't have a strong opinion,
I'm using last year's rating. So no, I'm not simply throwing out all of last
year's hard work. It worked well in testing so it's a great place to start from.
In this regard, the groups will be especially important on
rookies.

I have
rated about 350 players so far. I hope to have the league finished by the end of
the second round of the playoffs...at which point, we'll send the ratings out to
the different groups for their opinions.

Currently 176 games into a '07-'08 Replay using the
actual dressed lineups foreach NHL team and I have to say last
year's disk was extremely well done. Scores, penalties called, etc.
are very realistic. Only noticed a few tweakshere and there that
could be made to the disk. Considering the number ofplayers on the
disk, it's a testament to the great work Dave, Herb & the commit
tees put into the disk.

With a week remaining in the '08-'09 NHL
schedule, I'm wondering if thecommittees for each team are ready to
begin the rating process startingnext week. That was our one fault
last year, in not having the committees in place and active
right after the NHL season ended. Hopefully, that won't bethe case
this year and we'll produce another fantastic disk in a moretimely
manner.

Bill Corfield

Hey Herb- ? I respect your decision to go the route you are, but I have some concerns...Its important to me you know this isn t personal, but rather voiced

Message 4 of 19
, Apr 7, 2009

Hey Herb-
I respect your decision to go the route you are, but I have some concerns...Its important to me you know this isn't personal, but rather voiced only with the hope for the best disk possible.

There was an impression amongst some in the apba community last time where "Herb won all ties." That is, in the vast majority of cases where a ratings team or representative disagreed with you on a rating, your ratings usually prevailed. I understand the impossible nature of trying to please everyone and still generate an accurate disk. Its also a given the respect you have across the apba community for the work you put in on this.

The first thought I had when I read your mail was this. Why doesn't Herb just do the disk? Period...

Leagues have and are always free to make any edits they chose to, so you work up the ratings the way you believe are best, see how they survive the test sims, make your adjustments then send it out. If a league doesn't like it, there's a zillion different systems they can implement to allow tweaking within in their league.

I wonder if this approach wouldn't take us to the same destination they we're bound for using the approach your detailed in your email. It might be significantly quicker and easier for all parties involved...

We've
decided to go with a slightly different approach this season but the ratings
groups will still be involved.

Here's
what we're doing. I am going to rate the entire league and then send those
ratings to the groups for discussion. I know this probably worries some of you,
but to (hopefully) ease your concerns, here is why we believe this process will
work better.

1/
Consistency: Last year we had 30 different groups each using their own
criteria. Some groups mandated that there had to be a certain number of 1s,
2s, 3s etc., some gave out very few 2s and 1s, some gave fringe players 2s
across the board, other gave them 1s. That is not to say anyone was wrong.
They were right using their criteria. But with so many different criteria, there
wasn't much consistency from team to team. By having one rater, throughout, we
can eliminate these issues and then use the groups to correct/tweak
ratings.

2/
Timing. The committees were actually in place by about February last year. Some
started right away, some waited until the end of the season, some didn't start
rating until July, some just never happened. I was also encouraging groups to
start ratings as far back as February so they could evaluate while games were
still being played. Unfortunately, by the end of the playoffs probably 2/3
of the groups had not started ratings.

We're
hoping that worst case, if groups are slow to respond, at least we'll have my
baseline to work from and Dave can start testing much, much earlier. I'm hoping
end of June, beginning of July, which will lead to us getting the disk much
earlier and Dave actually being able to enjoy some of his
summer.

In
doing my ratings, I am using last year's ratings as a baseline. If there is no
evidence to support a particular rating, or I don't have a strong opinion,
I'm using last year's rating. So no, I'm not simply throwing out all of last
year's hard work. It worked well in testing so it's a great place to start from.
In this regard, the groups will be especially important on
rookies.

I have
rated about 350 players so far. I hope to have the league finished by the end of
the second round of the playoffs...at which point, we'll send the ratings out to
the different groups for their opinions.

Currently 176 games into a '07-'08 Replay using the
actual dressed lineups for
each NHL team and I have to say last
year's disk was extremely well done.
Scores, penalties called, etc.
are very realistic. Only noticed a few tweaks
here and there that
could be made to the disk. Considering the number of
players on the
disk, it's a testament to the great work Dave, Herb & the
commit
tees put into the disk.

With a week remaining in the '08-'09 NHL
schedule, I'm wondering if the
committees for each team are ready to
begin the rating process starting
next week. That was our one fault
last year, in not having the committees
in place and active
right after the NHL season ended. Hopefully, that won't be
the case
this year and we'll produce another fantastic disk in a more
timely
manner.

Hey Herb- I respect your decision to go
the route you are, but I have some concerns...Its important to me you know
this isn't personal, but rather voiced only with the hope for the best disk
possible.

There was an impression amongst some in the apba
community last time where "Herb won all ties." That is, in the vast majority
of cases where a ratings team or representative disagreed with you on a
rating, your ratings usually prevailed. I understand the impossible nature of
trying to please everyone and still generate an accurate disk. Its also a
given the respect you have across the apba community for the work you put in
on this.

The first thought I had when I read your mail was
this. Why doesn't Herb just do the disk? Period...

Leagues have
and are always free to make any edits they chose to, so you work up the
ratings the way you believe are best, see how they survive the test sims, make
your adjustments then send it out. If a league doesn't like it, there's a
zillion different systems they can implement to allow tweaking within in their
league.

I wonder if this approach wouldn't take us to the same
destination they we're bound for using the approach your detailed in your
email. It might be significantly quicker and easier for all parties
involved...

We've decided to go with a slightly different
approach this season but the ratings groups will still be
involved.

Here's what we're doing. I am going to rate the
entire league and then send those ratings to the groups for discussion. I know
this probably worries some of you, but to (hopefully) ease your concerns, here
is why we believe this process will work better.

1/
Consistency: Last year we had 30 different groups each using their own
criteria. Some groups mandated that there had to be a certain number of
1s, 2s, 3s etc., some gave out very few 2s and 1s, some gave fringe players 2s
across the board, other gave them 1s. That is not to say anyone was
wrong. They were right using their criteria. But with so many different
criteria, there wasn't much consistency from team to team. By having one
rater, throughout, we can eliminate these issues and then use the groups to
correct/tweak ratings.

2/
Timing. The committees were actually in place by about February last year.
Some started right away, some waited until the end of the season, some didn't
start rating until July, some just never happened. I was also encouraging
groups to start ratings as far back as February so they could evaluate while
games were still being played. Unfortunately, by the end of the playoffs
probably 2/3 of the groups had not started ratings.

We're hoping that worst case, if groups are slow to
respond, at least we'll have my baseline to work from and Dave can start
testing much, much earlier. I'm hoping end of June, beginning of July, which
will lead to us getting the disk much earlier and Dave actually being able to
enjoy some of his summer.

In
doing my ratings, I am using last year's ratings as a baseline. If there is no
evidence to support a particular rating, or I don't have a strong
opinion, I'm using last year's rating. So no, I'm not simply throwing out all
of last year's hard work. It worked well in testing so it's a great place to
start from. In this regard, the groups will be especially important on
rookies.

I
have rated about 350 players so far. I hope to have the league finished by the
end of the second round of the playoffs...at which point, we'll send the
ratings out to the different groups for their opinions.

Currently 176 games into a '07-'08 Replay using the
actual dressed lineups foreach NHL team and I have to say last
year's disk was extremely well done. Scores, penalties called,
etc. are very realistic. Only noticed a few tweakshere and there
that could be made to the disk. Considering the number ofplayers
on the disk, it's a testament to the great work Dave, Herb & the
commit tees put into the disk.

With a week remaining in the
'08-'09 NHL schedule, I'm wondering if thecommittees for each team
are ready to begin the rating process startingnext week. That was
our one fault last year, in not having the committees in
place and active right after the NHL season ended. Hopefully, that
won't bethe case this year and we'll produce another fantastic
disk in a moretimely
manner.

Hey Herb- I respect your decision to go the route you are, but I have some concerns...Its important to me you know this isn't personal, but rather voiced only with the hope for the best disk possible.

There was an impression amongst some in the apba community last time where "Herb won all ties." That is, in the vast majority of cases where a ratings team or representative disagreed with you on a rating, your ratings usually prevailed. I understand the impossible nature of trying to please everyone and still generate an accurate disk. Its also a given the respect you have across the apba community for the work you put in on this.

The first thought I had when I read your mail was this. Why doesn't Herb just do the disk? Period...

Leagues have and are always free to make any edits they chose to, so you work up the ratings the way you believe are best, see how they survive the test sims, make your adjustments then send it out. If a league doesn't like it, there's a zillion different systems they can implement to allow tweaking within in their league.

I wonder if this approach wouldn't take us to the same destination they we're bound for using the approach your detailed in your email. It might be significantly quicker and easier for all parties involved...

We've decided to go with a slightly different approach this season but the ratings groups will still be involved.

Here's what we're doing. I am going to rate the entire league and then send those ratings to the groups for discussion. I know this probably worries some of you, but to (hopefully) ease your concerns, here is why we believe this process will work better.

1/ Consistency: Last year we had 30 different groups each using their own criteria. Some groups mandated that there had to be a certain number of 1s, 2s, 3s etc., some gave out very few 2s and 1s, some gave fringe players 2s across the board, other gave them 1s. That is not to say anyone was wrong. They were right using their criteria. But with so many different criteria, there wasn't much consistency from team to team. By having one rater, throughout, we can eliminate these issues and then use the groups to correct/tweak ratings.

2/ Timing. The committees were actually in place by about February last year. Some started right away, some waited until the end of the season, some didn't start rating until July, some just never happened. I was also encouraging groups to start ratings as far back as February so they could evaluate while games were still being played. Unfortunately, by the end of the playoffs probably 2/3 of the groups had not started ratings.

We're hoping that worst case, if groups are slow to respond, at least we'll have my baseline to work from and Dave can start testing much, much earlier. I'm hoping end of June, beginning of July, which will lead to us getting the disk much earlier and Dave actually being able to enjoy some of his summer.

In doing my ratings, I am using last year's ratings as a baseline. If there is no evidence to support a particular rating, or I don't have a strong opinion, I'm using last year's rating. So no, I'm not simply throwing out all of last year's hard work. It worked well in testing so it's a great place to start from. In this regard, the groups will be especially important on rookies.

I have rated about 350 players so far. I hope to have the league finished by the end of the second round of the playoffs...at which point, we'll send the ratings out to the different groups for their opinions.

Currently 176 games into a '07-'08 Replay using the actual dressed lineups foreach NHL team and I have to say last year's disk was extremely well done. Scores, penalties called, etc. are very realistic. Only noticed a few tweakshere and there that could be made to the disk. Considering the number ofplayers on the disk, it's a testament to the great work Dave, Herb & the commit tees put into the disk.

With a week remaining in the '08-'09 NHL schedule, I'm wondering if thecommittees for each team are ready to begin the rating process startingnext week. That was our one fault last year, in not having the committees in place and active right after the NHL season ended. Hopefully, that won't bethe case this year and we'll produce another fantastic disk in a moretimely manner.

Hey Bill, No worries, definitely not taking anything personally. I m not at all surprised by the Herb won all ties impression. In the majority of cases, yes,

Message 7 of 19
, Apr 7, 2009

Message

Hey
Bill,

No
worries, definitely not taking anything personally.

I'm
not at all surprised by the 'Herb won all ties' impression. In the majority of
cases, yes, I probably did. In some cases, the reason I did was to acheive the
goals we set out for at the beginning of the process (stop everything from
gravitating toward 3, lower the average rating to increase shots, more 1s).

In
some cases, I pushed really hard for the sake of consistency. Some groups
insisted that their goons were straight 2s, other groups had them as 1s. (this
is a big generalization, not all goons are created equal, but for the most part,
these guys don't have the skills to play, which says 1....again, it's a
generalization, but it's the easiest illustration of the problem I could think
of). Wait, thought of another one. There were some teams where say 26 or the 28
players on a team would be 3 SPD or better. If every other team had only 20 of
28, you can see why a changes would need to be made.

In a
lot of cases I did conceed ratings and then Dave tested, and testing dictated
that changes had to be made. At that point, we were running so far behind,
I didn't go back and consult with the groups, I simply changed the ones I had
conceeded (providing they helped fix whatever problem testing was showing). So
from the ratings teams' perspective, when they saw the final disk, it probably
appeared that I didn't take their suggestion.

The
opposite also happened. The one that really stands out is that I lobbied the SJ
group really hard to have Joe Thornton dropped from a 4D to a 3. After finally
convincing them, San Jose tested low and Thornton ended up as a 4 on the final
disk. I still don't agree with it, but just so you know, I don't always get my
way :)

Why
don't I just do it myself? The simple answer is I don't profess to know the
skating ability of every Phoenix fourth liner. I don't know everything. If I
don't know, I try really, really hard to find out. But there's 800+ players, 10
ratings per player. Even if I did a remarkable job and got 90% of them
right, that's still 800 wrong. And as I said, I do whatever research I can to
get them right. Asking people who watch the teams on a regular basis, is
part of that research.

So why
not the same process as last year? It was just way to slow and tedious. I
think this process will move much faster if the ratings teams are given a
finished set where they can point out ratings they don't agree with instead of
having them debating every single skill, every single player. And I think we'd
all like to have the disk done a lot earlier.

Yes,
leagues can make edits but I doubt very many do. They live with the ratings and
I'd rather give those who care enough to get involved some input into the
disk.

Hope this answers some questions, alleviates
some concerns. But feel free to ask any more questions or bring up any more
concerns.

Herb

Just to address the last two
e-mails (yes, I know you are kidding but there are probably others out
there wondering so....) there are always going to be 'problem' players on every disk. They're not
really problems. We play one season with the disk. The disk is going to based on
the average of 1000s of test seasons. In those 1000s of seasons every player is
going to have some spectacular years and some dreadful years. When you play just
one of those 1000s, some of the 800+ players on the disk are going to have
those remarkable seasons. In our league, it's Matt Cullen and Shane Doan.
And actually Iginla is lighting it up too.

In the strike year, we used the same disk
two years in a row. The first year with the disk, Chara had 27 goals, 76 points.
Second year, same disk, no real changes to the team, Chara had 10 goals and 34
points.

Hey Herb- I respect your decision to go
the route you are, but I have some concerns...Its important to me you know
this isn't personal, but rather voiced only with the hope for the best disk
possible.

There was an impression amongst some in the apba
community last time where "Herb won all ties." That is, in the vast majority
of cases where a ratings team or representative disagreed with you on a
rating, your ratings usually prevailed. I understand the impossible nature of
trying to please everyone and still generate an accurate disk. Its also a
given the respect you have across the apba community for the work you put in
on this.

The first thought I had when I read your mail was
this. Why doesn't Herb just do the disk? Period...

Leagues have
and are always free to make any edits they chose to, so you work up the
ratings the way you believe are best, see how they survive the test sims, make
your adjustments then send it out. If a league doesn't like it, there's a
zillion different systems they can implement to allow tweaking within in their
league.

I wonder if this approach wouldn't take us to the same
destination they we're bound for using the approach your detailed in your
email. It might be significantly quicker and easier for all parties
involved...

We've decided to go with a slightly different
approach this season but the ratings groups will still be
involved.

Here's what we're doing. I am going to rate the
entire league and then send those ratings to the groups for discussion. I know
this probably worries some of you, but to (hopefully) ease your concerns, here
is why we believe this process will work better.

1/
Consistency: Last year we had 30 different groups each using their own
criteria. Some groups mandated that there had to be a certain number of
1s, 2s, 3s etc., some gave out very few 2s and 1s, some gave fringe players 2s
across the board, other gave them 1s. That is not to say anyone was
wrong. They were right using their criteria. But with so many different
criteria, there wasn't much consistency from team to team. By having one
rater, throughout, we can eliminate these issues and then use the groups to
correct/tweak ratings.

2/
Timing. The committees were actually in place by about February last year.
Some started right away, some waited until the end of the season, some didn't
start rating until July, some just never happened. I was also encouraging
groups to start ratings as far back as February so they could evaluate while
games were still being played. Unfortunately, by the end of the playoffs
probably 2/3 of the groups had not started ratings.

We're hoping that worst case, if groups are slow to
respond, at least we'll have my baseline to work from and Dave can start
testing much, much earlier. I'm hoping end of June, beginning of July, which
will lead to us getting the disk much earlier and Dave actually being able to
enjoy some of his summer.

In
doing my ratings, I am using last year's ratings as a baseline. If there is no
evidence to support a particular rating, or I don't have a strong
opinion, I'm using last year's rating. So no, I'm not simply throwing out all
of last year's hard work. It worked well in testing so it's a great place to
start from. In this regard, the groups will be especially important on
rookies.

I
have rated about 350 players so far. I hope to have the league finished by the
end of the second round of the playoffs...at which point, we'll send the
ratings out to the different groups for their opinions.

Currently 176 games into a '07-'08 Replay using the
actual dressed lineups foreach NHL team and I have to say last
year's disk was extremely well done. Scores, penalties called,
etc. are very realistic. Only noticed a few tweakshere and there
that could be made to the disk. Considering the number ofplayers
on the disk, it's a testament to the great work Dave, Herb & the
commit tees put into the disk.

With a week remaining in the
'08-'09 NHL schedule, I'm wondering if thecommittees for each team
are ready to begin the rating process startingnext week. That was
our one fault last year, in not having the committees in
place and active right after the NHL season ended. Hopefully, that
won't bethe case this year and we'll produce another fantastic
disk in a moretimely
manner.

Bill and anyone else that may have this concern. ... where Herb won all ties. That is, in the vast majority of cases where a ratings team or representative

Message 8 of 19
, Apr 7, 2009

Bill and anyone else that may have this concern.

> ? There was an impression amongst some in the apba community last time

where "Herb won all ties." That is, in the vast majority of cases
where a ratings team or representative disagreed with you on a
rating, your ratings usually prevailed. I understand the impossible
nature of trying to please everyone and still generate an accurate
disk.

I worked with Herb on 7 teams last season and I did not see anything like
*Herb won all ties*. We had an honest discourse on each and every one came
to decisions based on a multitude of things, but it was definitely NOT
*Herb won all ties*, I actually think that it was about 60/40 in my favor.

Just saying.

Dave Atkinson

I like this open discussion. I like getting the issues out on the table. I want to weigh in on this, but I m traveling and typing this on a blackberry. I ll

Message 9 of 19
, Apr 7, 2009

I like this open discussion. I like getting the issues out on the
table. I want to weigh in on this, but I'm traveling and typing this
on a blackberry. I'll wait til I get home to a real keyboard to jump
in. I was glad to see the email from Bill with concerns...... I lwant
to keep this process transparent, so I'll address that. The rating
teams are valuable and I still like that approach, it's just last year
had out of control issues that Herb mentioned. Look at this as a
tweak, not a total change. I expect lots of changes to the base rating
set.

D

On 4/7/09, Andy Bartalone <buffalo@...> wrote:
>
> Bill and anyone else that may have this concern.
>
>> ? There was an impression amongst some in the apba community last time
> where "Herb won all ties." That is, in the vast majority of cases
> where a ratings team or representative disagreed with you on a
> rating, your ratings usually prevailed. I understand the impossible
> nature of trying to please everyone and still generate an accurate
> disk.
>
> I worked with Herb on 7 teams last season and I did not see anything like
> *Herb won all ties*. We had an honest discourse on each and every one came
> to decisions based on a multitude of things, but it was definitely NOT
> *Herb won all ties*, I actually think that it was about 60/40 in my favor.
>
> Just saying.
>

dadadove@aol.com

I was involved in 2 groups with Herb and didn t feel there was any bias on his part.? He steered the groups through the process and listened to all discussion.

Message 10 of 19
, Apr 8, 2009

I was involved in 2 groups with Herb and didn't feel there was any bias on his part. He steered the groups through the process and listened to all discussion. Certainly if he felt strongly enough on something he stood his ground, but so did the rest of us.

I like this open discussion. I like getting the issues out on thetable. I want to weigh in on this, but I'm traveling and typing thison a blackberry. I'll wait til I get home to a real keyboard to jumpin. I was glad to see the email from Bill with concerns.... .. I lwantto keep this process transparent, so I'll address that. The ratingteams are valuable and I still like that approach, it's just last yearhad out of control issues that Herb mentioned. Look at this as atweak, not a total change. I expect lots of changes to the base ratingset.

D

On 4/7/09, Andy Bartalone <buffalo@guisarme. net> wrote:>> Bill and anyone else that may have this concern.>>> ? There was an impression amongst some in the apba community last time> where "Herb won all ties." That is, in the vast majority of cases> where a ratings team or representative disagreed with you on a> rating, your ratings usually prevailed. I understand the impossible> nature of trying to please everyone and still generate an accurate> disk.>> I worked with Herb on 7 teams last season and I did not see anything like> *Herb won all ties*. We had an honest discourse on each and every one came> to decisions based on a multitude of things, but it was definitely NOT> *Herb won all ties*, I actually think that it was about 60/40 in my favor.>> Just saying.>

I don t know if there is an absolutely ideal way to come up with the ratings each year. Let s give this new method a chance and see if it s better than how

Message 11 of 19
, Apr 8, 2009

I don't know if there is an absolutely ideal way to come up with the ratingseach year. Let's give this new method a chance and see if it's better thanhow things went last year. As Herb said, it will eliminate the problem of having to wait on getting all the ratings in for testing to begin.

One thing that could be a problem is that some of the committees couldsee that the ratings are "done" and may not want to be bothered withgoing through them to do any tweaking to them. Last year, some committeesjust went along with all of Herb's suggested changes. I'm hopeful that thiswon't happen but realistic enough to think some committees will take thatapproach.

As for the perception that Herb got his way on
all the ties, there's perceptionand then there's reality (often times the line gets blurred between these two).I believe the reality is that the extensive disk testing is what eventuallyends up determining if some ratings need to be adjusted before the ratingis made final.

I hope everyone remains open minded about this process and particates intweaking/adjusting, within the commitees, the ratings that Herb will supply tous. As Herb said he's no expert on every single NHL players skills. However, I found Herb to be very thoughful & knowledgeable in the ratings suggestionshe made to our commitee last season. He backed up his suggestions with stats like HITS per MIN and didn't force any changes on us. That along with all the tremendous work he's doing in putting together a set of ratings for each team gives us a great start on what should hopefully be another great disk.

D. Atkinson

As I promised am going to put in my 2 cents here. However, Herb has already covered the topic so well, I don t know how much more I can add. I do want to

Message 12 of 19
, Apr 11, 2009

As I promised am going to put in my 2 cents here. However, Herb has
already covered the topic so well, I don't know how much more I can add.

I do want to change from how we did the disk last year. It was an
experiment, and it failed. The disk ended up coming out the latest
ever, and that was with me needing to take vacation time from work to
get the final stuff done. Not that the review team didn't do good work.
There was some fabulous discussions and committees adding huge value to
our process. But, there were also some apathetic committees that didn't
even get moving until August, some unwanted bickering, some strong
homerism, and groups being very different on the definition of a 1 or
5. This is the part that made the disk more difficult.

Bill Corfield suggested that "Herb just do the disk ratings". I don't
see that as a good solution (if we want a good disk). Nothing against
Herb, he will be the first to tell he doesn't know the whole league in
detail. However, Herb is fair and balanced, and knows enough to give a
good first draft. The testing can only do so much.....yes, some ratings
like DEF have a strong tie to performance. However, many of the more
objective ratings give APBA it's "feel" and I can't really make a
direct tie (within statistical significance) to the performance of the
players. Getting it right when a player is a 4 SKT, 5 SPD versus 5 SKT,
4 SPD is the subtle differences that diehard APBA fans want, as it
shows up in the flow of the game. So, I think the concept of Herb
getting out a rough set to be tweaked by committees is a good fix.
First, it will get people past that "writer's block" of having to do
all the ratings from scratch, which will save time and give us a equal
metric basis. Second, it will stimulate discussions immediately, which
will also save time and have these discussions take place while the
season is still fresh on everyone's mind. Third, it will give a fair
and balanced set where the same metric is used as a starting point. I
don't want to inhibit the review teams from making changes...that's not
the goal. However, I will likely run some stats on the changes, and the
teams that get modified by a statisitically significant amount
(especially in one direction) will get a close look, with discussions
to follow. I want to avoid homerisms that unbalance the entire disk.
For example, if all DEF 1 and 2 ratings are bumped up on a team, this
will get a hard look. And, the changes will have to be borne out by the
testing, within the limits of the APBA game engine. Last year, we had
major issues with Montreal. Not that the review team did a bad job, but
the APBA engine has a difficult time handling a team that has a very
strong won/loss record, but gives up a ridiculous amount of shots. We
had to really suppress some ratings on Montreal to get them to perform
as they did in the NHL, if we didn't do this, the Habs would have been
an unbeatable juggernaut, an unrealisitic result. I know this didn't
sit well with the review committees (and rightly so to a point), but in
the end, the realistic performance of the disk is the final, critical
metric. So, only as many tweaks as needed to get it right were
done....in cases where a tweak didn't help the issue, we tweaked it
back. Nearly 1000000 games were run in testing to get things realistic.
But the quality of the review teams really makes the disk what it is in
the end.

d

Herb Garbutt wrote:

Hey guys,

We've decided to go with a slightly
different approach this season but the ratings groups will still be
involved.

Here's what we're doing. I am going to rate
the entire league and then send those ratings to the groups for
discussion. I know this probably worries some of you, but to
(hopefully) ease your concerns, here is why we believe this process
will work better.

1/ Consistency: Last year we had 30
different groups each using their own criteria. Some groups
mandated that there had to be a certain number of 1s, 2s, 3s etc., some
gave out very few 2s and 1s, some gave fringe players 2s across the
board, other gave them 1s. That is not to say anyone was wrong. They
were right using their criteria. But with so many different criteria,
there wasn't much consistency from team to team. By having one rater,
throughout, we can eliminate these issues and then use the groups to
correct/tweak ratings.

2/ Timing. The committees were actually in
place by about February last year. Some started right away, some waited
until the end of the season, some didn't start rating until July, some
just never happened. I was also encouraging groups to start ratings as
far back as February so they could evaluate while games were still
being played. Unfortunately, by the end of the playoffs probably 2/3 of
the groups had not started ratings.

We're hoping that worst case, if groups are
slow to respond, at least we'll have my baseline to work from and Dave
can start testing much, much earlier. I'm hoping end of June, beginning
of July, which will lead to us getting the disk much earlier and Dave
actually being able to enjoy some of his summer.

In doing my ratings, I am using last year's
ratings as a baseline. If there is no evidence to support a particular
rating, or I don't have a strong opinion, I'm using last year's rating.
So no, I'm not simply throwing out all of last year's hard work. It
worked well in testing so it's a great place to start from. In this
regard, the groups will be especially important on rookies.

I have rated about 350 players so far. I
hope to have the league finished by the end of the second round of the
playoffs...at which point, we'll send the ratings out to the different
groups for their opinions.

Currently 176 games into a '07-'08 Replay
using the actual dressed lineups for
each NHL team and I have to say last year's disk was extremely well
done.
Scores, penalties called, etc. are very realistic. Only noticed a few
tweaks
here and there that could be made to the disk. Considering the number of
players on the disk, it's a testament to the great work Dave, Herb
& the
commit tees put into the disk.

With a week remaining in the '08-'09 NHL schedule, I'm wondering if the
committees for each team are ready to begin the rating process starting
next week. That was our one fault last year, in not having the
committees
in place and active right after the NHL season ended. Hopefully, that
won't be
the case this year and we'll produce another fantastic disk in a more
timely manner.

Your message has been successfully submitted and would be delivered to recipients shortly.