WEBVTT
00:00:05.813 --> 00:00:08.597
Leslie: So, welcome to our webinar sponsored by
00:00:08.597 --> 00:00:11.055
the National Institute of Corrections. I
00:00:11.055 --> 00:00:13.494
am Leslie LeMaster, with NIC, your
00:00:13.494 --> 00:00:18.936
host and one of the facilitators for
today's webinar. You will notice
00:00:18.936 --> 00:00:21.502
as you join today, that you
are muted and the
00:00:21.502 --> 00:00:23.791
voices you're going
to be hearing are
00:00:23.791 --> 00:00:26.520
mine, as your host, those of your
00:00:26.520 --> 00:00:30.036
facilitators, and your
producer. Your way
00:00:30.036 --> 00:00:32.679
of communicating with
us is via the chat
00:00:32.679 --> 00:00:35.408
function, which is
located in the lower
00:00:35.408 --> 00:00:38.138
right hand corner of
the Webex webinar
00:00:38.138 --> 00:00:41.756
screen. To ensure that
your chat function
00:00:41.756 --> 00:00:44.103
is turned on because
we are going to be
00:00:44.103 --> 00:00:47.252
using it a lot during
the webinar to
00:00:47.252 --> 00:00:50.404
engage you in the
content and activities,
00:00:50.404 --> 00:00:53.253
I want you to make
sure it is turned on.
00:00:53.253 --> 00:00:58.100
So, hover your your
mouse over the bottom
00:00:58.100 --> 00:01:00.142
center of your screen
and you'll see
00:01:00.142 --> 00:01:03.174
some tools pop up. I
want you to click on
00:01:03.174 --> 00:01:05.222
a tool that looks like
a person talking.
00:01:05.222 --> 00:01:08.598
That will turn on the
participant panel
00:01:08.598 --> 00:01:10.746
for you in the upper
right hand corner
00:01:10.746 --> 00:01:14.186
of your screen. And
then turn on a tool
00:01:14.186 --> 00:01:16.177
by clicking it that
looks like a chat
00:01:16.177 --> 00:01:17.995
bubble and that will
turn on the chat
00:01:17.995 --> 00:01:20.713
function for you. And
then to make sure
00:01:20.713 --> 00:01:22.492
it's working, and a
whole bunch of people
00:01:22.492 --> 00:01:24.018
have done this, but
some probably have
00:01:24.018 --> 00:01:26.220
not yet who are just joining us and
00:01:26.220 --> 00:01:29.211
continue to join us,
if you would chat to
00:01:29.211 --> 00:01:31.712
us where you are
joining us from today
00:01:31.712 --> 00:01:34.869
or what the weather
is like and why you
00:01:34.869 --> 00:01:37.146
are interested in
the topics of this
00:01:37.146 --> 00:01:40.115
webinar, which is
"Training from Analysis
00:01:40.115 --> 00:01:42.826
(A) To evaluation (E):
Are you applying the
00:01:42.826 --> 00:01:45.342
Science of Learning
and Performance?" And
00:01:45.342 --> 00:01:47.720
if you would send that to all
00:01:47.720 --> 00:01:53.177
of participants in the
program, and you
00:01:53.177 --> 00:01:55.044
do that by choosing that on the
00:01:55.044 --> 00:01:57.787
drop-down menu. Oh
wow! It's snowing in
00:01:57.787 --> 00:02:00.372
South Dakota, and
then just click enter
00:02:00.372 --> 00:02:06.174
and your chat will go
out to all. So yes,
00:02:06.174 --> 00:02:10.103
oh boy. We've got all
kinds of things
00:02:10.103 --> 00:02:12.815
happening here and,
well, definitely it's
00:02:12.815 --> 00:02:14.119
winter in the country.
00:02:14.119 --> 00:02:18.036
It was snowing here,
also, too. So, training
00:02:18.036 --> 00:02:20.097
commanders we've got
lots of people from
00:02:20.097 --> 00:02:25.443
training. Yes, oh boy!
00:02:25.443 --> 00:02:30.783
Lots of things going
on here. So, it looks
00:02:30.783 --> 00:02:32.830
like the chat function
is working. In the
00:02:32.830 --> 00:02:34.447
first couple of times
we use it, we're
00:02:34.447 --> 00:02:38.072
gonna ensure that
everybody has got
00:02:38.072 --> 00:02:40.341
that working for us
because we are going
00:02:40.341 --> 00:02:42.803
to be using it a lot.
So, we're just all
00:02:42.803 --> 00:02:47.578
happy that weather
is rain--raining or
00:02:47.578 --> 00:02:51.291
snowing or sun shining
or sleeting or
00:02:51.291 --> 00:02:53.546
whatever it could be
doing that you are all
00:02:53.546 --> 00:02:55.317
joining us here today.
That's the power
00:02:55.317 --> 00:02:57.819
of virtual training,
right? We can do it
00:02:57.819 --> 00:02:59.915
no matter what the weather
conditions might be.
00:03:00.554 --> 00:03:02.521
So, we wanted to let you know that
00:03:02.521 --> 00:03:04.785
the webinar is being
recorded and you
00:03:04.785 --> 00:03:06.894
will have access to
that recording at a
00:03:06.894 --> 00:03:10.278
later date. We will
be posting it on the
00:03:10.278 --> 00:03:13.319
NIC website when it
is available. We'll
00:03:13.319 --> 00:03:15.367
also have the opportunity
for questions
00:03:15.367 --> 00:03:17.248
and answers so please do chat your
00:03:17.248 --> 00:03:19.359
questions and comments
all along the way
00:03:19.359 --> 00:03:23.197
using the chat function
as the webinar
00:03:23.197 --> 00:03:26.964
progresses. So, without
further ado, we do
00:03:26.964 --> 00:03:31.360
want to get started,
and I am going to
00:03:31.360 --> 00:03:34.876
start that out by,
actually, having my
00:03:34.876 --> 00:03:37.147
colleagues introduce themselves.
00:03:37.383 --> 00:03:40.605
I'll turn it over to
my chief, first, for
00:03:40.605 --> 00:03:43.114
him to introduce
himself to all of you.
00:03:43.114 --> 00:03:45.615
Jeff: Well, good morning
everyone my name is
00:03:45.615 --> 00:03:47.219
Jeff Hadnot on the chief of NIC's
00:03:47.219 --> 00:03:49.005
academy division. I'm
happy you guys were
00:03:49.005 --> 00:03:51.431
able to join us this
morning, especially
00:03:51.431 --> 00:03:54.092
on the short notice,
for the rescheduled
00:03:54.092 --> 00:03:57.494
event. We're excited to bring this
00:03:57.494 --> 00:03:59.672
webinar to you today on training. A
00:03:59.672 --> 00:04:01.905
personal connection
I kind of have with
00:04:01.905 --> 00:04:04.030
this is being a former
training director,
00:04:04.030 --> 00:04:06.607
I can share some of
the frustrations
00:04:06.607 --> 00:04:08.201
that that we all
have in the training
00:04:08.201 --> 00:04:10.890
field when it comes
to training and the
00:04:10.890 --> 00:04:12.606
role of training and
organization. So,
00:04:12.606 --> 00:04:15.650
we're excited to bring
some ideas to you
00:04:15.650 --> 00:04:16.878
today that you might be able to
00:04:16.878 --> 00:04:18.698
implement into your
bag of tricks and
00:04:18.698 --> 00:04:21.037
we're going to get started. So,
00:04:21.037 --> 00:04:22.655
Bernie, I'll turn it over to you.
00:04:23.979 --> 00:04:25.507
Bernie: Good morning, everyone.
00:04:25.990 --> 00:04:28.365
Am I coming through, Leslie?
00:04:28.365 --> 00:04:30.456
Leslie: Absolutely loud and
clear, Bernie.
00:04:30.456 --> 00:04:37.186
Bernie: Okay, from cold, but really
beautiful Colorado,
00:04:37.186 --> 00:04:39.395
I've been with the
academy division now
00:04:39.395 --> 00:04:42.185
for ten years. I can't
believe it's been
00:04:42.185 --> 00:04:45.045
ten years, it's gone
by so quickly. My
00:04:45.045 --> 00:04:47.326
particular interest
is in learning and
00:04:47.326 --> 00:04:50.514
performance. And, the
ADDIE model has been
00:04:50.514 --> 00:04:53.478
around for a while
and, of course, in the
00:04:53.478 --> 00:04:55.178
learning performance
field, things change,
00:04:55.178 --> 00:04:57.599
there's new technologies,
there's all
00:04:57.599 --> 00:04:59.940
kinds of new things
going on, but the
00:04:59.940 --> 00:05:02.344
ADDIE model has still stood up as a
00:05:02.344 --> 00:05:05.260
foundational piece in designing and
00:05:05.260 --> 00:05:07.402
delivering curriculum and can be
00:05:07.402 --> 00:05:10.132
effective in the
pieces we're talking
00:05:10.132 --> 00:05:13.289
about today and
connecting what we do in
00:05:13.289 --> 00:05:16.456
the training to what
the agency needs. So,
00:05:16.456 --> 00:05:18.883
good morning, everyone,
especially good
00:05:18.883 --> 00:05:20.924
morning to North
Dakota because that's
00:05:20.924 --> 00:05:22.610
where I'm from. And,
yeah, I can--I don't
00:05:22.610 --> 00:05:24.658
even have to guess
that it's cold there,
00:05:24.658 --> 00:05:28.016
but glad you're all
here from all parts
00:05:28.016 --> 00:05:29.768
of the country. Leslie.
00:05:29.978 --> 00:05:33.379
Leslie: All right. Thanks, Bern.
Thanks, Jeff. Again,
00:05:33.379 --> 00:05:36.994
I'm Lesley Lemaster
and that's me. I have
00:05:36.994 --> 00:05:39.603
been with the academy
division of NIC
00:05:39.603 --> 00:05:42.470
for 20 years and I
feel like I just got
00:05:42.470 --> 00:05:44.444
off the plane yesterday.
I can't believe
00:05:44.444 --> 00:05:47.510
it's been 20 years.
And this is one of my
00:05:47.510 --> 00:05:50.439
passion areas because
I believe that
00:05:50.439 --> 00:05:55.862
excellent analysis,
development, design,
00:05:55.862 --> 00:05:58.026
delivery, and evaluation
is really the
00:05:58.026 --> 00:06:01.137
basis and foundation
for everything we
00:06:01.137 --> 00:06:04.053
do in corrections.
That leads to the
00:06:04.053 --> 00:06:06.146
great delivery of
services. Because if we
00:06:06.146 --> 00:06:08.950
don't train people well and then
00:06:08.950 --> 00:06:10.956
evaluate the training
that we do to see
00:06:10.956 --> 00:06:12.472
if it really does
what we needed to do
00:06:12.472 --> 00:06:17.325
then kind of that
loss. So, we're really
00:06:17.325 --> 00:06:20.109
happy that we're able
to bring you this
00:06:20.109 --> 00:06:23.476
webinar, and that all
of our technology
00:06:23.476 --> 00:06:25.648
is working fabulously today. We're
00:06:25.648 --> 00:06:27.909
hoping it's working
fabulously for you on
00:06:27.909 --> 00:06:31.999
your end, as well. So,
there's someone else
00:06:31.999 --> 00:06:34.465
who is joining us, also
today, who's gonna
00:06:34.465 --> 00:06:36.370
bet going to be
behind the scenes, but
00:06:36.370 --> 00:06:39.310
we would love for her to introduce
00:06:39.310 --> 00:06:41.510
herself to you, as well.
Linda.
00:06:42.600 --> 00:06:44.853
Linda: Thank you,
Leslie. Hello, everyone
00:06:44.853 --> 00:06:47.013
and good morning. Happy
to have you here.
00:06:47.013 --> 00:06:50.897
I'm Linda, as Leslie
said. I've been a
00:06:50.897 --> 00:06:53.040
management assistant
for 10 years with
00:06:53.040 --> 00:06:56.139
NIC. And I work with
Jeff, Leslie, and
00:06:56.139 --> 00:06:57.793
Bernie in various learning and
00:06:57.793 --> 00:07:01.085
performance projects,
but today I will
00:07:01.085 --> 00:07:03.600
be behind the scenes
as your webinar
00:07:03.600 --> 00:07:06.735
producer. And now I
will hand it back to
00:07:06.735 --> 00:07:07.436
Leslie.
00:07:07.593 --> 00:07:09.660
Leslie: All right. Thanks, Linda. So, Linda's
00:07:09.660 --> 00:07:11.245
going to be responding
to you. If you
00:07:11.245 --> 00:07:12.621
happen to have any technical
00:07:12.621 --> 00:07:14.829
difficulties, but she, also, will be
00:07:14.829 --> 00:07:17.677
keeping track of any
questions that you
00:07:17.677 --> 00:07:19.490
happen to have along the way or any
00:07:19.490 --> 00:07:21.869
intriguing and
interesting comments. So,
00:07:21.869 --> 00:07:23.218
that when we get to
the question and
00:07:23.218 --> 00:07:25.796
answer period, we're
all ready to go. So,
00:07:25.796 --> 00:07:27.585
very important role
here in the webinar.
00:07:27.585 --> 00:07:30.771
So, to get us started,
Bernie, I think
00:07:30.771 --> 00:07:33.020
you've got an engagement
activity to get
00:07:33.020 --> 00:07:35.459
us going and up and running today.
00:07:35.459 --> 00:07:41.040
Bernie: Okay, so technology-wise
we've got 80 people
00:07:41.040 --> 00:07:44.649
online. Hopefully, you
all have audio and
00:07:44.649 --> 00:07:47.782
you got your computer
working and we can
00:07:47.782 --> 00:07:49.547
all hear each other or you can
00:07:49.547 --> 00:07:52.361
hear us and see what's
going on. That's
00:07:52.361 --> 00:07:54.652
part of the engagement for distance
00:07:54.652 --> 00:07:56.225
learning. The other part of the
00:07:56.225 --> 00:07:59.933
engagement is how
engaged will you be in
00:07:59.933 --> 00:08:02.117
this webinar. So, if
you can rate yourself
00:08:02.117 --> 00:08:04.272
on a scale of one to
seven, seven being
00:08:04.272 --> 00:08:08.090
incredibly engaged
00:08:08.090 --> 00:08:10.925
and one being not engaged at all,
00:08:10.925 --> 00:08:13.325
and kind of rate where
you are in that.
00:08:13.325 --> 00:08:16.621
And and just put that
into the chat.
00:08:19.730 --> 00:08:21.313
Jeff: All right, looks like we've got a lot of
00:08:21.313 --> 00:08:23.470
fives all the way up
to seven. I saw some
00:08:23.470 --> 00:08:24.904
seven pluses in there. We got some
00:08:24.904 --> 00:08:28.003
motivated individuals
today for you.
00:08:28.410 --> 00:08:32.135
Bernie: It looks like that.
There's all kinds of
00:08:32.135 --> 00:08:35.117
different ways that
we get disengaged
00:08:35.117 --> 00:08:37.701
with technology and we're in the
00:08:37.701 --> 00:08:41.100
classroom. So, just
getting ourselves
00:08:41.100 --> 00:08:43.319
oriented. So, thank you
for putting it in
00:08:43.319 --> 00:08:45.759
there. We're got five,
sixes, lots of five
00:08:45.759 --> 00:08:49.858
and six, and fours.
So, considering where
00:08:49.858 --> 00:08:53.086
you are, if you wanted to raise your
00:08:53.086 --> 00:08:55.142
number, I want you to put into chat
00:08:55.142 --> 00:08:57.537
what would you do to increase your
00:08:57.537 --> 00:09:01.248
number? What would
you do to increase
00:09:01.248 --> 00:09:04.571
your number to be more
engaged?
00:09:07.080 --> 00:09:09.447
And just put that into the
chat. Just put your
00:09:09.447 --> 00:09:12.183
ideas into chat. In-
person training.
00:09:12.183 --> 00:09:14.869
Sometimes, in-person
can be more engaging,
00:09:14.869 --> 00:09:16.999
but we know even with
classroom training
00:09:16.999 --> 00:09:21.462
people check out on
us. "Lunch and a nap"
00:09:21.462 --> 00:09:26.242
would help. "Starbucks"
Okay, "Stop eating
00:09:26.242 --> 00:09:29.032
lunch." "More coffee."
00:09:29.032 --> 00:09:32.309
"Stay focused." "Clear
my desk." "Away from the
00:09:32.309 --> 00:09:37.596
work desk". Cool. Jeff...
what else have we
00:09:37.596 --> 00:09:38.187
got?
00:09:40.024 --> 00:09:42.149
Jeff: Looks like I saw
"Shut down the email."
00:09:42.149 --> 00:09:44.587
I think that there's
some value to be in
00:09:44.587 --> 00:09:46.643
on that. Maybe an
out-of-office on the
00:09:46.643 --> 00:09:48.093
phone. It looks like we've got some
00:09:48.093 --> 00:09:51.001
information there...
"Hanging a sign on your
00:09:51.001 --> 00:09:53.603
door" is also a good
way to raise that
00:09:53.603 --> 00:09:55.812
number up. Looks like
we got some pretty
00:09:55.812 --> 00:09:58.022
good answers here, Bernie.
00:09:59.484 --> 00:10:05.587
Bernie: Okay. Thanks, Jeff. So, we
know whether it's a webinar,
00:10:05.587 --> 00:10:08.622
whether it's classroom,
we struggle with
00:10:08.622 --> 00:10:11.537
the concentration. We
struggle with being
00:10:11.537 --> 00:10:13.681
engaged because we
have so many more
00:10:13.681 --> 00:10:16.922
things pulling us away from
00:10:16.922 --> 00:10:19.352
concentration. Why
should we care about
00:10:19.352 --> 00:10:21.758
that? Well, we want
you to get the most
00:10:21.758 --> 00:10:25.737
that you possibly can
from this webinar,
00:10:25.737 --> 00:10:29.782
and so, being engaged and distancing
00:10:29.782 --> 00:10:32.274
ourselves from
distractions can help. The
00:10:32.274 --> 00:10:35.201
whole idea that people
can multitask is
00:10:35.201 --> 00:10:37.922
really a lie. The
research tells us there
00:10:37.922 --> 00:10:40.938
is no such thing as
multitasking, which
00:10:40.938 --> 00:10:44.772
you cannot do three
things at once and
00:10:44.772 --> 00:10:48.444
give them all attention
at the same time.
00:10:48.444 --> 00:10:51.069
So, we encourage you
to be as engaged as
00:10:51.069 --> 00:10:52.493
you can, just because
we want you to get
00:10:52.493 --> 00:10:55.972
the most out of this,
but certainly when
00:10:55.972 --> 00:11:00.052
it comes to webinars,
there's things you
00:11:00.052 --> 00:11:03.438
can do. I make sure
one thing is, I do
00:11:03.438 --> 00:11:05.384
is make sure my cell
phone is shut off
00:11:05.384 --> 00:11:08.585
and I can't see it,
so, I'm not tempted
00:11:08.585 --> 00:11:10.853
to grab it when I'm in a webinar.
00:11:12.218 --> 00:11:15.881
And so, these things whether
in the classroom or webinar,
00:11:15.881 --> 00:11:20.624
and in the classroom, what I see is
people check out and you'll see them
00:11:20.624 --> 00:11:22.367
move, you'll see them
look away. You just
00:11:22.367 --> 00:11:24.639
know, you have this
sense when you work
00:11:24.639 --> 00:11:26.730
with classrooms after
a while is "Okay...
00:11:26.730 --> 00:11:27.946
they've checked out right now."
00:11:28.464 --> 00:11:30.333
So, and we know--we know
it's just a
00:11:30.333 --> 00:11:33.267
human thing to do and,
so we encourage
00:11:33.267 --> 00:11:35.371
you to do everything you can to be
00:11:35.371 --> 00:11:38.307
engaged in this. So,
we encourage you to
00:11:38.307 --> 00:11:42.707
be here now with us
in this webinar in
00:11:42.707 --> 00:11:48.913
the present, fully
engaged. And, so now,
00:11:48.913 --> 00:11:51.556
that you are here, now, with us fully
00:11:51.556 --> 00:11:55.014
engaged, what Leslie
will be focusing on
00:11:55.014 --> 00:11:56.700
during today's session?
00:11:57.642 --> 00:12:02.353
Leslie: Well, Bernie. I'm so glad you
asked that.Before I actually
00:12:02.353 --> 00:12:08.552
focus on that, I just
wanted to point out
00:12:08.552 --> 00:12:12.749
one particular thing about this
00:12:12.749 --> 00:12:14.649
objective. You'll
notice there's two
00:12:14.649 --> 00:12:16.414
words that are highlighted here in
00:12:16.414 --> 00:12:18.823
different colors, which
is analysis in and
00:12:18.823 --> 00:12:22.207
evaluation. We just have one,
you know, two
00:12:22.207 --> 00:12:24.675
focus areas, instead of the entire
00:12:24.675 --> 00:12:26.780
ADDIE framework, we're
going to be working
00:12:26.780 --> 00:12:28.951
on applying the ADDIE principles of
00:12:28.951 --> 00:12:31.281
analysis and evaluation
to the training
00:12:31.281 --> 00:12:34.352
in your organization.
So, that's our real
00:12:34.352 --> 00:12:37.862
focus area. So, the
first part the webinar
00:12:37.862 --> 00:12:40.337
is about an analysis
and some activities
00:12:40.337 --> 00:12:42.480
around that, and then,
the second part is
00:12:42.480 --> 00:12:45.019
evaluation. So, we hope will also be
00:12:45.019 --> 00:12:47.432
modeling for you how
you can do very
00:12:47.432 --> 00:12:50.573
interactive training
in a webinar format
00:12:50.573 --> 00:12:55.242
and then you can use these ideas in
00:12:55.242 --> 00:12:57.468
doing virtual training in your own
00:12:57.468 --> 00:12:59.891
agency. So, we're hope
where we're gonna
00:12:59.891 --> 00:13:01.998
kind of an unwritten
objective here
00:13:01.998 --> 00:13:08.388
for you, as well. So,
if you're like me,
00:13:08.515 --> 00:13:11.593
sometimes you just
have the best idea
00:13:11.593 --> 00:13:14.626
ever and "I just have
to do something
00:13:14.626 --> 00:13:18.079
with it and I know
just what I'll do. I'm
00:13:18.079 --> 00:13:21.867
going to design training." But, then
00:13:21.867 --> 00:13:24.572
I think to myself, "Is
that actually going
00:13:24.572 --> 00:13:28.688
to meet a need?" So, what
I'd like for you
00:13:28.688 --> 00:13:33.096
to do, is chat to me
what do you do when
00:13:33.096 --> 00:13:35.993
you have the best idea ever.
00:13:41.354 --> 00:13:44.065
What do you do when
you have the best
00:13:44.065 --> 00:13:45.171
idea ever?
00:13:48.388 --> 00:13:50.623
Ah! "Share it with peers
to get feedback,"
00:13:50.623 --> 00:13:53.760
says James. "Write it
down," says Fanmaron
00:13:53.760 --> 00:13:55.506
says, "share it with
somebody." "Tell
00:13:55.506 --> 00:13:57.766
somebody else." "Share
the well." Ooh "Sticky
00:13:57.766 --> 00:14:01.356
note," "Write an outline." Haha.
"Forget it," "Research
00:14:01.356 --> 00:14:07.117
it." "Write an outline."
Ooh! All kinds of
00:14:07.117 --> 00:14:09.287
great ideas coming
down here! Be sure and
00:14:09.287 --> 00:14:10.866
look at the chat. Oh I love it!
00:14:10.866 --> 00:14:12.951
"The iPhone memo pad,"
I love it! Don't you
00:14:12.951 --> 00:14:17.663
love that? "Journal it."
"I brain storm it."
00:14:17.663 --> 00:14:23.798
I use a little... I like
to mind map things.
00:14:25.005 --> 00:14:28.414
But, almost inevitably,
sometimes I always
00:14:28.414 --> 00:14:33.077
seem to want to design
training for some
00:14:33.077 --> 00:14:37.341
reason. That always
be my first place I'd
00:14:37.341 --> 00:14:40.792
go, but I have to
stop myself. So, what
00:14:40.792 --> 00:14:42.529
we're going to do now,
is we're going to
00:14:42.529 --> 00:14:45.156
visit... we're going
to kind of peek in on
00:14:45.156 --> 00:14:47.329
a conversation that's
happening right
00:14:47.329 --> 00:14:50.021
now. Somewhere here
in the United States
00:14:50.021 --> 00:14:53.383
between a CEO and a
training director,
00:14:53.383 --> 00:14:59.565
and find out what's
going on with them.
00:14:59.565 --> 00:15:01.450
Let's begin.
00:15:01.839 --> 00:15:03.730
[knock, knock, knock, knock, knock, knock]
00:15:05.730 --> 00:15:09.355
CEO: Bernie, if I see one more CO mess of a
pat-down search, I'm going to
00:15:09.355 --> 00:15:10.003
blow a gasket.
00:15:10.816 --> 00:15:12.578
Training Director: What's the problem?
00:15:13.187 --> 00:15:16.503
CEO: We spend thousands of dollars on eight
weeks training these new correctional
00:15:16.503 --> 00:15:17.824
officers and they can't
even do a simple
00:15:17.824 --> 00:15:19.937
pass search correctly.
I want you to
00:15:19.937 --> 00:15:21.746
double the hours in basic that we
00:15:21.746 --> 00:15:23.665
trained on pat searches
and I want you
00:15:23.665 --> 00:15:25.261
to add four hours of it to annual
00:15:25.261 --> 00:15:26.370
refresher training.
00:15:26.923 --> 00:15:31.349
Training Director: That could take some
analysis and design time.
00:15:31.532 --> 00:15:33.181
CEO: No, no, Bernie. We don't
have any time for
00:15:33.181 --> 00:15:35.062
that. We need this done ASAP.
00:15:35.740 --> 00:15:38.437
Training Director: So, what is it that you
want to accomplish
00:15:38.437 --> 00:15:39.460
with this training?
00:15:40.033 --> 00:15:41.853
CEO: I want the pat search done
correctly.
00:15:45.189 --> 00:15:48.785
Leselie: Oh, boy. That's a really
interesting kind of a thing
00:15:48.785 --> 00:15:50.740
going on. So what's
going on in that
00:15:50.740 --> 00:15:52.814
conversation, you guys
that we were just
00:15:52.814 --> 00:15:55.023
eavesdropping on? Chat,
what is happening
00:15:55.023 --> 00:15:58.801
in that situation?
And, have you ever
00:15:58.801 --> 00:16:02.392
experienced that?
Haha, Yeah right, Spencer,
00:16:02.392 --> 00:16:04.784
"Blaming training."
Leslie says, "Pretty
00:16:04.784 --> 00:16:08.107
typical." "Reactive
approach." Yeah, Courtney,
00:16:08.107 --> 00:16:10.403
you got it. "Blaming
training." "Policies are
00:16:10.403 --> 00:16:13.693
not clear." Oh. "Trying
to solve a bigger
00:16:13.693 --> 00:16:16.050
problem and assuming
it's training."
00:16:16.060 --> 00:16:18.723
Oh, yeah. Maria says, "All the time."
00:16:19.261 --> 00:16:21.714
"Something bad's
happened. Fix it now."
00:16:21.714 --> 00:16:23.691
Ah! "They're not really
getting to the bottom
00:16:23.691 --> 00:16:27.045
of what's happening."
"Staff not being held
00:16:27.045 --> 00:16:28.023
accountable."
00:16:28.023 --> 00:16:31.146
Oh, "Not looking for
a root cause," says
00:16:31.146 --> 00:16:34.298
Courtney. "A communication
shortfall," says
00:16:34.298 --> 00:16:36.813
Linda. "Expectations
are not clear" from
00:16:36.813 --> 00:16:40.606
Jeff. "A knee-jerk
reaction" said Kathleen.
00:16:40.606 --> 00:16:43.725
"Without looking into
it to see the real
00:16:43.725 --> 00:16:46.842
problem." I think
everybody is spot on
00:16:46.842 --> 00:16:50.425
with that. So, Bernie, um
how does this begin to
00:16:50.425 --> 00:16:52.829
play out in the
workplace and and then,
00:16:52.829 --> 00:16:55.560
what do we need to do
to find out what's
00:16:55.560 --> 00:16:57.849
really going on in this situation?
00:16:59.423 --> 00:17:03.370
Bernie: Well, we see that a lot
of times what happened
00:17:03.370 --> 00:17:05.481
is this what actually,
what everybody
00:17:05.481 --> 00:17:07.245
said in chat is that
it's a knee-jerk
00:17:07.245 --> 00:17:09.631
reaction. We experience
somewhere there's
00:17:09.631 --> 00:17:12.041
an issue and they
were asked to design
00:17:12.041 --> 00:17:15.802
something. Usually,
really quickly, to fix
00:17:15.802 --> 00:17:18.523
it in response to this on-the-job
00:17:18.523 --> 00:17:21.627
problem. And so, that
happens often we
00:17:21.627 --> 00:17:23.711
hear that often. I
remember it happening
00:17:23.711 --> 00:17:27.174
when I was in the field.
And so,... so, let's
00:17:27.174 --> 00:17:29.395
begin to look and see
how this plays out
00:17:29.395 --> 00:17:31.545
in the workplace. And
we already saw how
00:17:31.545 --> 00:17:35.632
this plays out with
CEOs. So, what can we
00:17:35.632 --> 00:17:39.380
do to...and yes, some of
you already address
00:17:39.380 --> 00:17:42.601
this. What can we
do to get to what's
00:17:42.601 --> 00:17:46.346
really going on here?
Why? And why conduct
00:17:46.346 --> 00:17:49.668
a needs analysis before
designing training?
00:17:52.284 --> 00:17:54.764
So, why conduct a needs
analysis? In the
00:17:54.764 --> 00:17:57.716
chat, we already had
some people tell us.
00:17:57.716 --> 00:17:59.941
already. You already picked up on
00:17:59.941 --> 00:18:01.943
that, but if you have a knee-jerk
00:18:01.943 --> 00:18:05.666
reaction, and it may
not be... that may not
00:18:05.666 --> 00:18:08.747
be the issue. To see
if it's actually a
00:18:08.747 --> 00:18:14.760
training issue...so,
what other ideas... we
00:18:14.760 --> 00:18:16.934
used... "We trained
using training as
00:18:16.934 --> 00:18:19.391
punishment." Yes, that
happens, doesn't it?
00:18:19.391 --> 00:18:22.463
"To look at the full
picture." Okay. "Why do
00:18:22.463 --> 00:18:26.919
it needs analysis? See...
if it's a training issue or if the
00:18:27.042 --> 00:18:28.355
supervisor/leadership."
00:18:28.972 --> 00:18:30.956
What if it's... if
it's a widespread
00:18:30.956 --> 00:18:35.544
issue or just just local?"
Okay, so,
00:18:35.544 --> 00:18:37.659
anything else, Leslie?
00:18:39.280 --> 00:18:42.419
Leslie: Oh, yeah."Identifying the problem. If it's
time, money and
00:18:42.419 --> 00:18:44.325
resources that are
being spent in the
00:18:44.325 --> 00:18:47.184
wrong area." It's all
kinds of clarifying.
00:18:47.184 --> 00:18:50.063
It seems to be a
theme here of what's
00:18:50.063 --> 00:18:53.503
really going on versus
the perspective
00:18:53.503 --> 00:18:58.128
that the CEO is bringing in
that conversation.
00:18:59.877 --> 00:19:05.986
Bernie: Thanks, Leslie. So, we see...
you understand that if we don't diagnose the
00:19:05.986 --> 00:19:08.216
problem correctly, how
are we sure that
00:19:08.216 --> 00:19:11.044
our fix is going to
fix it? How are we
00:19:11.044 --> 00:19:13.183
sure that what we do
is going to fix it?
00:19:13.183 --> 00:19:15.213
I saw one of them said
too, "What if this
00:19:15.213 --> 00:19:18.267
is a small issue
in one place or is
00:19:18.267 --> 00:19:20.135
it widespread?" That could make a
00:19:20.135 --> 00:19:23.606
difference, as well.
And, "Finding out how
00:19:23.606 --> 00:19:26.154
is it going to be
addressed by training?"
00:19:26.154 --> 00:19:28.886
The other problem is
that sometimes when
00:19:28.886 --> 00:19:30.943
we are...just do the
knee jerk and click
00:19:30.943 --> 00:19:34.041
deliver something if we haven't
00:19:34.041 --> 00:19:38.297
addressed the real
issue, then what I see
00:19:38.297 --> 00:19:40.321
is training turns
around and gets blamed.
00:19:40.321 --> 00:19:42.966
And. you'll hear, "Well,
you know. See that
00:19:42.966 --> 00:19:44.543
didn't make any
difference anyway. So,
00:19:44.543 --> 00:19:46.785
training really doesn't
matter." Well, it
00:19:46.785 --> 00:19:48.079
really doesn't matter when it isn't
00:19:48.079 --> 00:19:52.284
fixing the right
problem. So a lot of
00:19:52.284 --> 00:19:54.143
great ideas here.
That it really is
00:19:54.143 --> 00:19:56.211
understanding what's
really going on is
00:19:56.211 --> 00:20:01.355
important. And so, doing
a needs analysis,
00:20:01.355 --> 00:20:03.354
and when you look at that, does...
00:20:03.354 --> 00:20:07.656
Can we again and chat
talk about this?
00:20:07.656 --> 00:20:11.235
Does your agency
conduct a needs or gap
00:20:11.235 --> 00:20:14.170
analysis prior to
designing a solution?
00:20:14.170 --> 00:20:17.258
In chat tell us, do
you do that with your
00:20:17.258 --> 00:20:19.792
training? Do you do a
needs analysis?
00:20:31.872 --> 00:20:34.934
"No." Got some yeses. "We've
started to in some
00:20:34.934 --> 00:20:38.790
cases." "Our agency is moving toward
00:20:38.790 --> 00:20:41.771
calibration techniques." Oh, that's
00:20:41.771 --> 00:20:43.532
interesting! I want
to know more about
00:20:43.532 --> 00:20:47.345
that. "Basically, train
according to what
00:20:47.345 --> 00:20:49.407
is sent down to us."
Yeah, from the top
00:20:49.407 --> 00:20:53.867
down. "Sometimes, yeah."
So it appears...
00:20:54.430 --> 00:20:59.838
Somewhat spotty... ties
back to that. So, some
00:20:59.838 --> 00:21:03.354
agencies do, some don't.
Some do certain
00:21:03.354 --> 00:21:07.418
kinds of needs assessment.
And others... and
00:21:07.418 --> 00:21:08.768
it says, "In some forms, but not
00:21:08.768 --> 00:21:09.029
necessarily, but, officialy your best
00:21:10.450 --> 00:21:14.270
practices." And "Some
some of us don't know
00:21:14.270 --> 00:21:17.329
how to do a proper
needs analysis." So,
00:21:17.329 --> 00:21:20.566
let's look at analysis
a little bit and
00:21:20.566 --> 00:21:22.340
keep putting things
in there. Really
00:21:22.340 --> 00:21:25.335
curious about that one,
that I'd heard not of
00:21:25.335 --> 00:21:27.019
before. We need to
find a lot more about
00:21:27.019 --> 00:21:33.913
that. So, let's look at
needs analysis, and
00:21:33.913 --> 00:21:35.895
let's take a closer look at how we
00:21:35.895 --> 00:21:40.004
determine what is
going on by answering
00:21:40.004 --> 00:21:43.012
two questions to close
that gap. So, we're
00:21:43.012 --> 00:21:44.757
going to look at two
questions to ask
00:21:44.757 --> 00:21:47.790
that we when we see in the graphic
00:21:47.790 --> 00:21:52.723
between performance
and the solution. So
00:21:52.723 --> 00:21:56.952
the first question is
the "why." Why don't
00:21:56.952 --> 00:21:59.997
people perform as
desired? Why don't they
00:21:59.997 --> 00:22:02.074
do what we trained
them to do? Put your
00:22:02.074 --> 00:22:03.817
answers in chat. Why...what are the
00:22:03.817 --> 00:22:05.063
things that you see, or you've
00:22:05.063 --> 00:22:07.719
experienced? Why don't
they do what we
00:22:07.719 --> 00:22:09.777
trained them to do
with training? "They
00:22:09.777 --> 00:22:14.866
did it in training."
"Lazy sometimes." "It's
00:22:14.866 --> 00:22:16.937
not reinforced on the floor."
00:22:21.514 --> 00:22:22.911
"Folks want to take
shortcuts."
00:22:24.843 --> 00:22:27.152
Oh. "The senior staff show
them how it's really
00:22:27.152 --> 00:22:29.549
done." That's pretty
standard when people
00:22:29.549 --> 00:22:32.502
get back to their
workplace that they're
00:22:32.502 --> 00:22:34.656
told and certain times, you know
00:22:34.656 --> 00:22:36.229
this. I know what you
learned at training,
00:22:36.229 --> 00:22:38.476
Forget all that. Here's
how we do it here."
00:22:38.476 --> 00:22:41.504
"Complacency."
00:22:45.395 --> 00:22:48.552
Oh. "Not feeling the training
policies are worthwhile." So they're
00:22:48.559 --> 00:22:51.360
really not connecting
the reason why it
00:22:51.360 --> 00:22:54.101
needs to be done a
certain way, or why
00:22:54.101 --> 00:22:55.557
their tasks need to
be done a certain,
00:22:55.557 --> 00:22:57.576
way with why...why
bother with it doing
00:22:57.576 --> 00:22:59.820
it that way."
00:23:03.174 --> 00:23:06.095
Okay, so... so again, why
don't they do what...
00:23:06.095 --> 00:23:08.371
So, you see there's
multiple reasons, right?
00:23:08.371 --> 00:23:10.222
There's multiple
reasons why we have
00:23:10.222 --> 00:23:14.768
performance gaps, and
it's different from
00:23:14.768 --> 00:23:17.316
person to person. It may be. But, as
00:23:17.316 --> 00:23:20.416
trainers, we want to
look at that. The...
00:23:21.111 --> 00:23:23.798
And so, finding out, that's
our first question,
00:23:23.798 --> 00:23:28.143
is why don't people
perform as desired.
00:23:28.845 --> 00:23:30.494
What's... what's
going on? And you've
00:23:30.494 --> 00:23:32.329
already given us a
list of some of those
00:23:32.329 --> 00:23:33.888
things. What's going
on that they aren't
00:23:33.888 --> 00:23:35.493
performing? So, finding
out what that is
00:23:35.493 --> 00:23:38.510
can be really helpful.
The second one is...
00:23:38.838 --> 00:23:42.650
The second step is...
to begin to create a
00:23:42.650 --> 00:23:44.927
possible solution
to those identified
00:23:44.927 --> 00:23:47.328
performance gaps. And
so, what are some
00:23:47.328 --> 00:23:49.600
possible solutions
to performance gaps?
00:23:49.600 --> 00:23:52.278
And, we... please put
those in chat. What are
00:23:52.278 --> 00:23:54.213
some possible solutions that you've
00:23:54.213 --> 00:23:58.304
tried or experienced
to performance gaps?
00:24:01.293 --> 00:24:04.351
And put those in...
PIPs.
00:24:05.426 --> 00:24:08.833
"I don't know what pips are."
Performance improvement plan.
00:24:08.833 --> 00:24:11.681
That's what... that is
probably. "Evaluation."
00:24:11.681 --> 00:24:14.079
"Post-training." "Quality assurance
00:24:14.079 --> 00:24:15.245
practices.
00:24:16.629 --> 00:24:20.656
Leslie: All kinds of interesting... oh,
"Focus performance meetings." "Regular
00:24:20.656 --> 00:24:22.621
proficiency checks.
There's all kinds of
00:24:22.621 --> 00:24:24.422
things here, Bernie.
Lots of great ideas.
00:24:24.422 --> 00:24:26.598
Don't you love it when
we all learn from
00:24:26.598 --> 00:24:27.978
each other as we're
here?
00:24:29.800 --> 00:24:31.939
Bernie: "Incentivize performerformance,"
okay.
00:24:31.963 --> 00:24:34.149
Leslie: Touchdown.
00:24:40.184 --> 00:24:42.106
Bernie: "Hold supervisors
accountable." If you
00:24:42.106 --> 00:24:43.811
noticed, I've seen a
lot of that in here.
00:24:43.811 --> 00:24:45.942
Supervisors,
supervisors/leadership.
00:24:48.567 --> 00:24:50.815
Leslie: Yeah, so, who should be
coaching back on the
00:24:50.815 --> 00:24:54.646
job after training
is done to make sure
00:24:54.646 --> 00:24:56.703
that performance is
truly transferred
00:24:56.703 --> 00:24:58.436
into the workplace?
Who should be doing
00:24:58.436 --> 00:24:59.433
that everybody?
00:25:00.753 --> 00:25:03.399
Bernie: Oh, that's interesting. "Creating an
environment of safety, so
00:25:03.399 --> 00:25:04.887
staff feel they could
admit when they
00:25:04.887 --> 00:25:08.071
don't understand." Yes,
we all want to be
00:25:08.071 --> 00:25:10.595
seen as competent. So,
it can be difficult
00:25:10.595 --> 00:25:12.323
to admit that we
don't know how to do
00:25:12.323 --> 00:25:14.806
something or can't
do it right.
00:25:16.572 --> 00:25:19.300
Great. Lots of great ideas.
"Get buy-in from the
00:25:19.300 --> 00:25:22.090
staff." Buying in is
important, isn't it?
00:25:22.090 --> 00:25:24.021
Because you can teach
them how to do it.
00:25:24.021 --> 00:25:25.748
But if they don't
really believe it's
00:25:25.748 --> 00:25:27.942
important or valuable
how do you get
00:25:27.942 --> 00:25:30.923
them to actually do it?
Oh, good. Good.
00:25:31.570 --> 00:25:35.020
Bernie: So, we see that some of
the solutions that
00:25:35.020 --> 00:25:36.818
you've tried or worked
with have to do
00:25:36.818 --> 00:25:39.491
with supervisors... have to do with
00:25:39.491 --> 00:25:42.801
leadership. So, training
isn't just about
00:25:42.801 --> 00:25:45.364
creating training,
putting it out in the
00:25:45.364 --> 00:25:47.309
classroom, or however
you want to deliver
00:25:47.309 --> 00:25:50.040
it. And, then they go
and do it. This is...
00:25:50.040 --> 00:25:52.400
Everybody needs to be
involved with this,
00:25:52.400 --> 00:25:55.318
especially supervisors. What do
00:25:55.318 --> 00:25:57.620
supervisors expect
back on the job?
00:25:59.589 --> 00:26:02.175
They may not have the
resources we've talked
00:26:02.175 --> 00:26:04.596
about that. Or authority
needed to do the
00:26:04.596 --> 00:26:08.120
job, or sometimes a
job training changes.
00:26:08.430 --> 00:26:12.772
Oftentimes, if I've
looked back at any
00:26:12.772 --> 00:26:15.924
job description and, I
have, it isn't
00:26:15.924 --> 00:26:19.130
very long until a
year or two -- until my job
00:26:19.130 --> 00:26:21.343
shifts from that.
So, oftentimes, that
00:26:21.343 --> 00:26:23.301
happens. "The job is
changed." "They don't
00:26:23.301 --> 00:26:25.883
get feedback." "Feedback
from supervisors
00:26:25.883 --> 00:26:30.386
or leadership." One way
or the other as to
00:26:30.386 --> 00:26:32.228
whether they're
specifically doing the
00:26:32.228 --> 00:26:34.215
job right or not. Or
"They're punished for
00:26:34.215 --> 00:26:37.585
doing the right thing."
They want it. They,
00:26:37.585 --> 00:26:40.377
oftentimes, will new
pre-service training
00:26:40.377 --> 00:26:41.953
and all excited and
ready to go do their
00:26:41.953 --> 00:26:43.785
job and it's done and then
00:26:43.785 --> 00:26:46.601
I get there and they're
told, "Well, you
00:26:46.601 --> 00:26:47.749
know forget all that
stuff you learned
00:26:47.749 --> 00:26:50.333
at training, here's how
we really do it."
00:26:52.632 --> 00:26:54.246
And so, sometimes they can even be
00:26:54.246 --> 00:26:55.817
rewarded for not doing
in the way they
00:26:55.817 --> 00:26:57.701
were trained. There's
a lot of different
00:26:57.701 --> 00:26:59.717
reasons and a lot of
different ways that
00:26:59.717 --> 00:27:01.886
we can come up with
solutions for that.
00:27:03.982 --> 00:27:08.663
So, there's only one
analysis that should
00:27:08.663 --> 00:27:12.102
lead to training.
One answer to these
00:27:12.102 --> 00:27:13.830
analysis questions
that should lead to
00:27:13.830 --> 00:27:14.387
training...
00:27:15.869 --> 00:27:20.274
...and that is they don't know how
to do it. Because you're right... all the
00:27:20.274 --> 00:27:21.924
things you've listed
in the chat. All
00:27:21.924 --> 00:27:23.838
those things, a lot
of them were not
00:27:23.838 --> 00:27:26.410
training. A lot of
them were other things
00:27:26.410 --> 00:27:28.542
that around training,
or have to do with
00:27:28.542 --> 00:27:30.032
training, but aren't specifically
00:27:30.032 --> 00:27:33.221
training. And that's
all training can
00:27:33.221 --> 00:27:37.124
really do, is teach
them knowledge skills
00:27:38.103 --> 00:27:39.766
and attitude. "KSAs"
00:27:40.290 --> 00:27:42.728
Some of these other things are not going
to get fixed with
00:27:42.837 --> 00:27:43.956
training. We know that.
00:27:45.493 --> 00:27:47.801
So, the only answer that should
lead to the creation of
00:27:47.801 --> 00:27:50.061
training is they don't
know how to do it.
00:27:51.432 --> 00:27:54.069
And this could be so
important. And again
00:27:54.069 --> 00:27:58.394
getting to some solution,
because if you
00:27:58.394 --> 00:28:01.321
aren't focusing on
the right thing,
00:28:01.776 --> 00:28:03.979
then, you're not going to
get the outcomes
00:28:03.979 --> 00:28:06.025
that you want. And we
know there's times
00:28:06.025 --> 00:28:09.063
where we're asked to
create training, and
00:28:09.063 --> 00:28:11.699
it has more to do
with legal issues
00:28:12.359 --> 00:28:15.359
and someone freaking out
over something that
00:28:15.359 --> 00:28:16.666
happened. And quick we got to do
00:28:16.666 --> 00:28:18.531
something, so that
we can show we did
00:28:18.531 --> 00:28:20.673
something! We know
that we're called on
00:28:20.673 --> 00:28:23.105
to do things like that, and we know
00:28:23.105 --> 00:28:26.692
there's times when
that's the main point
00:28:26.692 --> 00:28:27.828
of what's going on.
00:28:30.468 --> 00:28:32.926
As learning professionals in this
field, and you've
00:28:32.926 --> 00:28:34.932
pointed it out, there's times where
00:28:34.932 --> 00:28:37.979
leadership needs to understand what
00:28:37.979 --> 00:28:39.788
training can and can't
do and what can
00:28:39.788 --> 00:28:42.554
be expected of us and what kind of
00:28:42.554 --> 00:28:44.111
solutions we can offer them.
00:28:45.882 --> 00:28:48.353
So, identifying the solution
for the problem
00:28:48.353 --> 00:28:50.452
is training. Let's say
that your answers
00:28:50.452 --> 00:28:53.058
are, it does go to
training. How do we
00:28:53.058 --> 00:28:54.607
figure out what should be in the
00:28:54.607 --> 00:28:57.344
training? How do we
know what the content
00:28:57.344 --> 00:28:58.189
should be?
00:28:58.583 --> 00:29:00.586
Well, one of the ways, this is just
00:29:00.586 --> 00:29:02.363
an example there's
other ways of doing
00:29:02.363 --> 00:29:06.786
this analysis, but
this is one way and
00:29:06.786 --> 00:29:10.134
one part of a way to
develop training.
00:29:10.271 --> 00:29:13.348
So, we're going to say,
we have an issue
00:29:13.348 --> 00:29:15.959
a performance gap.
Here's how we're going
00:29:15.959 --> 00:29:18.438
to address this and we're gonna
00:29:18.438 --> 00:29:20.113
address this in pre-service before
00:29:20.113 --> 00:29:21.871
people get the work.
There's certain
00:29:21.871 --> 00:29:24.073
things they need to
know. One of the
00:29:24.073 --> 00:29:26.235
things that I see often
when I go in the
00:29:26.235 --> 00:29:29.503
field, is training and particularly,
pre-service.
00:29:29.574 --> 00:29:32.142
Pre-service is based on policy
procedure. Policy
00:29:32.142 --> 00:29:34.821
procedure. Policy
procedure. And policy
00:29:34.821 --> 00:29:38.352
procedure can tell
you the limitations
00:29:38.352 --> 00:29:40.818
of what you can and
can't do, but it
00:29:40.818 --> 00:29:43.335
doesn't get to...what it
often doesn't get to
00:29:43.335 --> 00:29:45.121
is what people actually
do on the job.
00:29:45.121 --> 00:29:47.620
And so, one of the
things we use that NIC
00:29:47.620 --> 00:29:50.148
is what's called a DACUM, which is
00:29:50.148 --> 00:29:53.612
really just a job
task analysis. And it
00:29:53.612 --> 00:29:57.035
is just "what does a person do on
00:29:57.035 --> 00:29:59.740
the job." It's the
process of identifying
00:29:59.740 --> 00:30:01.904
and documenting what is done on a
00:30:01.904 --> 00:30:05.260
specific job. What do
they actually do on
00:30:05.260 --> 00:30:08.823
the job? And it's a
process starting with
00:30:08.823 --> 00:30:11.519
the job itself.
00:30:16.279 --> 00:30:18.551
The title...and that's a position or the
title, usually a sentence
00:30:18.551 --> 00:30:21.699
or two. And this is
just a particular job,
00:30:21.699 --> 00:30:28.898
like I'm a correctional program
specialist. That's my title or position. A
00:30:28.898 --> 00:30:31.403
corrections officer
would be a job title.
00:30:32.789 --> 00:30:34.961
And so, that's a basic... that's
00:30:34.961 --> 00:30:36.757
where you start, is
with that job title.
00:30:36.757 --> 00:30:39.134
And having it clearly...
clearly defined
00:30:39.134 --> 00:30:41.984
as possible. Do that
because that can
00:30:41.984 --> 00:30:45.065
change over time too.
I'm sure there's
00:30:45.802 --> 00:30:47.716
people in positions
where you may still
00:30:47.716 --> 00:30:49.209
be called a corrections
officer, but you
00:30:49.209 --> 00:30:52.021
may do other things.
So jobs, starting
00:30:52.021 --> 00:30:53.451
with the job title
and then you work
00:30:53.451 --> 00:30:55.574
your way down through this process
00:30:55.574 --> 00:30:57.816
called a DACUM.
Developing a curriculum.
00:30:59.322 --> 00:31:02.142
And as you work your
way down it, within
00:31:02.142 --> 00:31:04.532
that job, what are
the duties? And you
00:31:04.532 --> 00:31:06.072
know what duty is
something that's a
00:31:06.072 --> 00:31:08.387
responsibility, but
it has no discrete
00:31:08.387 --> 00:31:10.703
beginning or ending.
It's just ongoing.
00:31:11.135 --> 00:31:14.532
So, part of corrections
officer's duties
00:31:14.532 --> 00:31:17.183
would be security and
00:31:17.183 --> 00:31:20.582
safety of inmates in
the facility. That
00:31:20.582 --> 00:31:23.793
would be a duty. That
is ongoing. That
00:31:23.793 --> 00:31:27.265
never stops. That's
always part of their
00:31:27.265 --> 00:31:30.894
job. Within a duty, there
are tasks, and the
00:31:30.894 --> 00:31:32.644
tasks have a discrete beginning and
00:31:32.644 --> 00:31:35.038
ending and they're what you do to
00:31:35.038 --> 00:31:37.902
fulfill that. And
one of those may be
00:31:37.902 --> 00:31:41.292
count. Count could
be a task. Running
00:31:41.292 --> 00:31:45.474
count. That would be a
task. So, that would
00:31:45.474 --> 00:31:47.745
be a task that you
do to fulfill your
00:31:47.745 --> 00:31:49.995
duty, which is security
and safety within
00:31:49.995 --> 00:31:52.513
the institution. And
so, you get down to
00:31:52.513 --> 00:31:55.781
task and within a
task there steps.
00:31:56.283 --> 00:31:58.001
There's a certain
way to do a count.
00:31:58.868 --> 00:32:00.454
There's certain...
there may be,
00:32:00.454 --> 00:32:02.681
there may be paperwork
involved. There may be...
00:32:03.080 --> 00:32:05.180
But, there's certain steps to
doing that. And when
00:32:05.180 --> 00:32:06.850
you do a DACUM, you
get all the way to
00:32:06.850 --> 00:32:08.774
task. When it comes
to steps you do it
00:32:08.774 --> 00:32:11.568
after that. The DACUM
process for us at
00:32:11.568 --> 00:32:14.670
NIC, is...take several days, you get a
00:32:14.670 --> 00:32:18.012
group of high performers
that enter in
00:32:18.012 --> 00:32:20.228
that position. You
work your way through
00:32:20.228 --> 00:32:23.642
these steps to find
out what they do on
00:32:23.642 --> 00:32:25.876
a regular basis. And
then, once you get to
00:32:25.876 --> 00:32:29.706
tasks and you take
your DACUM charts,
00:32:30.228 --> 00:32:31.736
you take your policies
and procedures,
00:32:31.736 --> 00:32:33.297
you take all the other
things you need
00:32:33.297 --> 00:32:35.277
to create training,
and that's what you
00:32:35.277 --> 00:32:36.965
use to create your
curriculum.
00:32:37.454 --> 00:32:40.573
Because, you want to get to
steps. You want to
00:32:40.573 --> 00:32:42.782
get to behaviors. You
want to get to the
00:32:42.782 --> 00:32:44.704
place where you know what you can
00:32:44.704 --> 00:32:47.610
teach them... what
they actually do. And
00:32:47.610 --> 00:32:49.192
I've seen curriculum
that doesn't get
00:32:49.192 --> 00:32:52.305
there at all. I mean, I've seen
00:32:52.305 --> 00:32:55.000
pre-service curriculum
where it's the
00:32:55.000 --> 00:32:56.694
policy and procedure dumped on a
00:32:56.694 --> 00:33:00.234
powerpoint and read
off to the class.
00:33:01.805 --> 00:33:03.044
Which is not going to be terribly
00:33:03.044 --> 00:33:05.041
helpful for that new
corrections officer
00:33:05.041 --> 00:33:07.294
when they have to go
in that first day
00:33:07.294 --> 00:33:08.435
and do what they do.
00:33:10.270 --> 00:33:12.255
So, that's a Dacum...
that's an example of
00:33:12.255 --> 00:33:13.622
one way to do that.
00:33:16.522 --> 00:33:19.879
So, I don't know if anybody, if you
put in a chat, whether you
00:33:19.879 --> 00:33:21.506
use a DACUM or not. You may do a
00:33:21.506 --> 00:33:24.562
different job task
analysis. And so on.
00:33:27.381 --> 00:33:31.168
Somebody lost audio.
Okay, so, that's one
00:33:31.168 --> 00:33:33.110
way to do this analysis.
There are other
00:33:33.110 --> 00:33:34.720
ways. If there's other
ways that you do
00:33:34.720 --> 00:33:36.321
this let us know.
00:33:38.290 --> 00:33:43.638
Then, we move to the middle part of ADDIE
because we're
00:33:43.638 --> 00:33:46.625
focusing, again, on A
and E. So, we're kind
00:33:46.625 --> 00:33:48.912
of jumping over the middle part.
00:33:49.492 --> 00:33:52.628
Not because it's unimportant. It is
00:33:52.628 --> 00:33:54.937
important. But it's
usually the part that
00:33:54.937 --> 00:33:57.138
we focus on and we
forget the A and the
00:33:57.138 --> 00:34:02.294
E. Or we can't do it.
So we're just gonna
00:34:02.294 --> 00:34:04.451
work-- Again, hopping
over the top of
00:34:04.451 --> 00:34:08.098
this because we're
very familiar, most of
00:34:08.098 --> 00:34:09.405
us, with the middle
part of this.
00:34:11.121 --> 00:34:14.525
So, the design part is actually
creating this.
00:34:14.525 --> 00:34:16.269
You take everything
you have around you,
00:34:16.269 --> 00:34:18.347
like I said, the DACUM if you have
00:34:18.347 --> 00:34:19.326
that, and so on,
00:34:20.730 --> 00:34:24.045
You create your outline. You
come up with your performance
00:34:24.045 --> 00:34:25.482
objectives. And these are really
00:34:25.482 --> 00:34:28.737
important here, because
you can't measure
00:34:28.737 --> 00:34:30.595
something that you don't have as a
00:34:30.595 --> 00:34:32.108
performance objective.
What are you gonna
00:34:32.108 --> 00:34:35.504
measure? So, need
to have those up
00:34:35.504 --> 00:34:37.012
front. And then, how
are you going to
00:34:37.012 --> 00:34:39.697
deliver this? The
development part is
00:34:39.697 --> 00:34:41.452
where you're actually
starting to create
00:34:41.452 --> 00:34:44.535
the pieces of it
that you use in your
00:34:44.535 --> 00:34:46.667
lesson plans, your
facilitator guide, your
00:34:46.667 --> 00:34:49.262
participant guide,
your slides, all your
00:34:49.262 --> 00:34:49.895
materials.
00:34:51.301 --> 00:34:54.853
And also, creating your evaluation tools.
Because evaluation,
00:34:54.853 --> 00:34:56.739
again, needs to be
all the way through
00:34:56.739 --> 00:34:59.119
here. And then implementing.
This is where
00:34:59.119 --> 00:34:59.952
you're piloting.
00:35:03.230 --> 00:35:08.661
Piloting your training on getting some feedback,
evaluating the
00:35:08.661 --> 00:35:12.086
process, and also evaluation phase.
00:35:12.133 --> 00:35:14.710
And see are they
understanding, are they
00:35:14.710 --> 00:35:17.510
getting it? So, this
is the middle part of
00:35:17.510 --> 00:35:19.828
ADDIE, and again we're
kind of skipping
00:35:19.828 --> 00:35:22.796
over this quickly
because it is the part
00:35:22.796 --> 00:35:23.992
we tend to focus on.
00:35:25.031 --> 00:35:29.602
And so, in talking about these parts,
talking about the
00:35:29.602 --> 00:35:32.891
analysis, we're now
going to move to
00:35:32.891 --> 00:35:33.912
evaluation.
00:35:35.953 --> 00:35:38.925
Jeff: Thanks, Bernie. As we begin to
explore the
00:35:38.925 --> 00:35:41.331
last piece of the ADDIE process,
00:35:41.331 --> 00:35:45.422
"evaluation", I want
you guys to think
00:35:45.422 --> 00:35:47.828
about how you guys are working on
00:35:47.828 --> 00:35:50.667
evaluation in your
agency. How do you
00:35:50.667 --> 00:35:53.971
define training in
your agency? In your
00:35:53.971 --> 00:35:55.961
current position? Thinking about,
00:35:56.000 --> 00:35:58.312
however, you guys do
evaluation. How would
00:35:58.312 --> 00:36:00.351
you define training
evaluation?
00:36:06.164 --> 00:36:08.070
You guys can go ahead and chat
me some responses
00:36:08.070 --> 00:36:10.968
in there. Anything you
guys are doing for
00:36:10.968 --> 00:36:12.203
evaluating training.
00:36:14.999 --> 00:36:16.612
Leslie: "Job performance," says Louie.
00:36:20.924 --> 00:36:23.283
"Connect to the mission and vision."
00:36:26.252 --> 00:36:30.385
"Does it do what we designed it to do and
does it give the skills we wanted?"
00:36:30.735 --> 00:36:33.063
Ah! "A good video system."
00:36:37.563 --> 00:36:41.573
Jeff: Alright, so it looks like we've got a
pretty varied way of
00:36:41.573 --> 00:36:45.161
doing some evaluations.
Everything from a
00:36:45.388 --> 00:36:47.302
pre and post assessment,
survey responses,
00:36:47.302 --> 00:36:49.058
all kinds of good...
good answers there.
00:36:49.058 --> 00:36:53.751
Okay, before we move on too far
00:36:53.751 --> 00:36:55.512
past this, what I'd
like to do is... let's
00:36:55.512 --> 00:36:58.865
talk about how OPM actually defines
00:36:58.865 --> 00:37:01.826
training evaluation
and let's see how it
00:37:01.826 --> 00:37:05.117
either compares or
contrasts with what
00:37:05.117 --> 00:37:07.251
we've already answered,
here. So, the OPM
00:37:07.251 --> 00:37:09.384
says, "Training evaluation
is a continual
00:37:09.384 --> 00:37:11.721
and systematic process
of assessing the
00:37:11.721 --> 00:37:13.705
value or potential
value of a training
00:37:13.705 --> 00:37:16.091
program, course, activity
or event.
00:37:17.173 --> 00:37:19.211
So, yeah. I think it looks like a lot of the
00:37:19.211 --> 00:37:21.919
connections there... a
lot of these surveys...
00:37:21.919 --> 00:37:24.667
A lot of these pre and post-tests,
00:37:24.667 --> 00:37:26.805
evaluations and things
are going to kinda
00:37:26.805 --> 00:37:27.822
tie together with
this.
00:37:28.630 --> 00:37:31.828
Now, that we have defined training
evaluation and the
00:37:31.828 --> 00:37:33.793
many facets of it.
Let's take a look at
00:37:33.793 --> 00:37:35.972
what we can do with
some of the results
00:37:35.972 --> 00:37:37.676
of the training
evaluation.
00:37:38.504 --> 00:37:41.328
So, I'd like you guys to take a
look at this poll
00:37:41.328 --> 00:37:43.574
question. It answers
A through H and
00:37:43.574 --> 00:37:45.796
you may have to
scroll down. Leslie's
00:37:45.796 --> 00:37:47.577
going to open up
poll on the side for
00:37:47.577 --> 00:37:48.476
you to answer on.
00:37:49.234 --> 00:37:52.911
And, read through this, and what
do you think we can do
00:37:52.911 --> 00:37:54.244
with the results of a training
00:37:54.244 --> 00:37:55.187
evaluation?
00:37:56.726 --> 00:38:00.491
Leslie: And, you may have to open up the
polling function on your screen.
00:38:00.491 --> 00:38:03.562
They'll be... a polling
will pop up and
00:38:03.562 --> 00:38:05.656
hopefully, the poll has
popped up for you
00:38:05.656 --> 00:38:08.527
in the right hand
side of your screen.
00:38:08.914 --> 00:38:12.006
Just click on the
little polling word
00:38:12.006 --> 00:38:14.256
and it should pop up
for you. Let me open
00:38:14.256 --> 00:38:16.099
up the poll and it
is open.
00:38:16.646 --> 00:38:17.284
Jeff: Thank you.
00:38:28.577 --> 00:38:30.416
Leslie: We do have people
responding, Jeff.
00:38:30.983 --> 00:38:33.505
Jeff: All right. And if you're
not seeing the poll
00:38:33.505 --> 00:38:35.194
for some reason, if you click on the
00:38:35.194 --> 00:38:37.493
little three dots at
the bottom in your
00:38:37.493 --> 00:38:39.565
little tools, you'll
see an option where
00:38:39.565 --> 00:38:40.925
you can turn the polling
on.
00:38:42.016 --> 00:38:43.753
Just in case it's not showing up
for you.
00:38:48.823 --> 00:38:50.862
Leave it open just a little bit
longer, Leslie and
00:38:50.862 --> 00:38:52.403
see if we get some
more responses in.
00:38:52.497 --> 00:38:55.026
Leslie: Now, 32 out of 84 respond.
00:38:55.294 --> 00:38:55.620
Jeff: Ok.
00:38:58.536 --> 00:38:59.427
Leslie: Numbers are going up.
00:39:01.075 --> 00:39:02.444
In about another 20 seconds.
00:39:02.468 --> 00:39:07.306
Leslie: Got about half of people respond.
00:39:13.568 --> 00:39:14.923
About another 10 seconds.
00:39:29.095 --> 00:39:30.557
Okay, I'm closing the
poll.
00:39:36.661 --> 00:39:37.309
Calculating.
00:39:50.857 --> 00:39:54.193
Jeff: And while we are waiting for this to
calculate, I'd like you guys to think about what
00:39:54.193 --> 00:39:56.690
else... are there any other things that
we can do with the
00:39:56.690 --> 00:39:58.709
results of a training
evaluation? So if
00:39:58.709 --> 00:40:00.514
we're running training
evaluations, what
00:40:00.514 --> 00:40:02.334
else can we do? We see some of the
00:40:02.334 --> 00:40:04.107
answers here, but are
there other answers
00:40:04.107 --> 00:40:06.022
that we're thinking
about. Think about
00:40:06.022 --> 00:40:06.600
some of those.
00:40:08.175 --> 00:40:09.511
Leslie: Poll results should be up.
00:40:10.566 --> 00:40:11.473
Bernie: Leslie, I see we have a
few people...
00:40:13.863 --> 00:40:15.678
...about "determine the
effectiveness."
00:40:15.886 --> 00:40:18.932
A majority of our
people are "all of the
00:40:18.932 --> 00:40:22.504
above." They identified
"all of the above."
00:40:22.827 --> 00:40:24.726
It is overwhelming
majority Leslie.
00:40:25.687 --> 00:40:26.180
And Jeff.
00:40:26.679 --> 00:40:32.515
Thank you, Bernie. Alright. So, we can
go ahead and close out our poll. If you guys
00:40:32.515 --> 00:40:34.797
can minimize your
polling on the left or
00:40:34.797 --> 00:40:36.180
on the right hand
side there, you can
00:40:36.180 --> 00:40:38.075
either do it by that
down arrow next to
00:40:38.075 --> 00:40:39.847
polling, or you can
click on the X next
00:40:39.847 --> 00:40:40.932
to the word "Polling"
and that will get
00:40:40.932 --> 00:40:42.532
rid of that. You can open your chat
00:40:42.532 --> 00:40:45.009
window back up. If you guys have any
00:40:45.009 --> 00:40:47.299
answers on what else
you can do. Is there
00:40:47.299 --> 00:40:48.299
something that you're doing with
00:40:48.299 --> 00:40:49.635
training evaluations that we didn't
00:40:49.635 --> 00:40:50.571
already talk about,
00:40:50.854 --> 00:40:52.674
we'd be curious to
share those with the
00:40:52.674 --> 00:40:53.590
rest of the group.
00:40:59.222 --> 00:41:02.498
"Identifying training gaps and needs." Good.
Yeah, there's a lot
00:41:02.498 --> 00:41:05.452
of different things
we can do. We can use
00:41:05.452 --> 00:41:06.878
them to guide
decision-making around
00:41:06.878 --> 00:41:09.744
various components
of training, such as
00:41:09.744 --> 00:41:11.775
instructional design,
delivery results,
00:41:11.775 --> 00:41:13.753
and its overall continuation,
00:41:13.753 --> 00:41:15.480
modification, or even
elimination.
00:41:15.761 --> 00:41:18.346
Think about that. We can
determine if the
00:41:18.346 --> 00:41:20.074
program's objectives
were met.
00:41:20.886 --> 00:41:22.593
Assess whether or not
training was properly
00:41:22.593 --> 00:41:24.350
implemented. So, that's
something else to
00:41:24.350 --> 00:41:26.732
think about. Its effectiveness and
00:41:26.732 --> 00:41:27.209
ability.
00:41:28.240 --> 00:41:28.698
Good.
00:41:30.154 --> 00:41:33.624
All right. Now, that we've discussed what we
can do with the results of training
00:41:33.624 --> 00:41:38.932
evaluation, why then is evaluation such
a critical step in
00:41:38.932 --> 00:41:40.218
the ADDIE process?
00:41:41.058 --> 00:41:44.087
So, why do you think it's so critical that
we that we don't
00:41:44.087 --> 00:41:46.528
overlook this phase?
And a lot of times we
00:41:46.528 --> 00:41:48.967
do that, right? So, what
do you think? What?
00:41:48.967 --> 00:41:50.785
Why is it so critical that we don't
00:41:50.785 --> 00:41:53.221
overlook the evaluation
piece of it?
00:41:57.963 --> 00:41:58.485
Okay.
00:42:00.502 --> 00:42:04.372
"Identifying material to see if meets
the current issues." Excellent response.
00:42:05.005 --> 00:42:07.060
"Efficient use of resources." Great, Maron.
00:42:07.060 --> 00:42:08.225
Appreciate that response.
00:42:08.895 --> 00:42:10.251
"Program effectiveness." Good.
00:42:11.106 --> 00:42:11.725
Definitely,
00:42:12.708 --> 00:42:16.389
Ah! "Many times, what's on paper doesn't
work in practice." We've seen that,right?
00:42:16.426 --> 00:42:18.976
"Policy comes down from people sitting
in headquarters and
00:42:18.976 --> 00:42:20.433
it doesn't work behind the wire." So,
00:42:20.433 --> 00:42:22.254
there's a lot of times
that that happens.
00:42:23.497 --> 00:42:24.996
Some good responses coming in here.
00:42:25.356 --> 00:42:26.853
"Training's expensive.
You have to get it
00:42:26.853 --> 00:42:28.727
right." That's exactly
a good point.
00:42:30.572 --> 00:42:31.949
If we won't evaluate the
results, we won't know
00:42:31.949 --> 00:42:33.830
if it works." We see that all the
00:42:33.830 --> 00:42:35.987
time, right? Unfortunately, many
00:42:35.987 --> 00:42:38.942
organizations do misunderstand the
00:42:38.942 --> 00:42:43.193
evaluation stage.
Sometimes we believe
00:42:43.193 --> 00:42:44.956
that if you evaluate learning by
00:42:44.956 --> 00:42:46.815
collecting the feedback
forms, then the
00:42:46.815 --> 00:42:49.115
process is done. But
we often know that
00:42:49.115 --> 00:42:50.593
that's not enough. One-and-done
00:42:50.593 --> 00:42:52.512
evaluations are not
the only part of the
00:42:52.512 --> 00:42:54.781
evaluation process, as it doesn't
00:42:54.781 --> 00:42:57.116
consider the long-term
impact of the
00:42:57.116 --> 00:42:59.087
training. So, as someone
said, the training
00:42:59.087 --> 00:43:00.384
is expensive. We want
to make sure that
00:43:00.384 --> 00:43:02.094
we get this done
right and what's the
00:43:02.094 --> 00:43:03.584
long-term impact
going to look like?
00:43:04.048 --> 00:43:06.643
Organizations that
do the evaluation
00:43:06.643 --> 00:43:08.902
process will continually
evaluate the
00:43:08.902 --> 00:43:10.735
impact the training is having on
00:43:10.735 --> 00:43:13.130
employees, the department,
and the entire
00:43:13.130 --> 00:43:15.740
organization. They do
this to ensure that
00:43:15.740 --> 00:43:17.918
the training is
achieving the desired
00:43:17.918 --> 00:43:21.124
results. If the
training is not doing
00:43:21.124 --> 00:43:23.608
this, then it is
changed or replaced
00:43:23.608 --> 00:43:25.256
with something more
appropriate.
00:43:25.795 --> 00:43:28.229
Now, sometimes that means
starting over with
00:43:28.229 --> 00:43:30.013
the ADDIE process again because a
00:43:30.013 --> 00:43:31.272
different direction is needed.
00:43:34.625 --> 00:43:35.734
Leslie: Alright. Thanks Jeff.
00:43:37.062 --> 00:43:41.216
So, now, that we've defined evaluation,
and we've explored
00:43:41.216 --> 00:43:44.648
the criticality of it
in its uses, now we
00:43:44.648 --> 00:43:46.945
need to move on to how to do it
00:43:46.945 --> 00:43:49.623
effectively. This
is where the rubber
00:43:49.623 --> 00:43:52.709
meets the road. As we
know, we just cannot
00:43:52.709 --> 00:43:55.681
stop at a participant
feedback form, if we
00:43:55.681 --> 00:43:58.410
truly want to measure
the impact and
00:43:58.410 --> 00:44:02.014
effectiveness of our
training. So how do
00:44:02.014 --> 00:44:04.285
we do it in the deliberate and
00:44:04.285 --> 00:44:06.770
structured way? And
what types of tools
00:44:06.770 --> 00:44:09.824
and evaluation methods should
00:44:09.824 --> 00:44:11.452
we really consider
using?
00:44:12.176 --> 00:44:13.557
Well, we here, at NIC,
00:44:13.557 --> 00:44:15.706
use Kirkpatrick's levels of
00:44:15.706 --> 00:44:18.285
evaluation. And what
we're going to do
00:44:18.285 --> 00:44:21.822
now, is take a look
at each one of those
00:44:21.822 --> 00:44:25.426
four levels. And some
examples of how you
00:44:25.426 --> 00:44:27.671
can measure each one
of the four levels.
00:44:29.672 --> 00:44:33.237
So, the first level,
level one, is reaction.
00:44:33.237 --> 00:44:35.695
And this is where
you're measuring the
00:44:35.695 --> 00:44:37.809
degree to which
participants find the
00:44:37.809 --> 00:44:40.542
training favorable,
engaging, and relevant
00:44:40.542 --> 00:44:41.405
to their jobs.
00:44:42.647 --> 00:44:45.718
So, when you look at that first little picture
there on the left,
00:44:45.718 --> 00:44:47.981
that's a person. That's
supposed to be a
00:44:47.981 --> 00:44:50.745
participant with
that wonderful ideal
00:44:50.745 --> 00:44:54.924
light bulb. And that's
an example of a
00:44:54.924 --> 00:44:58.367
trainer or a curriculum
designer, who is
00:44:58.367 --> 00:45:01.533
designed in pulse checks as an
00:45:01.533 --> 00:45:05.296
evaluation strategy.
And pulse checks are
00:45:05.296 --> 00:45:07.860
a facilitator determining
the level of
00:45:07.860 --> 00:45:10.374
engagement or knowledge
of participants
00:45:10.374 --> 00:45:12.779
during the training by observing,
00:45:12.779 --> 00:45:15.509
questioning, or
conducting any number of
00:45:15.509 --> 00:45:19.659
large or small group
activities. And so,
00:45:19.659 --> 00:45:22.833
this is actually
something any of these
00:45:22.833 --> 00:45:25.410
different evaluation
strategies that i'm
00:45:25.410 --> 00:45:27.386
going to be actually
sharing with you at
00:45:27.386 --> 00:45:29.884
each one of these four
levels, are things
00:45:29.884 --> 00:45:31.939
that would actually
be designed as the
00:45:31.939 --> 00:45:34.884
training is the bus
designed all along
00:45:34.884 --> 00:45:37.224
the way. So, the first
one there is a
00:45:37.224 --> 00:45:39.405
pulse check. And by the way, the
00:45:39.405 --> 00:45:42.713
definitions of all of
these are included
00:45:42.713 --> 00:45:45.994
in the white paper, in
which, this webinar
00:45:45.994 --> 00:45:49.813
is based on, as well
as, the OPM document
00:45:49.813 --> 00:45:51.872
that is in the
resources of the white
00:45:51.872 --> 00:45:52.306
paper.
00:45:53.322 --> 00:45:58.405
That second little picture there is supposed
to be kind of getting us to
00:45:58.405 --> 00:46:00.585
think about using surveys and
00:46:00.585 --> 00:46:04.420
questionnaires as a way
to measure level one.
00:46:04.420 --> 00:46:05.914
And that's kind of a time-honored
00:46:05.914 --> 00:46:09.528
way that we are used
to in a traditional
00:46:09.528 --> 00:46:12.702
way to measure level
one. But, there's all
00:46:12.702 --> 00:46:15.855
kinds of creative
ways to do level one
00:46:15.855 --> 00:46:18.381
surveys and questionnaires.
Some people
00:46:18.381 --> 00:46:21.296
like to do end-of-day
reactions. I know
00:46:21.296 --> 00:46:24.352
I'm a fan of doing
those. A lot of times,
00:46:24.352 --> 00:46:26.523
I like to get people
to self evaluate
00:46:26.587 --> 00:46:27.570
where they are in
00:46:27.570 --> 00:46:29.215
terms of their own
learning and what
00:46:29.215 --> 00:46:31.102
they need to do to
get the most out of
00:46:31.102 --> 00:46:34.157
the training, in
addition to, the typical
00:46:35.227 --> 00:46:38.372
closed-ended, open-ended,
end of course
00:46:38.372 --> 00:46:41.864
surveys and questionnaires.
In general,
00:46:41.864 --> 00:46:44.758
surveys are more
generally closed-ended,
00:46:44.758 --> 00:46:46.891
and questionnaires
have a tendency to be
00:46:46.891 --> 00:46:49.067
more-- be more open-ended. So, there's
00:46:49.067 --> 00:46:51.706
another way you can
measure level one.
00:46:52.238 --> 00:46:54.613
And then, that third
one, those people,
00:46:54.613 --> 00:46:56.937
they're talking
together in a group.
00:46:56.937 --> 00:46:59.964
That's an example of
using individual or
00:46:59.964 --> 00:47:03.637
group interviews. So,
you can open two-way
00:47:03.637 --> 00:47:06.244
communication channels about the
00:47:06.244 --> 00:47:08.248
training, the content, the delivery
00:47:08.248 --> 00:47:10.869
strategies. So, you
get lots of really
00:47:10.963 --> 00:47:13.088
valuable information as training
00:47:13.088 --> 00:47:16.177
designers and
facilitators about what
00:47:16.177 --> 00:47:17.431
people are learning. What's really
00:47:17.431 --> 00:47:20.353
working for them.
Because, sometimes if
00:47:20.353 --> 00:47:22.196
you just rely on a survey or
00:47:22.196 --> 00:47:23.564
questionnaire, you're only getting
00:47:23.564 --> 00:47:25.467
one-way communication
and it's kind of
00:47:25.467 --> 00:47:28.694
open to interpretation
about exactly
00:47:28.694 --> 00:47:30.890
what some of the
information means.
00:47:31.329 --> 00:47:36.031
So, chat to us, how do
you measure level one
00:47:36.031 --> 00:47:38.571
in your agency? What
are you doing to
00:47:38.571 --> 00:47:41.062
measure reaction,
the degree to which
00:47:41.062 --> 00:47:43.125
participants find the
training favorable,
00:47:43.125 --> 00:47:45.482
engaging, and relevant
to their jobs?
00:47:50.818 --> 00:47:52.767
Bernie: "Survey monkey software."
00:47:53.657 --> 00:47:59.076
Oh, survey monkey... Survey monkey is
really easy isn't it?
00:47:59.076 --> 00:48:01.416
It's easy for the
person putting it out
00:48:01.416 --> 00:48:05.322
and really easy to
do the analysis.
00:48:05.752 --> 00:48:08.643
Leslie: Ah! "Open forum with new
hires." Alright,
00:48:08.643 --> 00:48:09.246
Sarah.
00:48:11.877 --> 00:48:15.682
Bernie: Just so you know, there was a
discussion about level one and the value
00:48:15.682 --> 00:48:17.942
of it or not. And at
the end of this, when
00:48:17.942 --> 00:48:20.012
we have a few minutes
for questions, we
00:48:20.012 --> 00:48:22.770
can address that. So,
hang on to that idea.
00:48:22.895 --> 00:48:28.331
And "Verbal feedback."
Absolutely. "Dialogue"
00:48:28.411 --> 00:48:30.965
absolutely. We would
encourage you to do
00:48:30.965 --> 00:48:34.172
just more than surveys
or questionnaires
00:48:35.092 --> 00:48:38.878
to get more substantive
feedback and
00:48:38.878 --> 00:48:40.920
measurement for level one. We would
00:48:40.920 --> 00:48:43.356
totally encourage you
to do more.
00:48:43.818 --> 00:48:47.190
"Really find out things." There
you go.
00:48:48.112 --> 00:48:52.312
Alright, level two, which is
measuring learning.
00:48:52.924 --> 00:48:55.217
The increase of intended knowledge,
00:48:55.217 --> 00:48:57.751
skills, and attitudes,
as a result of
00:48:57.751 --> 00:48:59.143
participating in the
training. Because
00:48:59.143 --> 00:49:02.051
remember, that's what
Bernie said, is the
00:49:02.051 --> 00:49:03.668
only reason why we
would design training
00:49:03.668 --> 00:49:05.933
and use training as a
solution is because
00:49:05.933 --> 00:49:08.923
we want to increase
and change knowledge,
00:49:08.923 --> 00:49:11.607
skills, and attitudes or KSAs.
00:49:11.965 --> 00:49:14.096
So, at level two, that's
what we're actually
00:49:14.096 --> 00:49:16.418
trying to measure.
So, this first little
00:49:16.418 --> 00:49:18.594
person here, they are
actually working on
00:49:18.594 --> 00:49:21.747
the job. So, this is
an example of using
00:49:21.747 --> 00:49:23.976
skills and behavior
observations.
00:49:24.455 --> 00:49:27.025
And, that is where an expert,
peer, or
00:49:27.025 --> 00:49:30.303
supervisor observes
and documents if an
00:49:30.303 --> 00:49:32.321
employee is performing
the appropriate
00:49:32.321 --> 00:49:34.774
skills on the job
completely and correctly.
00:49:35.244 --> 00:49:38.079
And, that would be
using a checklist.
00:49:38.079 --> 00:49:42.257
So, that's a creative and
different way of
00:49:42.257 --> 00:49:44.752
measuring learning
as a result of the
00:49:44.752 --> 00:49:46.836
training that you're
delivering. So again,
00:49:46.836 --> 00:49:48.985
that evaluation strategy would be
00:49:48.985 --> 00:49:52.640
designed into the
training as you are
00:49:52.640 --> 00:49:55.556
designing it. On the
middle one is an
00:49:55.556 --> 00:50:00.366
example of using teach
backs as a way to
00:50:00.366 --> 00:50:04.323
evaluate the learning and behavior
00:50:04.323 --> 00:50:07.664
change as a result
of the training. And
00:50:07.664 --> 00:50:11.022
teach backs are when
a presentation is
00:50:11.022 --> 00:50:12.510
prepared and delivered to others to
00:50:12.510 --> 00:50:14.748
demonstrate the employees
new knowledge,
00:50:14.748 --> 00:50:16.952
skills, and attitudes.
This can be done
00:50:16.952 --> 00:50:20.272
during the training
itself, such as, you
00:50:20.272 --> 00:50:21.710
know, any kind of new knowledge and
00:50:21.710 --> 00:50:23.468
skills. I always
think of our trainer
00:50:23.468 --> 00:50:26.257
development, training
that we do. But I
00:50:26.257 --> 00:50:27.765
can think this would work with any
00:50:27.765 --> 00:50:30.938
skills based training
particularly. And
00:50:30.938 --> 00:50:33.160
it also could be done
to peers back on
00:50:33.160 --> 00:50:36.020
the job while a...
yes there's a learning
00:50:36.020 --> 00:50:40.954
theory. Absolutely,
Stan. Also, with a
00:50:40.954 --> 00:50:44.439
peer supervisor, watching, using a
00:50:44.439 --> 00:50:47.142
checklist for
evaluation. And then of
00:50:47.142 --> 00:50:48.697
course, time-honoured
and traditional
00:50:48.697 --> 00:50:51.556
pre-test and post-test
here. But there's
00:50:51.556 --> 00:50:52.793
lots of other ideas
00:50:52.793 --> 00:50:54.811
in the white paper, in the OPM
00:50:54.811 --> 00:50:58.257
document. For level
three, again, this is
00:50:58.257 --> 00:51:00.063
where learning and
behavior change kind
00:51:00.063 --> 00:51:03.435
of work together. And
there's one here
00:51:03.435 --> 00:51:06.112
that is shared that some skills
00:51:06.112 --> 00:51:08.091
observations on the
job. We have our
00:51:08.091 --> 00:51:10.341
little officer there at the control
00:51:10.341 --> 00:51:14.725
station. That's a
really fantastic way to
00:51:14.725 --> 00:51:17.787
measure if behavior
change, if the new
00:51:17.787 --> 00:51:19.481
knowledge skills and attitudes are
00:51:19.481 --> 00:51:21.788
actually being applied on the job.
00:51:21.788 --> 00:51:24.103
I, personally, can't
think of a better way
00:51:24.103 --> 00:51:28.397
to find out if that
is happening. The one
00:51:28.397 --> 00:51:30.451
in the middle is...
it might be a new
00:51:30.451 --> 00:51:32.218
one. It might be new
to you, and this is a
00:51:32.218 --> 00:51:36.151
work review. An expert,
peer, or supervisor
00:51:36.151 --> 00:51:38.028
reviews the employee's
work on the job
00:51:38.028 --> 00:51:40.146
and documents if the work is being
00:51:40.146 --> 00:51:43.970
performed correctly.
And that would be,
00:51:43.970 --> 00:51:46.678
probably, according
to policy, practice,
00:51:46.678 --> 00:51:49.465
and standards, generally,
using a checklist
00:51:49.465 --> 00:51:52.280
and a template. The on-the-job
00:51:52.280 --> 00:51:55.724
observation, that first
one to the left,
00:51:55.724 --> 00:51:58.091
that's a shorter term.
The work review is
00:51:58.091 --> 00:52:00.070
of longer duration,
looking at work,
00:52:00.070 --> 00:52:02.726
dockets, documents,
work product over a
00:52:02.726 --> 00:52:05.675
longer period of time.
The third one to
00:52:05.675 --> 00:52:08.020
the right, you notice
the people working
00:52:08.020 --> 00:52:11.173
together in a group. A level one...three
00:52:11.173 --> 00:52:14.157
behavior change is actually action
00:52:14.157 --> 00:52:18.151
learning, so again,
on-the-job behavior
00:52:18.151 --> 00:52:20.845
change, what new
knowledge, skills, and
00:52:20.845 --> 00:52:22.804
attitudes are being
applied back on the
00:52:22.804 --> 00:52:24.483
job as a result of the training.
00:52:24.843 --> 00:52:26.966
Participants create
a plan that they
00:52:26.966 --> 00:52:28.920
commit to carry out
when the training is
00:52:28.920 --> 00:52:29.628
over.
00:52:29.883 --> 00:52:32.074
Ideally, a supervisor
would follow up on
00:52:32.074 --> 00:52:33.889
the plan and provide coaching and
00:52:33.889 --> 00:52:36.192
reinforcement. This
is actually something
00:52:36.192 --> 00:52:40.518
that we do in our
leadership programs. We
00:52:40.518 --> 00:52:42.679
have action learning
projects, we work
00:52:42.679 --> 00:52:44.604
with the agencies
that we're working
00:52:44.604 --> 00:52:47.599
with to help develop
those. Those are
00:52:47.599 --> 00:52:51.587
real agency projects
to impact something
00:52:51.587 --> 00:52:53.588
new that needs to
happen in the agency,
00:52:53.960 --> 00:52:55.317
and they practice their new
00:52:55.317 --> 00:52:58.699
leadership skills on
the job with their
00:52:58.699 --> 00:53:01.843
peers, with supervisors,
with leaders in
00:53:01.843 --> 00:53:04.524
the agency. So, that's
something to think
00:53:04.524 --> 00:53:05.181
about.
00:53:05.181 --> 00:53:06.759
So, how are you currently
00:53:06.759 --> 00:53:09.122
actually measuring leveltwo2 and
00:53:09.122 --> 00:53:13.824
level three in your agencies? Just chat
00:53:13.824 --> 00:53:20.094
those ideas to us.
And we don't want to
00:53:20.094 --> 00:53:25.258
leave out level four.
This is the one we
00:53:25.258 --> 00:53:27.808
always hear is the
most difficult to do
00:53:27.808 --> 00:53:30.950
because level four gets to
00:53:30.950 --> 00:53:35.261
organizational results.
It measures the
00:53:35.261 --> 00:53:36.776
impact of the training on the
00:53:36.776 --> 00:53:41.194
organization as a whole.
And when we look
00:53:41.194 --> 00:53:45.316
at this, that one to
the left. You design
00:53:45.316 --> 00:53:51.002
the training and try
to tie in measures
00:53:51.002 --> 00:53:54.585
of key agency and human
resource metrics.
00:53:54.585 --> 00:53:58.153
And, you look at how
the results of the
00:53:58.153 --> 00:54:00.932
training tie in to
begin meeting the
00:54:00.932 --> 00:54:02.979
mission, the vision,
and the values of the
00:54:02.979 --> 00:54:06.182
agency and key hr
metrics, like looking
00:54:06.182 --> 00:54:10.568
at retention rates,
looking at recidivism
00:54:10.568 --> 00:54:12.456
rates for the agency, looking at
00:54:12.456 --> 00:54:15.651
other offender outcomes,
and other agency
00:54:15.651 --> 00:54:18.625
outcomes as a result
of your training.
00:54:19.263 --> 00:54:21.556
And then on the right,
a request for
00:54:21.556 --> 00:54:25.466
validation. And when you look at a
00:54:25.466 --> 00:54:28.495
request for a validation,
a request for
00:54:28.495 --> 00:54:31.334
validation might be a
new one to you. And
00:54:31.334 --> 00:54:33.531
this is a letter sent
to supervisors or
00:54:33.531 --> 00:54:36.138
managers asking for quantitative
00:54:36.138 --> 00:54:38.203
evidence that training
is being applied
00:54:38.203 --> 00:54:40.414
on the job. In other
words, it seeks
00:54:40.414 --> 00:54:42.376
evidence of results of the training
00:54:42.376 --> 00:54:45.301
being applied on the
job and impacting
00:54:45.301 --> 00:54:49.269
the organization as
a whole. And how do
00:54:49.269 --> 00:54:55.757
you measure level four for
your agency? How do
00:54:55.757 --> 00:54:58.285
you measure level four
results or impact
00:54:58.285 --> 00:55:01.337
of your training of
the organization?
00:55:07.662 --> 00:55:10.088
How is it happening for you?
00:55:12.467 --> 00:55:13.943
Oh, we've got all kinds of
00:55:13.943 --> 00:55:17.213
interesting things
popping up here. Oh,
00:55:17.213 --> 00:55:20.562
yeah. Very common is
follow up separate
00:55:20.562 --> 00:55:24.165
surveys months after the training.
00:55:25.690 --> 00:55:28.154
"Statistical data and
meeting required
00:55:28.154 --> 00:55:29.666
measurements." thanks, Denise.
00:55:34.908 --> 00:55:38.505
"Oral presentations,"
"Observation of staff,"
00:55:38.505 --> 00:55:45.814
"Supervisor audits".
Excellent. "After-action
00:55:45.814 --> 00:55:50.774
reviews," "Unannounced
evaluations and
00:55:50.774 --> 00:55:55.056
audits." Oh, there
we go Mike! You're
00:55:55.056 --> 00:55:58.567
talking about some
agency-level outcome
00:55:58.567 --> 00:56:00.648
measures. "Number of
diversions from court
00:56:00.648 --> 00:56:03.349
detentions." "Length of
supervision." "Client
00:56:03.349 --> 00:56:06.825
feedback." Tracking
stats from audits."
00:56:06.825 --> 00:56:12.793
Thanks, Tara. Sarah,
excuse me. Stanley, thanks.
00:56:12.793 --> 00:56:14.559
"Feedback from supervisors
and management."
00:56:14.559 --> 00:56:16.717
Great, that is fantastic.
00:56:17.440 --> 00:56:20.501
So, we do have... we're
going to see as a
00:56:20.501 --> 00:56:22.464
result of this webinar,
we're going to go
00:56:22.464 --> 00:56:25.944
back and peek in. We're
going to check in
00:56:25.944 --> 00:56:28.650
on our supervisor
and our CEO because
00:56:28.650 --> 00:56:31.234
they have been with us during this
00:56:31.234 --> 00:56:34.430
webinar. And see if
their conversation
00:56:34.430 --> 00:56:36.721
has changed as a result.
00:56:37.627 --> 00:56:39.207
[knock, knock, knock, knock, knock, knock]
00:56:41.010 --> 00:56:43.895
CEO: Bernie, if I see one more CO mess up
a pat-down search,
00:56:43.895 --> 00:56:45.096
I'm going to blow a
gasket.
00:56:46.151 --> 00:56:48.360
Training Director: Oh, this is a serious
issue for you.
00:56:48.888 --> 00:56:52.234
Obviously, correct pat searches
are a security issue.
00:56:53.856 --> 00:56:55.625
CEO: Bernie, we spend
thousands of dollars and
00:56:55.625 --> 00:56:56.812
eight weeks training these new
00:56:56.812 --> 00:56:58.243
corrections officers,
and they can't even
00:56:58.243 --> 00:57:00.267
do a simple pat-down
search correctly.
00:57:00.267 --> 00:57:03.273
Now, I'm not certain
this is a training
00:57:03.273 --> 00:57:05.433
issue, and I don't
want to overreact in
00:57:05.433 --> 00:57:08.867
this situation, but I...
and I know that we
00:57:08.867 --> 00:57:10.539
can just throw more
training at everyone,
00:57:10.539 --> 00:57:12.546
but sometimes that's not really the
00:57:12.546 --> 00:57:14.586
problem. I'd like you
to take a look at
00:57:14.586 --> 00:57:16.296
this issue and see if
you can figure out
00:57:16.296 --> 00:57:17.085
what's going on.
00:57:18.562 --> 00:57:21.262
So, if it is serious enough to invest some
staff, time, and
00:57:21.262 --> 00:57:21.609
money,
00:57:21.609 --> 00:57:23.151
we need to get this
right.
00:57:23.968 --> 00:57:26.632
CEO: Bernie, we have the responsibility
to ensure pat-down
00:57:26.632 --> 00:57:29.092
searches are done correctly. As the
00:57:29.092 --> 00:57:30.873
subject matter expert, what do you
00:57:30.873 --> 00:57:33.342
recommend we start
with, and how can I
00:57:33.342 --> 00:57:34.193
help support you?
00:57:35.412 --> 00:57:39.101
Training Director: Well, before we decide
on a solution, I recommend some analysis
00:57:39.101 --> 00:57:41.599
to identify the root
cause. We need to
00:57:41.599 --> 00:57:44.187
look for specific
performance issues,
00:57:44.187 --> 00:57:47.141
Once we've identified
00:57:47.141 --> 00:57:49.311
the root cause, we can design an
00:57:49.311 --> 00:57:51.396
effective solution
that we can evaluate.
00:57:51.396 --> 00:57:54.563
One way you can support
this process at
00:57:54.563 --> 00:57:56.492
the very beginning
is just give me some
00:57:56.492 --> 00:57:58.930
specifics on what
you're seeing with
00:57:58.930 --> 00:58:01.033
pat searches. We need to start with
00:58:01.033 --> 00:58:03.418
sitting down and going through what
00:58:03.418 --> 00:58:04.474
you're actually seeing.
00:58:05.857 --> 00:58:09.600
CEO: I understand that Bernie,
I'm seeing some specific issues
00:58:09.600 --> 00:58:12.077
as I walk around on
third shift with the
00:58:12.077 --> 00:58:13.491
new officers and they're taking
00:58:13.491 --> 00:58:15.325
shortcuts and not performing these
00:58:15.325 --> 00:58:17.233
searches the way that
we've instructed
00:58:17.233 --> 00:58:19.635
them to do that. But
of course, we need
00:58:19.635 --> 00:58:21.618
this done ASAP. How
long do you think
00:58:21.618 --> 00:58:23.875
this will take the
training team to do
00:58:23.875 --> 00:58:25.916
this analysis and bring me some
00:58:25.916 --> 00:58:27.126
potential solutions?
00:58:28.157 --> 00:58:32.236
Training Director: Well, give the training
team three weeks to analyze a
00:58:32.236 --> 00:58:34.063
problem and will recommend some
00:58:34.063 --> 00:58:36.964
solutions that will
get to the outcomes
00:58:36.964 --> 00:58:38.204
that you're looking for.
00:58:39.384 --> 00:58:41.938
CEO: Bernie, as always I thank you for your
leadership, your
00:58:41.938 --> 00:58:43.852
thoughtful approaches
to training, and
00:58:43.852 --> 00:58:45.601
ensuring that we're
meeting our mission.
00:58:45.601 --> 00:58:47.794
I look forward to
working with you as we
00:58:47.794 --> 00:58:49.679
correct this deficiency. Thank you.
00:58:50.140 --> 00:58:53.132
Leslie: All right, everybody.
What are you hearing
00:58:53.132 --> 00:58:56.175
in our roleplay
revisited between the
00:58:56.175 --> 00:58:58.815
CEO and training director? Has this
00:58:58.815 --> 00:59:04.413
webinar had any
impact? Chat us your
00:59:04.413 --> 00:59:07.945
ideas? What are you
hearing? Ah, "Better
00:59:07.945 --> 00:59:12.735
communication," says
Denise. "Sounds as if
00:59:12.735 --> 00:59:14.909
he's learned something,"
says James.
00:59:16.106 --> 00:59:19.119
"Management support!"
with an exclamation
00:59:19.119 --> 00:59:20.104
point," from Sarah.
00:59:21.411 --> 00:59:23.331
"Buy-in," says Yolanda.
00:59:23.950 --> 00:59:27.083
Oh yeah. "Not one size
fits all." "Systemic
00:59:27.083 --> 00:59:29.935
approach considered."
Oh yeah, Bernie with
00:59:29.935 --> 00:59:32.349
her managing up aproach.
Ah, he's listening
00:59:32.349 --> 00:59:33.411
to his...me!"
00:59:33.521 --> 00:59:36.774
"Problem-solving, more open input."
00:59:36.774 --> 00:59:40.488
Absolutely Kathleen.
Oh an "understanding
00:59:40.488 --> 00:59:43.355
of training needs," right
Marie. You got it.
00:59:43.574 --> 00:59:46.345
Absolutely. So a little bit of a
00:59:46.345 --> 00:59:50.721
different approach
there, and you know,
00:59:50.721 --> 00:59:54.556
sometimes you just have to do some
00:59:54.556 --> 00:59:59.382
groundwork layin, in
terms of that. So, we do
00:59:59.382 --> 01:00:01.603
have some time for Q & A. A couple of
01:00:01.603 --> 01:00:04.569
minutes. I did want
to open the floor
01:00:04.569 --> 01:00:08.616
from that. Linda I'm
not sure... the
01:00:08.626 --> 01:00:11.410
chat has been going
through fast and
01:00:11.410 --> 01:00:14.246
furious, but now, is
the time to chat any
01:00:14.246 --> 01:00:17.698
questions that you have
for your facilitators
01:00:17.698 --> 01:00:20.928
Jeff, myself, and Bernie. So, what
01:00:20.928 --> 01:00:23.194
questions do you
have? Linda, do you
01:00:23.194 --> 01:00:24.442
have a question?
01:00:24.603 --> 01:00:28.651
Bernie: Leslie, I did see somebody ask for
an explanation on
01:00:28.651 --> 01:00:29.932
what's a teach back.
01:00:29.932 --> 01:00:34.235
Leslie: Okay, great. And just to let you know, any
of those examples of
01:00:34.235 --> 01:00:38.072
evaluation methods
that I mentioned in
01:00:38.140 --> 01:00:41.391
Kirkpatrick, are
located on page 17 of
01:00:41.391 --> 01:00:45.302
the white paper, of
which, this webinar is
01:00:45.302 --> 01:00:48.708
based upon. But basically,
a teach back, or
01:00:48.708 --> 01:00:50.848
a presentation, or teach back is a
01:00:50.848 --> 01:00:54.353
presentation prepared
by the participant
01:00:54.353 --> 01:00:57.202
that is delivered to others to
01:00:57.202 --> 01:00:59.198
demonstrate the
employee's knowledge or
01:00:59.198 --> 01:01:01.292
skill. And this can
be done during the
01:01:01.292 --> 01:01:04.440
training, itself, or
to peers back on the
01:01:04.440 --> 01:01:09.380
job using some kind of checklist or
01:01:09.380 --> 01:01:11.506
evaluation form. And
of course, you would
01:01:11.506 --> 01:01:15.470
like to give feedback
from that. And so.
01:01:15.470 --> 01:01:19.571
Mike is asking more info and how
01:01:19.571 --> 01:01:22.091
other agencies use
teach backs. I know
01:01:22.091 --> 01:01:25.714
that we use teach
backs, not only in our
01:01:25.714 --> 01:01:28.004
learning and performance,
which are our
01:01:28.004 --> 01:01:30.131
trainer development
programs, but we also
01:01:30.131 --> 01:01:33.607
use them like in our
leadership programs.
01:01:33.607 --> 01:01:37.541
We use the peer feedback
delivery method
01:01:37.541 --> 01:01:40.736
as a way to actually
deliver some of the
01:01:40.736 --> 01:01:42.929
content in our leadership
programs. It's
01:01:42.929 --> 01:01:44.619
an instructional
strategy that we have
01:01:44.619 --> 01:01:46.809
designed into our
programs. That's how we
01:01:46.809 --> 01:01:49.791
at NIC use it. How
about others right out
01:01:49.791 --> 01:01:53.862
there on the webinar?
And Robin there was
01:01:53.862 --> 01:01:57.378
a link to the white paper in the
01:01:57.378 --> 01:01:59.978
advertisement for
this webinar.
01:02:04.375 --> 01:02:05.400
Other questions?
01:02:08.039 --> 01:02:11.383
The powerpoints will
be a part of the
01:02:11.383 --> 01:02:13.712
copy of this webinar.
01:02:16.178 --> 01:02:19.483
Bernie: And if I can address one, there was
a discussion going
01:02:19.483 --> 01:02:23.058
on about level one, with
the students... how
01:02:23.058 --> 01:02:25.224
they rate the class
by how fun the class
01:02:25.224 --> 01:02:27.552
is. Just so you know,
when we had our
01:02:27.552 --> 01:02:28.998
training administrators
network meeting
01:02:28.998 --> 01:02:33.130
last year, this was a
point of discussion
01:02:33.130 --> 01:02:35.064
when it came to evaluation. When we
01:02:35.064 --> 01:02:37.001
looked at... there were
some that found it
01:02:37.001 --> 01:02:38.961
useful, some that
didn't find it useful.
01:02:38.961 --> 01:02:42.076
Just so you know, that
even Kirkpatrick
01:02:42.076 --> 01:02:45.231
agency is looking at the one, two,
01:02:45.231 --> 01:02:47.391
three, four, and have
actually said that
01:02:47.391 --> 01:02:48.727
if they had to do it
over again, they'd
01:02:48.727 --> 01:02:52.184
flip it on its head.
That the numbering
01:02:52.184 --> 01:02:55.092
doesn't necessarily
mean value in that
01:02:55.092 --> 01:02:59.076
order, and that it
doesn't... and also that
01:02:59.076 --> 01:03:02.019
it's more based on
chronological, like,
01:03:02.019 --> 01:03:04.075
you know, students
reaction is right
01:03:04.075 --> 01:03:05.646
after the training,
right? Where the other
01:03:05.646 --> 01:03:07.940
stuff is at a later
date, and so they're
01:03:07.940 --> 01:03:10.693
looking at even changing
that around and
01:03:10.693 --> 01:03:13.163
also trying to help
people understand
01:03:13.163 --> 01:03:16.245
what is meant by levels
rather than that
01:03:16.245 --> 01:03:20.649
it's what what we
tend to think of. But,
01:03:20.649 --> 01:03:22.173
the discussions, just
so you know, that is
01:03:22.173 --> 01:03:24.489
a discussion that
goes on. And there is
01:03:24.489 --> 01:03:25.966
some of the stuff
on the Kirkpatrick
01:03:25.966 --> 01:03:27.793
website that you may
want to look at
01:03:27.793 --> 01:03:31.192
about that. Because there were some
01:03:31.192 --> 01:03:33.067
people who said, "Yes,
it's not useful at all."
01:03:33.067 --> 01:03:35.862
And it depends on what
you ask the class.
01:03:35.862 --> 01:03:38.098
It asks... it depends,
and then, also what
01:03:38.098 --> 01:03:39.586
you're trying to get.
What information
01:03:39.586 --> 01:03:40.499
are you trying to get?
01:03:41.436 --> 01:03:46.393
Leslie: I just posted the link to the white
paper that this is
01:03:46.393 --> 01:03:52.283
based upon in chat.
And I want to let you
01:03:52.283 --> 01:03:55.032
know as I mentioned
earlier on in the
01:03:55.032 --> 01:03:59.350
webinar, all NIC
webinars are recorded.
01:03:59.350 --> 01:04:00.473
As soon as we
01:04:00.473 --> 01:04:03.438
get this made 508
compliant, we will be
01:04:03.438 --> 01:04:06.713
actually posting this to the NIC
01:04:06.713 --> 01:04:08.961
website, and we will
post to the website
01:04:08.961 --> 01:04:12.515
when it is available. The actual
01:04:12.515 --> 01:04:14.597
recording of the
webinar will include
01:04:14.597 --> 01:04:15.536
the slides.
01:04:16.781 --> 01:04:21.919
So, that is all the time we have today. We do
thank you for being
01:04:21.919 --> 01:04:25.687
here with us today
and joining us in
01:04:25.687 --> 01:04:27.567
spite of the
difficulties that we had
01:04:27.567 --> 01:04:29.743
back on November the
15th. We thank you
01:04:29.743 --> 01:04:31.836
all for joining us
here today for this
01:04:31.836 --> 01:04:32.936
rescheduled webinar.
01:04:33.664 --> 01:04:36.517
These are selected
resources that we
01:04:36.517 --> 01:04:38.886
highly recommend, but
there's so much out
01:04:38.886 --> 01:04:42.844
there on training
evaluation. You can
01:04:42.844 --> 01:04:45.712
look at not only our
information center
01:04:45.712 --> 01:04:49.211
resources, but very
specifically, the
01:04:49.211 --> 01:04:52.992
OPM training evaluation field guide
01:04:52.992 --> 01:04:55.117
is just a wonderful
resource to have.
01:04:55.117 --> 01:04:58.640
It's available online
and that is in the
01:04:58.640 --> 01:05:02.728
resources link in the
white paper. Linda
01:05:02.728 --> 01:05:05.182
has posted for you
our names and the
01:05:05.182 --> 01:05:08.479
email addresses of
all of us who have
01:05:08.479 --> 01:05:12.672
been on here. And, if
you would like to
01:05:12.672 --> 01:05:15.336
contact any of us, if you have very
01:05:15.336 --> 01:05:17.688
specific questions,
I see there, Joshua
01:05:17.688 --> 01:05:20.309
that you have a very
specific question.
01:05:20.309 --> 01:05:22.376
We're not going to
be able to answer it
01:05:22.376 --> 01:05:26.390
here and right here
during the webinar,
01:05:26.390 --> 01:05:29.016
but please do email
us after it's over
01:05:29.016 --> 01:05:31.033
with or we'll try to
grab that question
01:05:31.033 --> 01:05:34.380
and get back to you.
On that, thank you
01:05:34.380 --> 01:05:37.599
all for being here
with us today. We
01:05:37.599 --> 01:05:41.788
really do appreciate
you being here, and
01:05:41.788 --> 01:05:45.545
you'll be getting an
email from us from
01:05:45.545 --> 01:05:48.791
the webex system on
how to retrieve your
01:05:48.791 --> 01:05:50.707
certificates when
those are available
01:05:50.707 --> 01:05:53.671
from the NIC Learning
Center. For those
01:05:53.671 --> 01:05:55.432
of you who completed
the webinar today
01:05:55.432 --> 01:05:57.164
and we really appreciate it.
01:05:57.164 --> 01:05:59.655
Have a wonderful
holiday season, and we
01:05:59.655 --> 01:06:02.063
hope to see you in
another nNIC webinar
01:06:02.063 --> 01:06:04.709
in the future. Thank
you, everybody.