Should FedEx Have Walked Into My House To Deliver Package?

Sam had an odd experience with FedEx back in February. He writes that when he didn’t come to the door right away, instead of leaving the package on the porch, the deliveryman just opened the door and walked in. This confused Sam a bit, and he wrote to Consumerist asking what he should do.

I just moved into a house, and about two days later (Thursday, Feb.
25) I got the first package delivered to my house. The FedEx delivery
person knocked twice, waited about five seconds, then opened my door
and walked in. I did not let him in, I did not shout “Come in!” or
anything, he just walked in. I was so surprised I just signed for my
package and let him leave without saying anything. Putting aside such
things as “What if I was naked?” what was he doing? What if I wasn’t
there and had forgotten to lock my door? Would I have returned to find
I’d been burglarized, package left nicely on the coffee table? Or do
they assume if the door’s unlocked they can come in and drop the
package off?

Mainly, I’m wondering three things: has anyone else had this happen to
them, should I be worried about this in the future, and should I
report the delivery person to FedEx management?

Sam contacted Fedex customer service, and heard nothing back. A few weeks later, he sent another e-mail, and got this response:

Dear Sam:

Thank you for contacting FedEx. We apologize for any problems you
experienced as a result of this incident.

In an effort to avoid similar incidents in the future, we have advised
FedEx management of what occurred. They are reviewing the situation
and will take any necessary corrective action. Your reference number
is #########.

Again, please accept our apology. We appreciate your notifying us of
this problem, and will do our best to make sure your next experience
with FedEx is more satisfactory.

[redacted]
FedEx Customer Service

In other words, FedEx seems to be taking this incident very seriously. Is that enough? Does Sam even have a right to be upset here?