Yeah< I just realized that aside from the Main Four...everyone else that's mainly affiliated with the Avengers Group are SHIELD operatives or government agents (Sharon Carter, Maria Hill, BW, Hawkeye, Fury, Falcon....heck, even Ms. Marvel)

Are you talking about the movies, or the comic books?
Because we don't know for sure that Falcon, Ms. Marvel (who has yet to even be discussed by Marvel Studios), or even Sharon Carter will be agents.
And in the comic books, the VAST majority of the Avengers are independents. At least, before the Initiative made them all civil servants.