Prime Directive? This isn’t Star Trek. They’re Human. They’re directly related to and share the same air and resources as their neighbors.
Don’t do the rationalizing for the evil Wakandans. They are likely far worse than Hitler. Or in your case Khan.
This was literally the beef Killmonger had.
And W'kabi expresses the isolationist feeling pretty well in one line, "when you invite refugees, you also invite their problems". You can't be a Utopia if you try to fix everyone's problems.
But he is also willing to remove all their problems by conquering the world, expressed directly to T'Challa ("if you give me the order, I will go and conquer the world" or some such) and in his betrayal of T'Challa and alignment with Killmonger.
They both are ok with leaving Wakanda's isolationism in the past, but W'Kabi wants to seat Wakanda on the top of the world, helping the people who look like them by bringing everyone else low and T'Challa tells the previous Black Panthers they were wrong during his resurrection, then offers to share Wakanda's knowledge with the world and opening outreach centers in black communities.
But yes, prior to the events of this movie, Wakanda was a bunch of villains who let not only Africans suffer under Colonialism, but in modern times, let the world suffer to ravages of disease, cancer, famine, whatever else bad you can think of.