The four “Middle Colonies”, New York, New Jersey, Pennsylvania, Delaware, were all initially colonized by countries other than England.
New York and New Jersey were settled by the Dutch, Delaware and Pennsylvania by the Swedes.
New Sweden, founded in 1638 with its base at what is now Wilmington, Delaware, was conquered by the Dutch in 1655.
The British conquered the Dutch colonies in 1664.