These are financial centers that operate outside the taxation and investment boundaries of the major country economic powers. Running from small specialist countries, they offer tax advantages or different financial reporting obligations for clients, eg Cayman Islands, Channel Islands (UK).
You were born in one country but have been so successful you find yourself paying horrendous amounts of tax - and you aren't happy! Why not live somewhere else (for tax purposes) in another 'offshore' country which doesn't have these penalty rates of interest, and where your original nation state cannot get at your capital or interest? That's going 'offshore'.