To properly set up your taxes you must edit the Sub-Region rates in your Regions & Taxes area of your shop. You will find this inside the Preferences -> Regions & Taxes area of your shop.

The tax rate that is shown on the main Regions & Taxes page applies for the entire region, the Sub-Regions area are the taxes that are applied at state / provincial level. Any values in the Sub-Region area are added to the over-all Region Tax to get your total tax for the customer.

Currently you likely have sub-region taxes configured that are being added to the total for the state in question. You should make the proper adjustments to get the tax rate you are looking for. If you wish to not charge any taxes to customers out side of your state / province, then remove the main Region tax percentage and have only the Sub-Region tax for your state.