Arsenic is not good for you. No huge revelation there, but the naturally-occurring element contaminates groundwater in many parts of the world, including the US, causing all kinds of health problems (neurotoxicity, various cancers, cardiovascular disease, etc).
If you get your water from a city system, it is tested for arsenic and must meet federal regulations from the Environmental Protection Agency (no more than 10 parts per billion since 2001), but if you have a private well, in most places it's up to you to get it tested and then decide if you want to further treat your water for arsenic.
The Center for Public Integrity has put together an interactive map using data from the U.S. Geological Survey. The map is based on arsenic readings from 45,000 wells taken over decades, but don't be fooled by the big clusters; some areas just have more readings than others - for example, Texas and Minnesota provided data gathered on arsenic in private wells, which is why they have denser data than most states - so it's not because your corner of the country looks clear that you shouldn't do a water test just in case.
Note: The map above might take a moment to fully load, as there are a lot of data points.
You can use the search feature in the top right of the map to look up your city more quickly.