Bigamy Is BAD!
Yes, you do have to get a divorce before getting remarried. For some strange reason this is a frequently asked question. While I know the cost of a divorce can be high(uncontested from me is a whole gasp 651.50 currently including court costs), especially if contested. It is required that you get a divorce before getting remarried or it is a crime. It is still a crime if you have been separated for twenty years and still know the other is alive, you really dislike each other, or even if your new spouse is okay with it.
The only exception would be an absence of five years with a lack of knowledge of life. Then if the dearly departed resumes their place among the living they have a right to have the next marriage dissolved. So unless you’ve seen the body, I’d suggest not doing so. It really would suck to have your marriage annulled by a long dead spouse.