Is there anything I can do about a ex that is angry at me for moving on ?

My ex boyfriend and I were together for 18 years. Over the years it came to a realization that he cheated on me more than a few times, never helped pay the bills, always demanding, and never spent time with the kids. I broke up with him last year. He moved on and found a new girlfriend after a few months after the breakup, but I guess the relationship didn't last too long. For the sake if our son, I tried to keep our friendship on a good level, but every time I wa nice to him, he wanted to fondle and grope me, which turned me totally off. Once he found out I had a new boyfriend, he calls me conversating like normal, he starts to ay negative things about me. A best friend of me and my ex was concerned about me and told me that my ex said if he sees me on the street anywhere with my new boyfriend, he was going to hit us in the face. Im trying to move on in my life and my ex is bitter because I won't take him back. Should I be concerned about my ex behavior ? I have no proof that he wants to harm me, but I know his temper.