OK you will need something like DB Browser for SQLite ( https://sqlitebrowser.org/).
Once installed you need to open the songs database file. (Use the Tools -> Open Data Folder... menu option in OpenLP) The songs db is in the sub folder songs and named "songs.sqlite".
First you need to find the id of the song with the broken audio link in the songs table.
Then you need to find the record that corresponds to this broken audio link in the media_files table.
By deleting this record I'm hoping that will then allow you to edit the original song within OpenLP.
I'm not sure what version of OpenLP you are using but in 3.1.7 broken audio links are handled better although the message "Usupported File" is a bit confusing.