You have to escape the curly bracket {} characters. Also, this pattern is too fragile, verbose and redundant. It can be shortened to "\{(?:\s*.*){5}\s*\}" if you need to find items with exactly 5 elements. If you just want anything inside curly brackets, you can achieve the same result with "\{.*?\}" (make sure to replace the newline characters).
Furthermore, did you create this text file? If so, you are probably better off using the JSON library to create JSON and then parse it. It will be much easier and more robust. Your songgrouplist.txt file is not valid JSON but it's close enough that you should have just used JsonGenerator to create it and the JsonParser to parse it.