This means in the logical font "Dialog" are no Bengali characters. Which real font is use for the different characters are define in the fontconfig.properties of the Java VM.

You have 2 options:
You modify the fontconfig.properties and add a range for Bengali characters. This can e a little tricky.
Or you set font for you label that can show Bengali characters. The problem can be that you show some of the other characters wrong. There does not exist a font that include all characters. That there are the logical (composite) fonts that mix multiple real font to one font.

Thanks for the reply!
I explicitly set a font having Bengali characters. Now the Bengali characters are displayed properly, but other characters display boxes. This solves my issue for the time being though.

I was wondering why Bengali (the issue is also observed for Oriya) characters are not displayed properly, while Devanagari ones are shown without problems. In this age of multi-language computing, things should work out of the box. Is this a bug that needs to be reported to the Java developers?

You can post it to Oracle. But I worry your language is not important enough. The same problem has also some European languages like Georgian which not use Latin characters.

The core question is: Has your OS (Windows?) a default font for your language or must install it self. The file fontconfig.properties is a static file. That it used only default fonts from the different OS.