I've got a fully bilingual 5 year old, and honestly, you need to get a better balance between the two languages.
For us, we got very lucky that the balance of our family/environment just kind of worked in a pretty 50-50 way:
Overall/default family language = A
1 parent alone (+ family) language = A
1 parent alone (+ family) language = B
Country/school language = B
Extended (6+weeks) summer trips to other country = A
So at school and in general society, as well when alone with one parent, and their family, he speaks B.
But when together with both parents, on extended summer trips, and alone with one parent and their family he speaks A.
That splits the two big areas of their life (School/nursery/kindergarten) and Family, between the two languages, meaning both end up being essential, and both have developed to a fluent/normal level now.
I have a lot of friends who are in your situation. which is the same as ours, except the family language is flipped to B, so that it matches the country/school's language, and honestly - most of them (at this age range) are struggling, because it unbalances the situation, and isolates the second language to one single parent/circumstance. The two dominant parts of their life, school nursery/kindergarten, and their family, are all the same language... of course that will become the dominant language.
Given that this isn't possible for you, I would suggest that you just keep forcing it through the following approaches, until they are old enough to value the language:
1 - As much extended time with grandparents, without an English speaker to translate. This will force them to communicate in their second language.
2 - Make sure that when you visit your home country, they socialize with other children in environments that don't contain any English.
3 - Make sure you have plenty of alone time without your husband, and discourage answering in English (playfully)
4 - Teach him to read (maybe not yet) in your language, because research has shown that reading cements 2nd languages. If they just speak but never read, the chances of them loosing a 2nd language later in life are much much higher.
And the biggest:
5 - Get your husband to learn your language.
He doesn't have to become fluent... but the act of him learning it, is inspirational for your child, and means that you can have some small times when that language is used as your family language.