I think it is a lovely idea and I can see why you want to do it, but it will not make your grandson bilingual.
Even children who grow up with parents speaking different language usually end up with a "dominant" language - usually that of the country they live in and are educated in. A huge amount of what children learn after their first year or two gets picked up in the playground.
Consider this: mum is English, dad is German. Each parent speaks their own language to their child. They live in the UK and their child goes to school. In this scenario, the child will most likely speak German, but English will be their dominant language. This is particularly likely to be the case if the dad is not strict about speaking to the child in German all the time, or if the child for some reason decides that they don't want to speak German (which happens a surprising amount).
Now consider this alternative scenario: mum and dad are both English. They move to Germany for work. Prior to the move they spoke very little German. They enrol their child in a local school. What is likely to happen? The child will learn to speak German very quickly, probably with a near-native accent, and will soon be better than his parents.
This happened with my friend's daughter, who went to school in Spain. At five, she had no Spanish at all. At seven, she was translating for her mother. However, her written English is appalling because she has had no formal education in English even though it is her mother tongue.
Children's language development is mostly influenced by the environment they live in. If your grandson is not living in a French-speaking environment, he will not grow up bilingual in French, despite your best efforts. (My mum is a languages teacher and I didn't grow up speaking any of the languages she speaks.)
That's not to say it isn't worth bothering with. But for now I would just enjoy your grandson as he is and let him be a baby. Once he's confidently speaking English, you can start trying to teach him some French. There are lots of resources you will be able to use which are aimed at little kids. You can start with basic vocabulary and conversation, find kids' TV programmes for him to watch, when he starts confidently reading in English you can introduce him to basic children's books in French. Take him on holiday to France and say he can't have an ice cream unless he orders it himself in French. All this stuff will be an advantage to him, and will hopefully spark a love of language learning in him which will inspire him to actually go and learn foreign languages throughout his life.
But unless he has a parent with a first language other than English, or lives in a foreign country, he will not grow up bilingual.