the one slice of cake or bread won't necessarily cause you to put on pounds of weight, but they will increase blood sugar levels, which will cause an insulin reaction, which will mean replenishing glycogen stores (which are depleted when you are in ketosis, and burning fat for fuel). the glycogen will then be used by the body for fuel before any fat. this will likely be followed by a blood sugar low, where you will likely crave carbs/sugar, and feel sluggish. you may also feel bloated and uncomfortable as a result of eating carbs
if you feel it's worth it, and are prepared to battle any cravings you might feel, then that's fine, but you have to go off-plan mindfully and aware that you might need to work hard to get back on track again. for a lot of people, carbs beget carbs, so a small deviation can easily lead to mindless eating. it's not just about weight gain, really
that said, there is a low carb diet plan (charles clark?) which allows a slice of wholemeal bread per day, so it's not necessarily incompatible. it's not something i would include myself, i have to say - i haven't eaten bread for nearly a year and don't miss it at all.
it's a while since I read briffa, and i stick to bootcamp rules so i can't quite remember what he allows, but for weight loss i would be careful with quantities of some root veg such as carrots and beetroot (it's worth checking carb counts for different vegetables, which you can find online), and legumes such as peas
the thing that briffa says which I think is really important, is that any way of eating needs, above all, to be sustainable, so if you're enjoying what you're eating on his plan and losing weight the way you want to and feel that it's a way of eating you can keep up long term (i.e. for the rest of your life with some deviations as needed), then that is absolutely right and the way it should be
if you feel you would like to join us for bootcamp, the rules are here, and there is always a thread in the big/slim/whatever weight loss club section (if you can't find it, then just PM one of us)
sorry for yet another mammoth post!