Yes. Also the fuse has an effect . I remember that happend to me too, specially when you have a fast burning BP or a small nozzle with a large fuse diameter.
I have a way of getting round this, but it is not usually neccesary with larger rockets. For the main part of the delay fuse use standard visco or blackmatch, then as the fuse enters the rocket you link some visquick (sometimes known as 'shogun fuse' or a number of other names). It burns away completely in the same way as normal visco, but burns at 2 seconds per foot which is fast enough for it to have almost vanished before the rocket produces enough pressure to launch (or CATO, which was the idea). Tested this a couple of times but since then just slightly slowed my fuel down and it is no longer needed.
I do not know many suppliers that stock fast visco, at least that will post to the UK. It seems every time it becomes available it has gone within a couple of days. Quickmatch is unsuitable due to the tube being left behind which would actually make the problem worse.